Assignment #2 - Gradient Domain Fusion

---

Brief Description

The primary goal of this assignment is to seamlessly blend an object or texture from a source image into a target image. The main objective of this assignment is to seamlessly integrate an object or texture from a source image into a target image.The challenge is to eliminate these seams without causing significant perceptual damage to the source region.

The approach in this assignment revolves around a key insight: people are generally more concerned with the gradient of an image than its overall intensity. Thus, we frame the problem as finding target pixel values that maximally preserve the gradient of the source region while leaving the background pixels unchanged.

Part 1

To achieve this, we formulate our objective as a least squares problem. Given the pixel intensities of the source image "s" and the target image "t," our goal is to determine new intensity values "v" within the source region "S". This method is called Poisson blending.

Part 1.1 Toy Problem (20 pts)

In this example, I tried to compute the x and y gradients from an image s, then use all the gradients, plus one pixel intensity, to reconstruct an image v. For each pixel in the image, I need to minimize the difference between the x-gradient and the y-gradient of image s and of image v. Besides, I should also minimize the top left pixel of two images.

!python proj2_starter.py -q toy

The following figure shows the original image s and the reconstructed image v. They are almost the same.

Part 1.2 Poisson Blending (60 pts)

Follow the instructions on the website to implement Poisson blending.

Step 1: Select source and target regions.

Select the boundaries of a region in the source image and specify a location in the target image where it should be blended.

Then, transform (e.g., translate) the source image so that indices of pixels in the source and target regions correspond.

Step 2: Solve the blending constraints.

Step 3: Copy the solved values v_i into your target image.

For RGB images, process each channel separately. Show at least three results of Poisson blending. Explain any failure cases (e.g., weird colors, blurred boundaries, etc.).

!python proj2_starter.py -q blend -s data/source_01_newsource.png -t data/target_01.jpg -m data/target_01_mask.png

The following figure shows the source and target images, the blended image with the source pixels directly copied into the target region, and the final blend result.

More results:

1. the source and target road image

!python proj2_starter.py -q blend -s ./data/dolphin_newsource.png -t ./data/road.png -m ./data/road_mask.png

• the blended image with the source pixels directly copied into the target region; (left)

• the final blend result. (right)

How it works: The source image is a dolphin, and the target image is a road. The source pixels are directly copied into the target region, and the final blend result is shown on the right. The dolphin is seamlessly blended into the road.

Although in this cse, the dolphin seems well blended into the road, but the color of the dolphin is not the same as the original one.

This is becaues the poisson blending method only considers the gradient of the source image, and the color of the source image is not considered. It makes the inplanted dolphin more like a part of the road. The blue dophin becomes a yellow one, and the background is also changed to fit the road.

2. the source and target lake image

!python proj2_starter.py -q blend -s ./data/dolphin_copy_newsource.png -t ./data/lake.png -m ./data/lake_mask.png

• the blended image with the source pixels directly copied into the target region; (left)

• the final blend result. (right)

When the target image is lake instead of road, the dolphin is more like to keep its original color and the background is also changed to fit the lake background.

Failure cases:

The figure above looks not good, the boundary is not smooth, and the color is not consistent. The reason is that the source and target images have different colors and the boundary is not clear. The source image is a dolphin, and the target image is the lake and the sky. The dolphin blockes the hill behind the lake.

Possible reasons for bad results:

The boundry of the source image and target image has a big difference in color and intensity.

Bells & Whistles (Extra Points)

Mixed Gradients (5 pts)

source image

The following figure shows the implementation of mixed gradients:

This one just put the dolphin into the white background:

If use poisson blending, the result will still be a white picture.

!python proj2_starter.py -q mixed -s ./data/shirt_newsource.png -t ./data/road.png -m ./data/road_mask.png

!python proj2_starter.py -q mixed -s ./data/dolphin_copy_newsource.png -t ./data/lake.png -m ./data/lake_mask.png

From this figure, we can see that compared to poisson blending, the mixed gradients keep the features that change more significantly. The blank space from the source image is filled with the background of the target image.

Color2Gray (2 pts)

Sometimes, in converting a color image to grayscale (e.g., when printing to a laser printer), we lose the important contrast information, making the image difficult to understand. For example, compare the color version of the image on right with its grayscale version produced by MATLAB’s rgb2gray.

1. convert the rgb image to HSV color space, because value saturation will be more sensitive to human eyes than the hue channel.
2. convert the saturation channel to grayscale.

!python proj2_starter.py -q color2gray -s ./data/colorBlindTest35.png