Assignment #2 - Gradient Domain Fusion

Description

Toy Problem

Problem Formation

In this toy problem, we will learn to translate the gradient constraints into linear equations, and use least square optimization to find the best solution.

 

When given pixel intensity of a source image S, we want to reconstruct an image V using the gradients from s, plus one pixel value. Denote the intensity of S at (x,y) as s(x,y), and similarly V at (x,y) as V(x,y).

 

For each pixel, we will have two gradient-based objectives:

1. \( argmin_v((v(x+1, y) - v(x, y)) - (s(x+1, y) - s(x,y)))^2 \)

2. \( argmin_v((v(x, y+1) - v(x, y)) - (s(x, y+1) - s(x,y)))^2 \)

 

And additionally, we add one pixel value constraint:

3. \( argmin_v((v(1, 1) - s(1, 1)))^2 \)

 

To translate this in to a least-square optimization problem, we want to reshape V into a vector x, and construct the constraints as \( (Ax - b)^2 \), where A denotes the coefficients of v(x,y) in the objectives, and b corresponds to all terms non-related to v(x,y).

 

 

Results

The result showed that we can successfully reconstruct the original image using the three constraints mentioned above. Next we will build our Poisson Blending algorithm based on this toy example.

Poisson Blending

Problem Formation

In Poisson Blending, we have a source image \( s \), and a target image \( t \). And the goal is to find the best value \( v \) with the following constraint: where \( N_i \) denotes the 4-neighbor of pixel i, \( S \) denotes the region of the source image. In this project, we also have a helper mask \( m \), which we will apply onto the original source image to get the final source image we will be using for Poisson Blending.

 

 

Results

Additional Blending Results

Favorite Results

Here I will present some of the successful results.

 

1. Penguin and Snow

2. Polar Bear and Snow

3. Airplane and Jungle

4. Brown Bear and Jungle

Failure Cases

Here I will present some of the failure cases. I observe that when the background color is significantly different from the colors of the source images, the blended source image will have colors more similar to the background and will diverge from it's original color. This problem especially exists for source images with lighted, less saturated colors.

 

1. Surfer and Beach

The color of the surfer's skin changes significantly.

2. Brown Bear and Snow

The color of the brown bear also changes significantly, more blended into the white background.

3. Polar Bear and Jungle

In this example, the color of the white polar bear almost turns into green.

Mixed Gradient

Algorithm Description

In Poisson Blending, we only use the gradient of the source image as constraints. In the mixed-gradient version, we will compare the gradient of the source image with that of the target image at each (x,y), and use the larger gradient to form the constraints.

 

Here \( d_{ij} = max(abs(s_i - s_j), abs(t_i - t_j)) \)

 

 

Results

We can notice that the mixed-gradient result better preserve the details of the target background compared to the original Poisson Blending result.

Color2gray

We explore an alternative to rgb2gray using gradient-based blending algorithm, first transforming the RGB images into HSV images, and then use the S channel as the source image, the V channel as the target image and perform mixed-gradient Poisson Blending.

 

 

Results