_________________________________________________________

Homework 2 - Gradient Domain Fusion

Introduction

The objective of this assignment is to seamlessly integrate a specific area or object from a source image with a target background image, achieving a natural-looking composite. A simplistic approach might involve directly overlaying the two images, but this often leads to noticeable artifacts, such as evident boundaries between the merged sections and discrepancies in shading and lighting conditions. Our methodology is designed to address these challenges.

Our strategy begins with a preliminary exercise using a toy problem, which employs a similar approach in constructing a matrix to resolve constraint issues. Subsequently, we delve into the core technique of our project: employing Poisson Blending to amalgamate the two images. While effective, this method encounters difficulties in certain scenarios.

Toy Problem

In this challenge, we aim to compute the x and y gradients of an image, denoted as 's', and then leverage these gradients along with the intensity of a single pixel to reconstruct another image, denoted as 'v'. To accomplish this, we propose the following objectives:

1. Minimize \((( v(x+1,y)-v(x,y)) - (s(x+1,y)-s(x,y)) )^2\), ensuring that the x-gradients of 'v' closely align with those of 's'.

2. Minimize \((( v(x,y+1)-v(x,y)) - (s(x,y+1)-s(x,y)) )^2\), ensuring that the y-gradients of 'v' closely align with those of 's'. It is important to note that solving for these gradients allows the addition of any constant value to 'v'. To address this, we introduce an additional objective:

3. Minimize \((v(1,1)-s(1,1))^2\), aiming for the top-left pixels of both images 'v' and 's' to match in color.

Matrix Construction, Least Squares Problem

To simplify the process, we'll approach the least squares problems through matrix operations. We represent the image we aim to generate as a vector 'v', having dimensions (imh*imw, 1).

The constraints are formulated using a sparse matrix 'A' and a vector 'b', with the goal of solving the equation Av - b = 0.

Given that there are (2*imh*imw - imh - imw + 1) constraints, the matrix 'A' will have dimensions (2*imh*imw - imh - imw + 1, imh*imw), and 'b' will be dimensioned (2*imh*imw - imh - imw + 1, 1).

Each row in matrix 'A' represents a constraint. To align with our objectives, we selectively assign some elements in 'A' to be 1, while the rest remain 0, thus constructing a sparse matrix 'A'.

Similarly, each element in vector 'b' corresponds to the constant term of its respective constraint in matrix 'A'.

Ultimately, solving these constraints will yield the image vector 'v'.

Results - Toy Problem

Poisson Blending

Having established a method for representing constraints using matrices, we can now proceed to explore the Poisson Blending technique.

This technique involves working with a source image (referred to as 'fg') and a target image (referred to as 'bg'), alongside a supporting mask.

The images consist of three channels, necessitating the extraction of values for each channel. The objective function for Poisson Blending is illustrated below:

Within the area defined by the mask, we will apply four constraints for each pixel.

By adopting a matrix construction method analogous to that used in the Toy Problem, we can achieve the desired blending results.

Results - Poission Blending

Source	Target

Source	Target

Nevertheless, there are instances where the technique does not yield perfect results. A notable example is depicted in the second image section above, where the skin color of the girl on the right appears unnatural. This discrepancy arises from the complexity of the target image's background coupled with the intricate boundaries of the source image. Such complexities can lead to alterations in the overall color when attempting to find the optimal solution.