In this assignment, I first implemented a toy reconstruction problem to test my code for computing image gradients and solving the optimization as a least squares problem (Section 1). Then, I implemented Poisson blending (Perez et al. 2003) to blend two images together while using the source image's x and y gradients as a guide (Section 2). Finally, I implemented mixed gradients, which compares the source and target images' gradients and uses the ones with the larger magnitude during Poisson blending to improve the blending result (Section 3).
For the toy reconstruction problem, I computed the x and y gradients from an image s, then used all the gradients, plus one pixel intensity, to reconstruct the image as v.
Then I solved the optimization problem as defined from the assignment, which is the sum of the 3 following objectives:
To solve the 3 objectives, I constructed a sparse matrix A with image_width * image_height * 2 - 1
rows (equations) and image_width * image_height columns (variables).
I implemented Poisson blending by constructing and solving the Poisson blending optimization objective for each
image channel:
I then constructed a vector b with the same number of elements as the number of rows in A. The values of b are
the gradients of the source image, except for the last row, which is the pixel intensity of the source image at
(0,0).
I looped through each (x, y) pixel coordinate and added 2 equations to the matrix for each pixel - one equation
for the x gradient, the other for the y gradient.
I then used the scipy.sparse.linalg.lsqr function to solve the least squares problem.
The reconstruction result is shown below:
I implemented Poisson blending by constructing and solving the Poisson blending optimization objective for each image channel:
For each channel, I construct the sparse matrix A and vector b like I did in the toy reconstruction problem, except I only iterated through the (x, y) pixel coordinates within the masked region.I used the source image's x and y gradients as a guide for the blending. For coordinates outside the masked region, I let the pixel intensity of the target image be the pixel intensity of the result image.
I tested my implementation on three examples: "Penguin", "Bird", and "Flowers". The results are shown below.
I obtained the bird source and target images, as well as the flower target image, from this website.
The remaining images were obtained from Google Images. I used masking_code.py to process all of the
images and generate the masks before running Poisson blending.
Here is an example where Poisson blending didn't work so well, as the edges between the source and target image are still a bit visible. This is likely because both the source and target images had lots of edges and textures, and therefore many gradients with large magnitudes, so the least-squares solution could not minimize the distance between the gradients as much. Also, because only the source gradients were used, the resulting blend has a reddish tint from the target image's pixel values.
I also implemented mixed gradients by comparing the source and target images' gradients and using the ones with the larger magnitude during Poisson blending to improve the blending result.
I tested my implementation on the "Flowers" example. Using mixed gradients results in a slight improvement compared to plain Poisson blending, as the the resulting blend has less of a reddish tint from the target image. This is likely because the target image's gradients were also used as a guide sometimes, so the resulting blend is more balanced. The result is shown below.
