Assignment 2¶
- Sihan Liu (sihanliu@andrew.cmu.edu)
0. Project Overview¶
In this assignment, given an area of the source image and target image, we need to combine them together into a single image in a very nature looking. A very naive idea is to directly stack them together, however, this causes a lot of obvious artifacts. For example, one can see the clear boundary between two crop images and also the shading and light environment mis-matching. So the following steps aim to solve this issue. Firstly, we will try our approach using a toy problem, since the toy problem shares the similar methodology to construc a matrix to solve the constraint problems. Then, we dive into our core part, using Poisson Blendin to merge two images. This method works quite well, but still has some cases that are hard to deal with. Lastly, I also implement the method using Maxied Gradients to fuse two images. This method works really well to preserve the texture of backgroung images. For Bells & Whistles, I also got a better implementation for Color2Gray.
1.1 Toy problem¶
To run the task:
python proj2_starter.py -q toy
Result:
1.2 Poisson Blending¶
To run the task:
python proj2_starter.py -q blend -s data/source_01_newsource.png -t data/target_01.jpg -m data/target_01_mask.png
Mixed Gradient Blending (Extra Credit)¶
To run the task:
python proj2_starter.py -q mixed -s data/source_01_newsource.png -t data/target_01.jpg -m data/target_01_mask.png
Blending Results:¶
How it works:
Firstly, we build a sparse.lil_matrix matrix of A by setting the four neighbors of one pixel to -1 and the pixel to 4 (plus 1 for each neighbor).
Then, we build vector b for each channel of the image. For each equation of b, we compute the value of one pixel by adding four times the foreground intensity of the pixel index and subtracting the foreground intensities of the pixel's four neighbors.
For pixel outside the source image, add the background intensity of the pixel to the corresponding equation as well.
By finding the least-squares solution with linalg.lsqr, we obtain the blended image.
| Source | Target | Result (Poisson) | Result (Mixed) |
|---|---|---|---|
 |
 |
 |
 |
More Visualizations¶
| Source | Target | Result (Poisson) | Result (Mixed) |
|---|---|---|---|
|
 |
 |
 |
Failure case - source image color distortion:
| Source | Target | Result (Poisson) | Result (Mixed) |
|---|---|---|---|
 |
 |
 |
 |
Reason of failure case:
- Poisson Blending prefers to process two images with similar color contrast, since the algorithm processes the whole source image influenced by its boundary color. If the boundary color is obviously different from the source object color, the converted source image may be distorted when blending.
Color2Gray¶
The way to do this is to first conver the rgb image to hsv image, and use s image as teh source image and v as the target image, and we also has a mask. Here is the result:
