16-726 SP22 Assignment #2 - Gradient Domain Fusion Report

Overview

• Toy problem

• Poisson Blending

• Mixed Gradient

• Color2gray

Run

First, place source and target image in data folder. Modified filename in masking_code.py and run masking_code.py to select region you want from source and apply it on the target image.

xxxxxxxxxx
# Change filename in the code bottom two lines
python masking_code.py

and move the generated three pictures into data folder. Then in terminal:

xxxxxxxxxx
# Toy problem
python main.py -q toy
# Poisson Blending
python main.py -q blend -s data/<source_image_name>_newsource.png -t data/<target_image_name>.jpg -m data/<target_image_name>_mask.png
# Mixed Gradiant
python main.py -q mixed -s data/<source_image_name>_newsource.png -t data/<target_image_name>.jpg -m data/<target_image_name>_mask.png

About this homework

Description(20pts)

This project is about gradient-domain processing which is used to make more smooth image blending.

Intuitively, the gradient from source images should be applied to the same position in the target pictures as well as possible. We need to get the pixel value by solving linear equation. The constraints comes from the pixel around. There are two conditions we need to consider. The first one is the neighbor pixel is also a variable. The second one is that the pixel is known and has value in it. After getting the value, we reshape it into a 2D array and later get the result.

The important part is to make the correct understanding of how linear algebra solves this problem. The solving process goes down to solve Av-b = 0. A is a [e, h*w] matrix and b is a [e,1] matrix so that v is a [h*w,1] matrix. After reshaping we get the desired image.

Also, using sparse matrix will significantly increse the speed of result calculation from about half an hour to seconds.  

 

Toy problem(20pts)

Toy problem help us get familiar with the linear algebra needed for Poisson Blending. For this part, the result should look the same as input. Here is my result.

 

Poisson Blending(60 pts)

My favorite blending result(35pts)

Thanks for Jackie Chan and an orange. I like this because it is humorous and it also reflects good comparison between the Poisson Blend and Mixed gradient Methods. I will talk more about this in Mixed blend part.

For details, we iterate through the whole picture and see whether it is in mask or not. If it is a masked pixel, then we will see the neighbor is masked or not. Treat this problem similiar with the toy problem, but the constraints from border pixels. Then we reshape the solution into a h*w image, and apply it only on masked picture in target picture.  

For Poisson blending:

Several more results(25pts)

Another good example Little penguin and people who is skiing

A sunset scene:

 

I have not good results as well. For example, no matter which of the cat or the mouse to be the source image, It still looks weird in the result. The color does not look well. the mouse seems very stiff in result.

The reasons could be that: First, the background color distribution has huge difference in the two pictures. The cat one is green and the mouse one is gray, so the gradient could not be matched well in each color channel. Second, the size of the two images does not match enough and each single picture has too many pixels. The target and source image resolution should be relatively small to make better solution for the minimize problem(less than 500px*500px is great). Third, the mask of source image is too small so the gradient with background cannot be identified well.

 

 

 

Bells and whistles

Mixed gradients(5pts)

It is not too complicated. I just let substitute b for some new value correlated with Dij. A function is implemented to compare absolute value of two input and return the one with larger absolute value.

The result is significantly better than just Poisson blending.

Compare two blending methods:

The texture of orange skin is considered in mixed blend approach.

The blending is more smooth and clear and you can hardly notice the edge in mixed blend approach.

The color is more smooth than the Poisson blend one.  

 Homepage