The goal of this assignment is to merge a source image with a target image in a more seamless manner than simple intensity copy-paste. To do this, we need to solve a minimization problem where we minimize the difference (gradients) between pixels that we map from the source to the target image. We break this into two problems.
First, we solve a toy version of the problem, where we map a grayscale image to a black image. From this, all we need to do is map the change between each image, and since we are mapping it to a black image, it results in a simple intensity copy-paste. To accomplish this, we need to minimize 3 objective functions.
We were instructed to solve these minimization problems by formulating the
problem as a least squares problem and then using a linear algebra solver to
get a solution. However, our matrices are huge and highly sparse because we only
solve the difference between two pixels at a time. For this reason, my laptop
(very old) is unable to process large matrices and crashes. To augment this, I
use scipy's scipy.sparse.csr_array() function to produce a compressed sparse row
matrix. I then use scipy.sparse.linalg.lsqr(a, b) to solve the linear
system.
After verifying the success of the toy problem, we move on to more complex problems, specifically, colored (3 channel) images, source/target images, and masking. Now, we need to rerun the Poisson blending for each channel. We also need to mask and map the target and source images to correctly overlay one another. Lastly, we need to adjust our minimization problem to account for the edges of the mask where the boundary is defined in the target image. As a result, our minimization objective function changes.
The main difference here is that at the boundary, we have a target pixel that our gradients should be adjusted to match. Since gradients don't care about the starting pixel, we can add any constant we want to get the gradients to match the boundaries. Then, we must "stretch" our gradients in a manner that fits the target boundaries. We accomplish this by adding these equations to our linear system and solving for this as well.
After this, I implemented iterative "bells and whistles" improvements:
The toy problem is fairly simple. I minimize the distance between the gradients.
My favorite blending result leveraged the fact that water is the perfect environment for Poisson blending. The water has little gradients and is mostly uniform in color. In addition, there is a lot of open space for me to work with and a lot of interesting images to work with. In particular, I was inspired by an art trend themed around the feeling of fear of the vastness of the ocean, specifically, what may lie in that vastness.
To accomplish this, I found a normal diving photo.
Then I threw in a giant sea monster I found.
I think the final result came out looking pretty cool. I would show a larger version, but my laptop crashes when it tries to create any matrix with dimensions larger than a few hundred.
This was fun.
For negative examples, what I found was important was to have high frequency details in the background. The gradients are calculated using only the pixels on the boundaries of the image. If the background is highly variable, then the gradients will try to match this highly variable gradients, which in actuality just messes up the internal coloring, leaving a distorted look.
In addition, if the cropping is done sloppily, with many details left in, they will not naturally align with the background's gradients, which causes a strange transition.
I didn't have many difficulties while working on this. In fact, I had to actively think and search for bad examples.
I just threw in the extra if statement that was provided in the instructions and it works great. Then, I just typed up a quote in a nice font and pasted on a picture of a road.
Looking at the second channel of hsv, which I believe is saturation, we can see that 35 stands out strongly.
We can use this as the source image that we will blend into the grayscale image.
