**Learning-Based Image Synthesis** **Assignment 2** - Gradient Domain Fusion Overview ============================================================== The goal of this project is to blend a source image into a target image seamlessly. The naive way of just copying the pixels from the source image to the target image will look unrealistic and will have noticeable seams. To combat this, we make use of the gradients in the image rather than its intensity. The idea behind this is that the gradient of the image is more noticeable to humans than the intensity. Toy Problem ============================================================== Just to understand the mechanics behind poisson blending, we do this toy problem first where we try to reconstruct an image using its gradients. We solve this using a least squares solution.  Poisson Blending ============================================================== Given a source image s and a target image t, poisson blending approaches this as a problem where we solve for a least squares solution as shown below. Here we solve for v which is the new pixel values in the source mask region inside the target. The first part of this equation forces the gradients inside the merged region to be similar as gradients in the source image merged region and the second part is forcing the values of the pixels at the boundary of the merged region to be the same as the target image. We solve the least square problem by constructing the matrices A, v and b and solving it in the form of (A*v - b)^2 = 0. We use lil_matrix and lsqr from the scipy library. Shown below are some results. | Source image | Target image | Naive blending vs Poisson blending | |----------|---------|---------| | | | | | | | | | | | | | | | | Failure case -------------------------------------------------------------- Due to drastic difference in the color and the texture of the source and the target images, the blending is changing the colour of the merged region. | Source image | Target image | Naive blending vs Poisson blending | |----------|---------|---------| | | | | Bells & Whistles ============================================================== Mixed Gradients -------------------------------------------------------------- To combat with the failure cases and limitations of Poisson blending, we consider the maximum difference among the source and target image gradients and give that as the goal in the least squares solution. In the examples shown below, we can see that the letters CMU are seamlessly blended into the wall with mixed blending compared to poisson blending. | Source image | Target image | Naive blending vs Poisson blending | |----------|---------|---------| | | | | | Source image | Target image | Naive blending vs Mixed blending | |----------|---------|---------| | | | | Color2Gray -------------------------------------------------------------- We can use mixed gradients method to convert RGB images into grayscale without losing the contrast information. To do this, first we convert the RGB into HSV colour space and use the saturation channel as the source and value channel as the target image. From the results below, it can be seen that mixed gradients method preserves contrast compared to naive grayscale conversion method. | Source image | Naive way vs Mixed gradients | |----------|---------|---------| | | | Source image | Naive way vs Mixed gradients | |----------|---------|---------| | |