16-726 Assignment #2 - Gradient Domain Fusion

by Zi Wang (ziwang2)

Overview

This project aims to implement some algorithms based on gradient to blend a source image into a target image seamlessly. I solved the questions in Part 1.1, Part 1.2, Mixed Gradients and Color2Gray in Bells & Whistles.

Part 1.1 Toy Problem (20 pts)

The goal of this task is to generate an output image closely resembling the original image through addressing a least squares challenge. The matrix A is to encompass specific constraints: the differences in the x and y gradients of the output image relative to the original should be minimal, and the pixel value at the (0,0) position in the output image ought to approximate that of the original. Given that each constraint involves merely two non-zero variables, a sparse matrix approach for solving least squares was employed.

Part 1.2 Poisson Blending (60 pts)

Source Image	Target Image	Blending Result

This technique determines the output intensities, denoted as v, based on the intensities from both the source image, s, and the target image, t. It tackles a least squares challenge aimed at optimizing a specific objective function. The constraints encapsulated in Matrix A ensure that the variance in output intensities closely aligns with that of the source image within its region, and that the discrepancy in output intensities relative to the target's mirrors the difference observed in the source image, particularly along the edges. For this optimization issue, a sparse matrix approach to least squares was utilized. Additionally, the optimization was performed individually for each of the RGB channels.

My favorite one is the blended image in the second row. Even though there are some noises around the edges, the shadows and the colors align pretty well with the background. This is because the backgrounds of the source image and the target image have similar colors. Then the source image don't have to change colors a lot to be inserted into the target image. Also, there was a shadow in the original source image, which increase the photo-realism of the blended image.

Noting that the phone in the last row presents weird color, which is a failure case. This is because the backgrounds of the source image and the target image have totally different colors (one is black and the other is light purple). The algorithm will change the color a lot to match the background in the target image.

Bells & Whistles (Extra Points)

Mixed Gradients (5 pts)

I employed the Mixed Gradient technique, which is similar to Poisson blending but selects the maximum gradient of the source and target pixels. This Mixed Gradient approach enhances the seamlessness and quality of the outcome, particularly in scenarios where the overlay image must seamlessly match the background's texture or color. From the visualization below, we see that Mixed Gradients is suitable for keeping the texture of the background.

Using Possion blending:

Using Mixed Gradients:

Color2Gray (2 pts)

The approach can be broken down into several steps:

1.Convert the RGB image to grayscale using a standard methodto maintain intensity information.

2.Convert the RGB image to HSV to analyze the gradients in the color channels, particularly focusing on the Value channel for intensity information and the Saturation channel for contrast information.

3.Compute gradients in both the grayscale image and the Value channel of the HSV image. 4.Mix the gradients from the grayscale and the Value channel to enhance contrast while preserving the overall intensity structure. 5.Reconstruct the grayscale image from the mixed gradients to ensure it has a similar intensity to the standard grayscale but with enhanced contrast that resembles the original color image.

Here are some visualization results. We see that the numbers in the pictures are more recognizable after processing.