Assignment #1 - Colorizing the Prokudin-Gorskii Photo Collection
1. Project Overview
I will introduce my method in this section. I got image alignment results based on three tricks: image cropping, edge detection, increasing bounding box size.
For the first one image cropping, I found that if we can cut additional edges of those images before conducting image RGB alignment, we can get a much better alignment score based on that cropped image since it reduces a lot of noise during the alignment.
For the second one edge detection, I found that it is hard to align R channel with B channel in one example because R channel has quite different distribution with B channel, thus making it hard to be aligned. Therefore, an alternative solution is that I implemented simple edge detection method based on convolution to detect the edges of the images and conducted the alignment afterwards.
For the third one increasing bounding box size, I found that increasing bounding box size can dramatically improve the quality of the image. Vectorizing convolution operation can speed up the whole process quickly.
2. Method
a. Small-size Colorizing
We use two metrics Squared Differences (SSD) distance and normalized cross-correlation (NCC) to select our best alignment between multiple channels. Two formulas for two metrics are below:
Empirically, I found that SSD and NCC works almost the same for the task and we only consider using SSD in my work.
We use cathedral.jpg as an example, when we select 30 as the bound size, cropping 10% boundry of the image, we can get a perfect alignment between RGB channels.
| Photo | Displacement for R channel | Displacement for G channel |
|---|---|---|
| cathedral.jpg | [12, 3] | [5, 2] |
b. Large-size Colorizing
For super large photos, the limitation of large bound would appear and it can be super slow to run experiments on these images. Therefore, we must use a paramaid-based method to accelerate this process. For each upper layer of the paramaid, its data is collected via gaussian noise addition and downsampling. For each photo that is larger than 64 pixels, we build a paramaid, each time the paramaid is the half size (1/2 width, 1/2 length) compared with the last layer. For each layer, we would have a [-30 ,30] displacement search.
Here is the result for each required photo:
| Photo | Displacement for R channel | Displacement for G channel |
|---|---|---|
| Emir | [107, 40] | [49,24] |
| harvesters | [124, 14] | [60, 18] |
| icon | [90, 23] | [41, 17] |
| lady | [120, 13] | [57, 9] |
| self_portrait | [175, 37] | [78, 29] |
| three_generation | [111, 9] | [54, 12] |
| train | [85, 29] | [42, 2] |
| turkmen | [117, 28] | [57, 22] |
| village | [137, 21] | [64, 10] |
Emir
harvesters
icon
Lady
self_portrait
three_generation
train
turkmen
village
| Photo | Displacement for R channel | Displacement for G channel |
|---|---|---|
| self-1 | [35, 37] | [12, 18] |
| self-2 | [63, 15] | [27, 12] |
| self-3 | [60, 29] | [21, 22] |
| self-4 | [132, -44] | [ 61, -19] |
| Self-5 | [13, 15] | [-13, 9] |
self-1
self-2
self-3
self-4
self-5
2.Bells & Whistles
I self-checked utillizing gradient-based robert score detection instead of the original picture to conduct the alignment. The gradient-based method helps different RGB channels like that in emir to align much better.
Before gradient-based edge detection:
After gradient-based edge detection:
