Assignment 1 - Colorizing the Prokudin-Gorskii Photo Collection

---

Overview

Text giving a brief overview of the project, and text describing your approach.

The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.

The key idea is to align the three color channels of the image, and then to combine them into a single color image. The alignment process involves finding the relative offsets of the green and red channels with respect to the blue channel.

The main steps of the project are:

1. Load the digitized images and split them into three separate images, one for each color channel.
2. Determine the relative offsets of the green and red channels with respect to the blue channel by finding the best match between the channels. In this step, I used the squared difference to measure the similarity between the channels under different offsets. Besides, using pyramid method to find the best match in a large range of offsets.
3. Use the offsets to align the green and red channels with the blue channel.
4. Combine the three channels into a single color image.

Problem Description

If you ran into problems on images, describe how you tried to solve them.

I ran into some problems and I solved them eventually:

1. Problem: The color of the images was not natural. For example, the cathedral image was yellowish. Solution: I found the reason was that the pixel values of the images were not in the range of 0-255. I normalized the pixel values to 0-255 and the color of the images became natural.

2. Problem: Generating large images was slow. Solution: I used the multiprocessing library to parallelize the code and the speed of generating large images increased significantly.

3. Problem: The alignment of the images was not good. Solution: I used the internal pixels of the images to compute the squared difference of the pixel values and found the best alignment of the images.

Results

Bells and Whistles

Describe any bells and whistles you implemented. For maximum credit, show before and after images.

1. I implemented an automatic cropping before aligning the images. The automatic cropping can remove the black or the white borders of the images. Use cathredal.jpg as an example:

   • The image without cropping the border before alignment (left)

   • The image with cropping the border before alignment (right)

     

   From the images above, we can see that the black and white borders are removed after cropping.

2. I implemented an automatic cropping after aligning the images. The automatic cropping can remove the part of single-color borders of the images. Use cathredal.jpg as an example:

   • The image without cropping the border after alignment (left)

   • The image with cropping the border after alignment (right)

     

   From the images above, we can see that the single-color borders are removed after cropping.

Discuss: in some results, the borders are not removed completely due to the noise in the borders.

Example Images

The website does not need to be pretty; you just need to explain what you did.

The result of your algorithm on all of our example images. List the offsets you calculated. Do not turn in the large .tiff images. Your web page should only display compressed images (e.g. jpg or png or gif if you want to animate something).

cathedral.jpg

• pyramid depth: 1 window: 20 scale: 0.5

• g_alignment offset: [0, 2]

• r_alignment offset: [0, 2]

• Time: 0.03 minutes

  

emir.tif

• pyramid depth: 1 window: 50 scale: 0.5

• g_alignment offset: [28, -4]

• r_alignment offset: [116, 14]

• Time: 2.61 minutes

  

harvesters.tif

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [60, 10]

• r_alignment offset: [120, 2]

• Time: 0.66 minutes

  

icon.tif

• pyramid depth: 1 window: 20 scale: 0.5

• g_alignment offset: [42, 16]

• r_alignment offset: [80, 20]

• Time: 0.60 minutes

  

lady.tif

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [58, -6]

• r_alignment offset: [120, -16]

• Time: 0.65 minutes

  

self_portrait.tif

• pyramid depth: 3 window: 20 scale: 0.5

• g_alignment offset: [66, 0]

• r_alignment offset: [160, -2]

• Time: 0.71 minutes

  

three_generations.tif

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [52, 6]

• r_alignment offset: [120, 8]

• Time: 0.62 minutes

  

train.tif

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [40, -2]

• r_alignment offset: [106, 2]

• Time: 0.66 minutes

  

turkmen.tif

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [58, 4]

• r_alignment offset: [116, 2]

• Time: 0.62 minutes

  

village.tif

• pyramid depth: 2 window: 35 scale: 0.5

• g_alignment offset: [56, -8]

• r_alignment offset: [136, -14]

• Time: 1.57 minutes

  

Examples of Own Choosing

The result of your algorithm on a few examples of your own choosing, downloaded from the Prokudin-Gorskii collection. If your algorithm failed to align any image, provide a brief explanation of why.

City of Shlisselburg

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [38, 2]

• r_alignment offset: [84, -2]

• Time: 0.66 minutes

  

• Explanation: From the image above, we can see that most parts of the images are aligned well. However, the bottom part of the image (people) is not aligned well, since we only use simple translation to align the images. There may be some distortions in the different color channels, which may cause the misalignment.

Studies on Lindozero

• pyramid depth: 2 window: 20 scale: 0.5

• g_alignment offset: [-46, -34]

• r_alignment offset: [-46, -44]

• Time: 0.60 minutes