Assignment #1 - Colorizing the Prokudin-Gorskii Photo Collection

Rohan Choudhury

I. Project Introduction

Overview

I implemented the single scale and multi scale image alignment for this assignment using the basic SSD metric. The only bell/whistle I included was to implement the assignment in pytorch.

Challenges

1. The borders caused a significant amount of issues, especially in the Emir and Self-portrait images. I used a basic cropping procedure where I removed the 10% of the image from all sides, but this is insufficient for the self-portrait image, which required 20% from each side. 2. The other challenge was figuring out the optimal search space size. I used 10 pixels in each direction (for both single and multi-scale), which results in nice images while keeping runtime under a minute.

II. Results

All results below are JPGs that I downsampled 6x on each side to save space. All alignments were computed using a multi-scale algorithm with pytorch.

Low Res Images

(3, 12) (2, 5)
Cathedral

High Res Images

(13, 123) (16, 59)
Harvesters
(23, 89) (17, 41)
Icon
(11, 112) (9, 51)
Lady
(33, 135) (29, 78)
Self Portrait
(55, 103) (24, 49)
Emir of Bukhara
(11, 112) (14, 53)
Three Generations
(32, 87) (5, 42)
Train
(28, 116) (21, 56)
Turkmen
(22, 135) (12, 64)
Village

Additional Images from the collection

Below are the results of my implementation on a few additional images from the collection.

(4, 78) (-16, 48)
Church
(32, -50) (-12, -12)
Train Tracks
(0, 18) (-78, 14)
House By a River

Analysis

The two main failures of my algorithm were the Emir and self-portrait image. The self-portrait can become a lot better if you crop the borders by 20% on all sides, but if you do so, the Emir image gets significantly worse. I imagine that if I implemented a successful adaptaive cropping method, both might be better. Furthermore, I used the basic SSD metric, and gradient based methods could potentially alleviate this issue as well.