For this assignment, we had to generate composite color images of the Prokudin-Gorskii photo collection. We were given digitized versions of the glass negatives, which are three black and white images containing color data for the R, G, and B channels respectively.
I accomplished this project using Matlab. I first cropped off the white border in order to make the images easier to work with. Then I seperated the R, G, and B images. I then aligned the R and G images to the B image, and finally concatenated them into a single image.
The bulk of the work was done in the alignment part of the algorithm. First, I created a brute force algorithm which shifts the R/G image by [-15,15] pixels, and computes a score between the shifted R/G image and the B image. The scoring metric I used was the sum of squared differences: I would find the shifted image with the smallest sum.
Since this brute force algorithm is slow on large images, I then created a recursive algorithm using a multi-level pyramid. The original image is scaled down by half to create each level of the pryamid. I found that best results tended to occur when I had 4-6 levels in the pyramid. Then, starting with the smallest image in the pyramid, my algorithm calculated the best alignment using the brute force method described above. It then uses this estimate of the best alignment in the next higher level. I multiply the estimate displacement by 2 when moving up to the next image in the pyramid. Then, I use this scaled estimate and look at the displacements within a certain window of the estimate from the previous pyramid level. I construct a new estimate in this way, and this new estimate goes up and is used in the next pyramid level, until all the levels of the pyramid have been recursed through.
While my algoirthm produces correct results on many of the images, there are some which are problematic. I did notice that adjusting how many layers in my pyramid effected how good some of the results looked (some images looked better with 4 levels and some with 6, for example.) I'm not sure if that means there is some sort of bug in my recursion, or if it's just the nature of the images. I'm not sure why my algorithm returns really good results for some images and worse results for others. I'm guessing it's either a problem in the recursion calculations, or the fact that I used SSD as a scoring metric and since it's a pretty basic scoring method it might not produce the opitmal solution all the time.