Low Res and High Res Images
Given the prokudin-Gorskii glass plate images, we extract the individual color channels and try to align them into a RGB image with as few visual artificats as possible. To do that, we extract the channels and exhaustively search for minimum "misalignment" by slightly shifting the image channels and keeping the shift that produces minimum alignment. The misalignment is measured by L2 norm. We try comparing distances between raw channel values as well as edge features. An important detail was to remove borders from the channels before searching for the alignment to make things less noisy because of the noisy borders. Concretely, we remove 15 pixels from each edge of the image.
Since this search can be slow, we also perform a multi-scale version, where we recursively compute the shift on half resolution image and scale back the shift by 2. we keep halving the resolution until the image becomes small sized (i.e. 300, 300).
We implement the code in Pytorch.
Low Res and High Res Images
We implement everything in pytorch, please check starter.py
We calculate the uncertainty in normalized channel values the sum of L2 norms between all pairs of channels. Then, we calculate the mean and std of the uncertainity and thresshold the uncertainty which is more than mean + 1.5 * std (1.5 was a hyperparameter). Then, we sum the uncertanity along both axis and threshhold the sum with 0.05 of the image size. This gives us the pixels which are uncertain along rows and columns. Then we calculate the border by finding the min/max non-zero element within the 0.1 * size of the image.
We use sobel filter to get the edges and use those as features
We observe slightly sharper results with edge features