Results

Overview

This project reconstructs historical photographs by aligning color channels from early 20th-century images. The images have exposures with three color channels: red, green, and blue. The reconstruction process involves computing the displacement between channels and aligning them to create full-color images.

Approach

First, the red, blue, and green plates are isolated from the image. Then, a window of [-15, 15] is used to try various displacements and find the one that results in the smallest Euclidean distance. I crop 5% from each side of the image in order to reduce noise from the borders. I do this for both green and red and align them to blue. This works well for small jpgs that are relatively small and can afford to search this window size.

However, for the larger .tif images, they are too large to attempt this same method. Thus, an image pyramid is used in order to gradually scale the image down by a factor of 0.5 repeatedly until the image reaches a similar size to the first example jpg I used. (I checked this by ensuring max[height, width] was less than 400 pixels). Then, a similar approach with the window search range of [-15, 15] is used for the smallest scale image in order to find the optimal displacement for red and green. These displacements are then reused and scaled up for each image respectively, with small searches in windows of size [-4, 4] in order to account for extra pixels that form when rescaling back up. Finally, at the end 5% from each edge is cropped to eliminate most of the noisy borders. For the most part, this aligned most images well, with only the Emir image being slightly misaligned.

A problem I had was that there would be instances in which there was slight misalignment at the end. I addressed this by increasing the search window on the later levels from [-1, 1] to [-4, 4]. Again, the Emir wasn't aligned perfectly, but this could be due to the fact that the alignment was on the blue channel, but the image also has a lot of blue and little green, which could explain the green plate being slightly out of place still.

Bells & Whistles

For my bells & whistles, I implemented Sobel edge detection in order to perfectly align the Emir image that was failing before. I approached it by applying Gaussian Blur in order to reduce noise and smoothen the image. This is important because derivatives are important in edge detection and they are especially sensitive to this. Then the convolution with the Sobel operator derivative masks for edge detection will be less sensitive to noise. Then, the gradient magnitude of the X and Y directions normalized is used as the metric for comparison. This fully aligned the Emir successfully, in combination with a later layer search window of [-2, 2] to account for extra pixels when scaling up. I imported Gaussian Blur and Convolution operations to make life a bit easier. This ultimately resulted in better alignment, but the runtime increased to just under an minute per .tif image. I also increased crop for alignment to 15% and increased the intial search window to [-50, 50].

Chatgpt Credits

Also referenced Bryan Li's formatting of images for his site. https://inst.eecs.berkeley.edu/~cs180/fa23/upload/files/proj1/bryanli0/