CS180 Project 1 - Colorizing Images from the Prokudin-Gorskii Photo Collection

Background

Sergei Mikhailovich Prokudin-Gorskii was a photographer living between the 19th and 20th century in Russia. As he travelled across the country he recorded his journeys onto three different plates for each photograph, one with a red filter, one with a green filter, and one with a blue filter. His intent would be to later combine his images into one colored image. The aim of this project is to finish what Sergei couldn't complete, and create a program to combine these digitized glass plate to create colored images.

Methods

Once the three red, green, and blue glass plates were obtained for a given image, we need to align them to stack on top of each other nicely. To help with alinement I first start off by manually cropping (in Python code, not with an image editor) each image by a certain percent to get rid of the uneven black borders. This way the alignment tends to work better. Here are some examples of the end result when I cropped vs when I didn't crop

Cropped (left), un-cropped (right)

After cropping, I attempted to do automatic cropping based on pixel values, but for some reason I was only cropping off one pixel by doing this. And I couldn't figure out why.

To help optimize the alignment I implemented pyramid alignment. In pyramid alignment I recursively shrink the image by scaling it down by a factor of 2 for each “level” in the pyramid. This way we essentially calculate the alignment best shift on the coarser images, and apply the best shift to the higher resolution images. In this case, I chose the number of levels based on the file extension. Tiffs were higher resolution and needed more levels to help speed up the process compared to the jpegs.

In the case of the actual alignment I searched $\pm 15$ pixels in each direction and used the L2 norm to calculate the best alignment. In general I found that the L2 metric left me with some of the best results and also it made the computing go a lot quicker. This helped me solve the problem of speed and memory. Because I ran these on data hub, often times it would take a long time to run some of my code. This helped improve both the quality and run time.

The biggest issue with alignment was the emir.tiff file. I think due to the nature of the borders of that image, I was unable to align it well. As you can see in the final result it's still a little blurry. In fact in this case, it was more blurry when I tried to crop it, even though I evenly cropped on both sides. Another issue I had, was that I noticed that the top 10 percent of my image was concatenated at the bottom of my image. So to overcome this, I took this 10 percent off and the prepended it to my image to reconstruct it properly. Lastly, something else I added was the color correction for the extra credit, however I liked the images prior to color correction better and didn't have time to find the best color correction hyper parameters.