Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

First I defined the partial derivatives as described in the project specifications as $D_x = [1 \ \ \ -1]$ and $D_y = \left[\begin{matrix}1\\-1\end{matrix}\right]$. Then I applied this to the one channel (as it was a black and white image) via a convolution operation, specifically the one suggested in the specs convolve2d from scipy.signal. After this I computed the gradient magnitude as $\sqrt{D_x^2 + D_y^2}$, which allows us to see the edges more clearly.

$D_x$

$D_y$

gradient magnitude

Now it was time to binarize the image after taking the gradient magnitude to get it to have a cleaner look. To do this I created a function that given a photo and a percentage it returns the corresponding pixel value threshold such that if applied to the image only that percent of pixel remain. I found that using a percentage of 3 seemed to the best results, as it gets rid of a lot of the noise while still maintaining the outline camera man. These are the results of using percents 1, 3, 5, and 10.

Final image chosen from the thresholds (3%)

1% threshold

3% threshold

(chosen image)

5% threshold

10% threshold

Part 1.2: Derivative of Gaussian (DoG) Filter

Our previous result is a little pixelated, so to help get a smoother and cleaner look, we will first apply a Gaussian smoother over the image. For choosing the kernel size, I used the rule of thumb that was presented in class which stated:“Rule of thumb for Gaussian: set filter half-width to about 3 σ”. Thus I set ksize = $2 \cdot (3 \cdot \sigma) + 1 = 6\sigma + 1$, where the +1 ensures that my kernel size is odd Throughout the rest of this project I set my kernel size according to this rule. These are the results I got

partial x (with blurring)

partial y (with blurring)

gradient magnitude (with blurring)

edges, 5% threshold (with blurring)

The main differences I see are the reduction in pixelation in all steps of the process specified in section 1.1. The whole look appears a lot smoother.

We can achieve these same results with one fewer convolution by only convolving the original image on the derivatives of the Guassians (DoG), as opposed to convolving with the Gaussian and then with the derivative.

DoG wrt x

DoG wrt y

DoG gradient magnitude

DoG edges, 5% threshold

Thus we have confirmed that we get the same results regardless if we do these steps in two separate convolutions or in one combined convolution.

Part 2: Fun with Frequencies!

Part 2.1: Image "Sharpening"

original

Here we are doing image sharpening, where we get the high frequencies from the original image and enhance them by adding them onto the original image. You can control how much of the higher frequencies you want to add by increasing or decreasing the coefficient ($\alpha$). The original image is shown on the left. Below is the results on different levels of $\alpha$.

$\alpha = 0.25$

$\alpha = 0.5$

$\alpha = 1$

$\alpha = 2$

$\alpha = 5$

$\alpha = 10$

More examples (expanding them to be larger to see the differences better)

Berkeley

Qazvin

In both results of my own example, the edges are sharper and more defined compared to the blurry version of the image. However it’s not quite as sharp as the original image prior to blurring.

Part 2.2: Hybrid Images

When we are looking at things up close, such as reading, we tend to focus on the higher frequencies. On the other hand, when we are squinting, looking far away, or in our peripheral we tend to pick up more on the lower frequencies. Thus we can take the lower frequencies of one image and the higher frequencies of another image and combine them to get a hybrid image. Here are some examples, note that I implemented some bells and whistles by using both color and grayscale.

Derek

Nutmeg

Derek + Nutmeg gray

Derek + Nutmeg color

Swimming - Mac Miller

Nurture - Porter Robinson

Swimming + Nurture gray

Swimming + Nurture color

Example with FFT frequencies displayed:

Nikola Jokić

FFT(Nikola Jokić)

Aaron Gordon

FFT(Aaron Gordon)

Jokić (low frequencies)

FFT[Jokić (low frequencies)]

Gordon (high frequencies)

FFT[Gordon (high frequencies)]

Hybrid Image Gray

Hybrid Image Color

FFT(Hybrid Image)

Failure

Note that I had to play around with the different values of $\sigma$. A lot of the time, such as the one to the left resulted in a blended image where one of the images wasn’t very visible. Here Aaron Gordon isn’t very visible.

Part 2.3: Gaussian and Laplacian Stacks

To make a Gaussian stack, you incrementally blur the image with a Gaussian convolution without scaling down the dimensions of the image. Then to create a Laplacian stack you take the differences of the layers in the Gaussian stack, with the last image in the stack being the last image in the Gaussian stack. Note these images have been normalized.

And note that the Laplacian stacks, when collapsed result in the initial image.

Part 2.4: Multiresolution Blending

In order to blend the images, we need to create a mask to apply to each of the Laplacian stacks and add them together creating a new blended stack. Then we take this blended stack and collapse it leaving it with our final image.

Apple

Orange

Aporange

Blended Laplacian stacks (apple, orange, aporange):

These are some examples of my own

Dwayne Johnson

Tanoai Reed (Dwyane’s Stunt Double)

Dwayne Reed

This next examples uses a more complex mask

Windows XP background

Back of dollar bill

Mask for pyramid

Blended

Note: I made the mask like this to include the plants at the bottom

Here are the Laplacian stacks, as this is my favorite example

Another example with irregular masks:

Real starry night

Starry Night - Vincent van Gogh

Mask for city

Blended

Final blended images:

Learning:

I think the most important thing I learned with this project, is the visualization of different frequencies. In other words, how we perceive different frequencies. I thought the hybrid images were really interesting and it taught me about my own perception. Furthermore, I had always wondered how software was able to clear up images and make them sharper, and this also had to do with frequencies