Part 1: Defining Correspondences

Note that throughout this project I didn’t use photos of myself. Instead for myself I used Tyler the Creator’s mugshot. So the first thing I did was crop the photos so that they would be the same size. I ended up choosing 48 points. Here they are with their corresponding Delaunay triangulation.

Part 2: Computing the "Mid-way Face"

Now to compute the mid-way face between Tyler and George, we first need to compute the mid-way triangulation. To do that we need to compute the midway points between each of the correspondences. From here we can then create a set of midway points which we can then find the Delaunay triangulation of. Which is plotted on the right:

From here we now need to warp both George and Tyler to this new mid-way mapping, and then take the average. But note to do this warping we need to use inverse warping so that we don’t have any missing parts in our resulting image. To do this inverse warping we need to calculate the inverse of the affine matrix that can transition our current points into the midway points. But once this is done and we combine the images We get the following

However, as we discussed in lecture, hair is often non-linear and it’s hard to combine these two images due to their different hair styles.

Part 3: The Morph Sequence

Using the functions from the previous part, now we can simply take multiple weighted averages (rather than a 50/50 split). Then we can take these many weighted averages, and shift the weight from one extreme to the other to get our resulting morph sequence as seen on the left. Here I used 60 frames for 30 ms per frame.

Part 4: The "Mean face" of a population

Here I am taking the mean of the Danish population. To do this I had to parse the .asf files and save the relative coordinates that made up their corresponding points. From here I calculate a Delaunay triangle of each of the points and loop through each triangle and inverse warp them using my previous code. I continuously keep adding these morphed photos to a total photo and divide by the total number of images. I tried various types of subpopulations to try and see which are the most distinct to use for the Bells and Whistles section. The overall average can be seen to the right, and a handful of subpopulations can be seen below.

Note that the average Danish person looks more male because the dataset is primarily composed of Danish males.

Here are some examples of individual faces getting morphed into the average shape:

I’ve also included Tyler’s face warped into the shape of an average Danish person, and the average Danish person warped into the shape of Tyler.

I think that Tyler being shaped to the average Danish person has a large forehead due to his short hair.

Part 5: Caricatures: Extrapolating from the mean

Here to extrapolate from the mean, I used the following schemes to both interpolate and extrapolate. First being $\text{car} = \alpha \mu + (1 - \alpha)x$, where $\mu$ is the mean face and $x$ is Tyler’s Face. Resulting in the following:

I also tried this extrapolation method prior to the other method on accident, but I switched one of the variables on accident. $\text{car} = \mu + \alpha(x - \mu)$, which once expanded looks simliar to the first scheme, but slightly different. This resulted in a similar output in reverse (still some small differences).

I thought it would be interesting to include both. NOTE click on the images to enlarge them.

Bells and Whistles

The first Bells and Whistles that I implemented is: Change age/gender/ethnicity/smile/etc of your (or your friend's) face. You can use average images off the web for this, no need to recompute the averages yourself (unless you want to). Show morphing just the shape, just the appearance, and both.

To demonstrate this, I chose the most distinctive mean from part 4, which to me seemed to be the average smiling Danish person. I then used it for creating the caricature (seen to the left). To create this I morphed with warp_frac and dissolve_frac being $\frac12$.

To show just the shape I set warp_frac to $\frac12$ and dissolve_frac to $1$. And vice versa for the appearance. This way the corresponding portion gets canceled out ($1 -1$) in the equations, and we are only left with either appearance or shape.

The second Bells and Whistles I implemented is PCA: Use one of the datasets to compute a PCA basis for the face space. Try performing caricatures and other transformations in the new basis. Compare your results to doing this in normal basis. Are the results better?

Here are the 10 most relevant Eigen Faces computed from the same Danish dataset that we’ve used throughout the project.

We can see that we can reconstruct images within the span or at least close to the span of the eigenfaces pretty well. For example on the first image in the dataset we can clearly see the face only using 5 eigenfaces.

However Tyler the Creator is not Danish, and neither are his features. (Ethnically his background is Igbo Nigerian and White Canadian). Thus projecting him into this new eigenspace will not turn out very well as seen below. And thus the results are definetly not better.

Although note at the same time you can make out his pink shirt and a darker overall skin done.

Yet, still we will construct caricatures from this image and we get there resulting, and slightly horrifying caricatures. Again note: click on the images to enlarge them.

The third Bells and Whistles I implemented is making a morphing music video of me throughout my different ages: Make a morphing music video on a theme. For instance, you can pick a photo of yourself from different ages and make a movie that shows how your face changed over time. Or you can morph between your friends/dorm mates etc.