Image of Nitya
Image of Neha
In this project, I produce a "morph" animation of my face into my sister's face, compute the mean of a population of faces, extrapolate from the population mean to create a caricature of myself and produce a video / gif of my face throughout the years from the ages of six to twenty-one.
First, I found an image of myself and an image of my sister. I resized and cropped it so that our faces were more aligned with each other. In order to morph my face into my sister's face, I first defined pairs of corresponding points on the two images by hand. I found that by increasing the number of key points, the morph becomes smoother. To do this, I used the labelling tool provided by a previous student: https://cal-cs180.github.io/fa23/hw/proj3/tool.html to ensure that there is a consistent labeling of the two faces. I used the same ordering of keypoints in the two faces as well. I then exported the correspondences into a json file and read them into my jupyter notebook when I needed them. Then, I averaged the two sets of correspondences (one for each image) to get the correspondences for the midway shape to reduce the likelihood of any potential triangle deformations. Using the points for the midway shape, I calculated the Delaunay triangulation, which is the optimal choice as it does not produce incredibly skinny triangles. Below, I have displayed the Delaunay triangulation overlaying each image:
In order to compute the mid-way face of me and my sister, I first computed the average shape by taking the average of each keypoint location in the two faces: (nitya_pts + neha_pts) / 2. Then, I warped both faces into that average shape and averaged the colors together too. To do this, I first iterated through each triangle in the Delaunay triangulation and generated a mask using a polygon with the points for each triangle. I then found the corresponding points in the image of my face and the image of my sister's face. I computed the affine matrix that transforms the points of the triangle in the shape of my face to the points of the triangle in the shape of the midway face before taking its inverse. Similarly, I computed the affine matrix that transforms the points of the triangle in the shape of my sister's face to the points of the triangle in the shape of the midway face before taking its inverse as well. I computed the dot products of the inverse affine matrices with the coordinates in the polygon. The outputs of the dot product calculations are then used to warp the images of me and my sister. Finally, I cross-dissolve the images using this formula: (1 - alpha) * warped_im_nitya + alpha * warped_im_neha. Below, I show you the original images, the warped images and the midway face:
In this task, I wrote a function: morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac) which essentially accomplishes what I had done in part two above but we can now easily change by how much we want to warp the images and by much we want to cross-dissolve them. Both warp_frac and dissolve_frac lie in the range [0,1]. They are also the only parameters that vary from frame to frame in the gif. For the very first starting frame, they will both equal 0, and for the last ending frame, they will both equal 1. Below is a gif showing my face morphing into my sister's face:
For this task, I used the FEI Face Database for research purposes. As mentioned on their website, it is "a Brazilian face database that contains a set of face images taken between June 2005 and March 2006 at the Artificial Intelligence Laboratory of FEI in São Bernardo do Campo, São Paulo, Brazil." I separated it into two subsets: one for neutral expressions and one for smiling expressions.
In order to compute the mean face of the population, I first found the average shape of each of the subsets by taking the mean of the correspondences. I then morphed each of the faces in the dataset into the average shape to get the warped faces. Below are some examples:
Finally, I computed the average face for each subset by taking the mean of all the warped faces in the corresponding subset. Then, I also warped my own face to the average shape and also warped the average shape to my own face. Below are the images:
To produce a caricature of myself, I extrapolated from the population mean that I calculated in the previous task. To accomplish this, I used the following formula: nitya_caricature_pts = avg_smile_pts + k * (nitya_pts - avg_smile_pts) where k > 1 e.g. below I have used k = 1.5. Using these caricature points, I computed the Delaunay triangulation and warped my face, as shown below:
I also created a morphing music video of my face at different ages (at 6, 11, 12, 13, 16, 17, 18, and 21 yrs old). I accomplished this by creating multiple gifs and concatenating them together into an mp4 video with some background music.
