Expressive Personalized 3D Face Models from 3D Face Scans

PhD project of Stella and ongoing research.

Note: this page is still work in progress, including the selection of nice images created during my Phd.

In 2012 I started my PhD project initially on Talking Heads. The idea was to generate a 3D facial animation of a person with accompanying speech given text. Eventually, I reached an intermediate step with a textured coarse 3D model, as seen below on the left.

The core of my final PhD project was a Higher-Order Singular Value Decomposition (HO-SVD) of a data tensor holding 3D face point clouds. Those were a result of 3D point correspondence estimation. For some databases a temporal alignment was necessary. This process unveiled an underlying expression subspace shown in the middle below. Using the decomposition and carefully designed constraints, we were able to make meaningful edits to the 3D face shapes as shown on the bottom right animation.

From Data to Aligned Data for Facial Models

Please find an overview of the whole process from 3D face scans to the 3D aligned faces in the following illustration:

Spatial Alignment

Multiple 3D face scans commonly differ in the number of points, and therefore need a general rigid alignment, followed by a registration and correspondence estimation procedure. In this domain prior knowledge in terms of facial landmarks is employed to improve the results.

Temporal Alignment

When recording facial motion in 3D, the resulting 3D face scans might differ in their sequence length, hence requiring a temporal alignment. To achieve this in this project prior knowledge can be employed which is that the facial motion in the recorded sequences follows a common patter, which consists of an increase followed by an decrease of a depicted facial expression forming an emotion. Hence, an approach was formulated to estimate the expression intensity for each frame. In the following the resulting 1D signals encoding the intensity of facial expression per frame are used to compute an alignment of the 3D face scans, as illustrated.

The 3D Face Model

After all the preprocessing and alignment, a balanced dataset with the same number of 3D points and number of frames has been obtained. This data can now be sorted into a matrix or data tensor and a factorisation method can be employed. In this work, the Higher-Order Singular Value Decomposition (HO-SVD) was employed to factorize the data tensor into different subspaces. Depending on the chosen order and dimension of the data tensor, new insights were gained. One of the main contributions was what we referred to as the expression subspace, which revealed a structure in a lower dimensional space.



  1. 2021_grasshof_tpami.png
    Multilinear Modelling of Faces and Expressions
    Stella Graßhof, Hanno Ackermann , Sami Sebastian Brandt , and 1 more author
    IEEE Transactions on Pattern Analysis and Machine Intelligence, Oct 2021


  1. 2019_phd.png
    Expressive Personalized 3D Face Models from 3D Face Scans
    Stella Graßhof
    Nov 2019
    PhD thesis


  1. 2019_brandt_iccv.png
    Uncalibrated Non-Rigid Factorisation by Independent Subspace Analysis
    Sami Sebastian Brandt , Hanno Ackermann , and Stella Graßhof
    In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) , Oct 2019
    ISSN: 2473-9944


  1. 2018_awiszus_cvpr.png
    Unsupervised Features for Facial Expression Intensity Estimation Over Time
    M. Awiszus , S. Graßhof , F. Kuhnke , and 1 more author
    In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , Jun 2018
    ISSN: 2160-7516


  1. 2017_grasshof_mva.png
    Projective structure from facial motion
    Stella Graßhof, Hanno Ackermann , Felix Kuhnke , and 2 more authors
    In 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA) , May 2017


  1. 2017_grasshof_fg.png
    Apathy Is the Root of All Expressions
    Stella Graßhof, Hanno Ackermann , Sami S. Brandt , and 1 more author
    In 2017 12th IEEE International Conference on Automatic Face Gesture Recognition (FG 2017) , May 2017


  1. 2015_grasshof_mva.png
    Estimation of face parameters using correlation analysis and a topology preserving prior
    Stella Graßhof, Hanno Ackermann , and Jörn Ostermann
    In 2015 14th IAPR International Conference on Machine Vision Applications (MVA) , May 2015


  1. 2013_psivt.png
    Performance of Image Registration and Its Extensions for Interpolation of Facial Motion
    Stella Graßhof, and J. Ostermann
    In Pacific-Rim Symposium on Image and Video Technology (PSIVT) Workshops , 2013