Generating 3D faces using Convolutional Mesh Autoencoders

Learned 3D representations of human faces are useful for computer visionproblems such as 3D face tracking and reconstruction from images, as well asgraphics applications such as character generation and animation. Traditionalmodels learn a latent representation of a face using linear subspaces orhigher-order tensor generalizations. Due to this linearity, they can notcapture extreme deformations and non-linear expressions. To address this, weintroduce a versatile model that learns a non-linear representation of a faceusing spectral convolutions on a mesh surface. We introduce mesh samplingoperations that enable a hierarchical mesh representation that capturesnon-linear variations in shape and expression at multiple scales within themodel. In a variational setting, our model samples diverse realistic 3D facesfrom a multivariate Gaussian distribution. Our training data consists of 20,466meshes of extreme expressions captured over 12 different subjects. Despitelimited training data, our trained model outperforms state-of-the-art facemodels with 50% lower reconstruction error, while using 75% fewer parameters.We also show that, replacing the expression space of an existingstate-of-the-art face model with our autoencoder, achieves a lowerreconstruction error. Our data, model and code are available athttp://github.com/anuragranj/coma