Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images

The 3D Morphable Model (3DMM), which is a Principal Component Analysis (PCA)based statistical model that represents a 3D face using linear basis functions,has shown promising results for reconstructing 3D faces from single-viewin-the-wild images. However, 3DMM has restricted representation power due tothe limited number of 3D scans and the global linear basis. To address thelimitations of 3DMM, we propose a straightforward learning-based method thatreconstructs a 3D face mesh through Free-Form Deformation (FFD) for the firsttime. FFD is a geometric modeling method that embeds a reference mesh within aparallelepiped grid and deforms the mesh by moving the sparse control points ofthe grid. As FFD is based on mathematically defined basis functions, it has nolimitation in representation power. Thus, we can recover accurate 3D facemeshes by estimating appropriate deviation of control points as deformationparameters. Although both 3DMM and FFD are parametric models, it is difficultto predict the effect of the 3DMM parameters on the face shape, while thedeformation parameters of FFD are interpretable in terms of their effect on thefinal shape of the mesh. This practical advantage of FFD allows the resultingmesh and control points to serve as a good starting point for 3D face modeling,in that ordinary users can fine-tune the mesh by using widely available 3Dsoftware tools. Experiments on multiple datasets demonstrate how our methodsuccessfully estimates the 3D face geometry and facial expressions from 2D faceimages, achieving comparable performance to the state-of-the-art methods.