Joint Voxel and Coordinate Regression for Accurate 3D Facial Landmark Localization

3D face shape is more expressive and viewpoint-consistent than its 2Dcounterpart. However, 3D facial landmark localization in a single image ischallenging due to the ambiguous nature of landmarks under 3D perspective.Existing approaches typically adopt a suboptimal two-step strategy, performing2D landmark localization followed by depth estimation. In this paper, wepropose the Joint Voxel and Coordinate Regression (JVCR) method for 3D faciallandmark localization, addressing it more effectively in an end-to-end fashion.First, a compact volumetric representation is proposed to encode the per-voxellikelihood of positions being the 3D landmarks. The dimensionality of such arepresentation is fixed regardless of the number of target landmarks, so thatthe curse of dimensionality could be avoided. Then, a stacked hourglass networkis adopted to estimate the volumetric representation from coarse to fine,followed by a 3D convolution network that takes the estimated volume as inputand regresses 3D coordinates of the face shape. In this way, the 3D structuralconstraints between landmarks could be learned by the neural network in a moreefficient manner. Moreover, the proposed pipeline enables end-to-end trainingand improves the robustness and accuracy of 3D facial landmark localization.The effectiveness of our approach is validated on the 3DFAW and AFLW2000-3Ddatasets. Experimental results show that the proposed method achievesstate-of-the-art performance in comparison with existing methods.