Dual Variational Generation for Low-Shot Heterogeneous Face Recognition

Heterogeneous Face Recognition (HFR) is a challenging issue because of thelarge domain discrepancy and a lack of heterogeneous data. This paper considersHFR as a dual generation problem, and proposes a novel Dual VariationalGeneration (DVG) framework. It generates large-scale new paired heterogeneousimages with the same identity from noise, for the sake of reducing the domaingap of HFR. Specifically, we first introduce a dual variational autoencoder torepresent a joint distribution of paired heterogeneous images. Then, in orderto ensure the identity consistency of the generated paired heterogeneousimages, we impose a distribution alignment in the latent space and a pairwiseidentity preserving in the image space. Moreover, the HFR network reduces thedomain discrepancy by constraining the pairwise feature distances between thegenerated paired heterogeneous images. Extensive experiments on four HFRdatabases show that our method can significantly improve state-of-the-artresults. The related code is available at https://github.com/BradyFU/DVG.