FaceNet: A Unified Embedding for Face Recognition and Clustering

Despite significant recent advances in the field of face recognition,implementing face verification and recognition efficiently at scale presentsserious challenges to current approaches. In this paper we present a system,called FaceNet, that directly learns a mapping from face images to a compactEuclidean space where distances directly correspond to a measure of facesimilarity. Once this space has been produced, tasks such as face recognition,verification and clustering can be easily implemented using standard techniqueswith FaceNet embeddings as feature vectors. Our method uses a deep convolutional network trained to directly optimize theembedding itself, rather than an intermediate bottleneck layer as in previousdeep learning approaches. To train, we use triplets of roughly aligned matching/ non-matching face patches generated using a novel online triplet miningmethod. The benefit of our approach is much greater representationalefficiency: we achieve state-of-the-art face recognition performance using only128-bytes per face. On the widely used Labeled Faces in the Wild (LFW) dataset, our systemachieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves95.12%. Our system cuts the error rate in comparison to the best publishedresult by 30% on both datasets. We also introduce the concept of harmonic embeddings, and a harmonic tripletloss, which describe different versions of face embeddings (produced bydifferent networks) that are compatible to each other and allow for directcomparison between each other.