FaceForensics++: Learning to Detect Manipulated Facial Images

The rapid progress in synthetic image generation and manipulation has nowcome to a point where it raises significant concerns for the implicationstowards society. At best, this leads to a loss of trust in digital content, butcould potentially cause further harm by spreading false information or fakenews. This paper examines the realism of state-of-the-art image manipulations,and how difficult it is to detect them, either automatically or by humans. Tostandardize the evaluation of detection methods, we propose an automatedbenchmark for facial manipulation detection. In particular, the benchmark isbased on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominentrepresentatives for facial manipulations at random compression level and size.The benchmark is publicly available and contains a hidden test set as well as adatabase of over 1.8 million manipulated images. This dataset is over an orderof magnitude larger than comparable, publicly available, forgery datasets.Based on this data, we performed a thorough analysis of data-driven forgerydetectors. We show that the use of additional domainspecific knowledge improvesforgery detection to unprecedented accuracy, even in the presence of strongcompression, and clearly outperforms human observers.