Adversarial Video Generation on Complex Datasets

Generative models of natural images have progressed towards high fidelitysamples by the strong leveraging of scale. We attempt to carry this success tothe field of video modeling by showing that large Generative AdversarialNetworks trained on the complex Kinetics-600 dataset are able to produce videosamples of substantially higher complexity and fidelity than previous work. Ourproposed model, Dual Video Discriminator GAN (DVD-GAN), scales to longer andhigher resolution videos by leveraging a computationally efficientdecomposition of its discriminator. We evaluate on the related tasks of videosynthesis and video prediction, and achieve new state-of-the-art Fr\'echetInception Distance for prediction for Kinetics-600, as well as state-of-the-artInception Score for synthesis on the UCF-101 dataset, alongside establishing astrong baseline for synthesis on Kinetics-600.