Auxiliary Deep Generative Models

Deep generative models parameterized by neural networks have recentlyachieved state-of-the-art performance in unsupervised and semi-supervisedlearning. We extend deep generative models with auxiliary variables whichimproves the variational approximation. The auxiliary variables leave thegenerative model unchanged but make the variational distribution moreexpressive. Inspired by the structure of the auxiliary variable we also proposea model with two stochastic layers and skip connections. Our findings suggestthat more expressive and properly specified deep generative models convergefaster with better results. We show state-of-the-art performance withinsemi-supervised learning on MNIST, SVHN and NORB datasets.