Domain Agnostic Learning with Disentangled Representations

Unsupervised model transfer has the potential to greatly improve thegeneralizability of deep models to novel domains. Yet the current literatureassumes that the separation of target data into distinct domains is known as apriori. In this paper, we propose the task of Domain-Agnostic Learning (DAL):How to transfer knowledge from a labeled source domain to unlabeled data fromarbitrary target domains? To tackle this problem, we devise a novel DeepAdversarial Disentangled Autoencoder (DADA) capable of disentanglingdomain-specific features from class identity. We demonstrate experimentallythat when the target domain labels are unknown, DADA leads to state-of-the-artperformance on several image classification datasets.