Recent deep networks that directly handle points in a point set, e.g.,PointNet, have been state-of-the-art for supervised learning tasks on pointclouds such as classification and segmentation. In this work, a novelend-to-end deep auto-encoder is proposed to address unsupervised learningchallenges on point clouds. On the encoder side, a graph-based enhancement isenforced to promote local structures on top of PointNet. Then, a novelfolding-based decoder deforms a canonical 2D grid onto the underlying 3D objectsurface of a point cloud, achieving low reconstruction errors even for objectswith delicate structures. The proposed decoder only uses about 7% parameters ofa decoder with fully-connected neural networks, yet leads to a morediscriminative representation that achieves higher linear SVM classificationaccuracy than the benchmark. In addition, the proposed decoder structure isshown, in theory, to be a generic architecture that is able to reconstruct anarbitrary point cloud from a 2D grid. Our code is available athttp://www.merl.com/research/license#FoldingNet