A longstanding question in computer vision concerns the representation of 3Dshapes for recognition: should 3D shapes be represented with descriptorsoperating on their native 3D formats, such as voxel grid or polygon mesh, orcan they be effectively represented with view-based descriptors? We addressthis question in the context of learning to recognize 3D shapes from acollection of their rendered views on 2D images. We first present a standardCNN architecture trained to recognize the shapes' rendered views independentlyof each other, and show that a 3D shape can be recognized even from a singleview at an accuracy far higher than using state-of-the-art 3D shapedescriptors. Recognition rates further increase when multiple views of theshapes are provided. In addition, we present a novel CNN architecture thatcombines information from multiple views of a 3D shape into a single andcompact shape descriptor offering even better recognition performance. The samearchitecture can be applied to accurately recognize human hand-drawn sketchesof shapes. We conclude that a collection of 2D views can be highly informativefor 3D shape recognition and is amenable to emerging CNN architectures andtheir derivatives.