In this paper, we consider the problem to automatically reconstruct garmentand body shapes from a single near-front view RGB image. To this end, wepropose a layered garment representation on top of SMPL and novelly make theskinning weight of garment independent of the body mesh, which significantlyimproves the expression ability of our garment model. Compared with existingmethods, our method can support more garment categories and recover moreaccurate geometry. To train our model, we construct two large scale datasetswith ground truth body and garment geometries as well as paired color images.Compared with single mesh or non-parametric representation, our method canachieve more flexible control with separate meshes, makes applications likere-pose, garment transfer, and garment texture mapping possible. Code and somedata is available at https://github.com/jby1993/BCNet.