ARCH++: Animation-Ready Clothed Human Reconstruction Revisited

We present ARCH++, an image-based method to reconstruct 3D avatars witharbitrary clothing styles. Our reconstructed avatars are animation-ready andhighly realistic, in both the visible regions from input views and the unseenregions. While prior work shows great promise of reconstructing animatableclothed humans with various topologies, we observe that there exist fundamentallimitations resulting in sub-optimal reconstruction quality. In this paper, werevisit the major steps of image-based avatar reconstruction and address thelimitations with ARCH++. First, we introduce an end-to-end point based geometryencoder to better describe the semantics of the underlying 3D human body, inreplacement of previous hand-crafted features. Second, in order to address theoccupancy ambiguity caused by topological changes of clothed humans in thecanonical pose, we propose a co-supervising framework with cross-spaceconsistency to jointly estimate the occupancy in both the posed and canonicalspaces. Last, we use image-to-image translation networks to further refinedetailed geometry and texture on the reconstructed surface, which improves thefidelity and consistency across arbitrary viewpoints. In the experiments, wedemonstrate improvements over the state of the art on both public benchmarksand user studies in reconstruction quality and realism.