WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild

In recent years, 3D hand pose estimation methods have garnered significantattention due to their extensive applications in human-computer interaction,virtual reality, and robotics. In contrast, there has been a notable gap inhand detection pipelines, posing significant challenges in constructingeffective real-world multi-hand reconstruction systems. In this work, wepresent a data-driven pipeline for efficient multi-hand reconstruction in thewild. The proposed pipeline is composed of two components: a real-time fullyconvolutional hand localization and a high-fidelity transformer-based 3D handreconstruction model. To tackle the limitations of previous methods and build arobust and stable detection network, we introduce a large-scale dataset withover than 2M in-the-wild hand images with diverse lighting, illumination, andocclusion conditions. Our approach outperforms previous methods in bothefficiency and accuracy on popular 2D and 3D benchmarks. Finally, we showcasethe effectiveness of our pipeline to achieve smooth 3D hand tracking frommonocular videos, without utilizing any temporal components. Code, models, anddataset are available https://rolpotamias.github.io/WiLoR.