Depth Pro: Get clear monocular depth measurements in less than a second

1. Tutorial Introduction

Depth Pro is a basic model for zero-shot metric monocular depth estimation that was open-sourced by Apple in October 2024. The related paper results are "Depth Pro: Sharp Monocular Metric Depth in Less Than a Second” by Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter and Vladlen Koltun.

Depth Pro can quickly generate a high-resolution 3D depth map from a single 2D image. This model is not only fast, taking only 0.3 seconds, but also provides metric-level depth information, and the generated depth map has a real world scale. Depth Pro does not rely on the camera's intrinsic parameters, such as focal length, and has strong versatility. It performs well in capturing boundary details and can clearly depict subtle structures such as hair and vegetation. Depth Pro can perform zero-shot learning and make accurate predictions without training data in a specific field, making it have broad application potential in multiple fields such as augmented reality, 3D reconstruction, and image editing.

Key features of Depth Pro include:

• Zero-shot Metric Depth Estimation: Generating a metric depth map with absolute scale from a single 2D image without intrinsic camera parameters.
• High-resolution output: The model can generate depth maps up to 2.25 megapixels, providing rich details.
• Fast processing: On a standard GPU, Depth Pro generates a depth map in 0.3 seconds, suitable for real-time applications.
• Detail capture: It is particularly good at capturing subtle structures such as hair and vegetation, and improving the clarity of boundaries.

In terms of technical principles, Depth Pro is based on an efficient multi-scale visual transformer (ViT) architecture, which captures the global image context while accurately identifying fine structures at high resolution. It combines real and synthetic datasets for training to achieve high-precision measurement and detailed boundary tracking. Depth Pro also estimates focal length from a single image, leading the field of zero-sample focal length estimation. In addition, it adopts a two-stage training strategy, the first stage aims to learn cross-domain robust features, and the second stage focuses on sharpening boundaries and revealing subtle details in the predicted depth map.

Effect Preview

2. Operation steps

After starting the container, click the API address to enter the Web interface

High-resolution depth map synthesis

There are two parameters to choose from during the generation process

• Auto Rotate: Automatic rotation
• Remove Alpha: Remove Alpha

Upload an image or select a sample as requested

Exchange and discussion

🖌️ If you see a high-quality project, please leave a message in the background to recommend it! In addition, we have also established a tutorial exchange group. Welcome friends to scan the QR code and remark [SD Tutorial] to join the group to discuss various technical issues and share application effects↓