Occupancy Network

Occupancy Network is a new representation of learning-based 3D reconstruction methods. This concept was first proposed in the paper “Occupancy Networks: Learning 3D Reconstruction in Function Space", which has been accepted by CVPR 2019.

The core idea of Occupancy Network is to obtain a simple 3D spatial representation by predicting the occupancy probability in 3D space. This method does not rely on traditional 3D object detection, but divides the world into tiny cubes or voxels and predicts whether each voxel is free or occupied. In this way, Occupancy Network can run at a speed of more than 100 FPS, is super memory-efficient, and can understand both moving and static objects.

Tesla introduced the concept of Occupancy Network at CVPR 2022 and Tesla AI Day, and demonstrated its application in perception systems. Tesla's Occupancy Network model structure includes extracting features from images from multiple perspectives, then predicting occupancy through attention modules and transformers, and finally outputting 3D space occupancy volume and occupancy flow.

In addition, Occupancy Network is also combined with Neural Radiance Field (NeRF) technology to verify whether the predicted 3D scene matches the actual scene by comparing the 3D volume generated by Occupancy Network with the 3D reconstructed scene generated by NeRF. This method helps to solve complex environmental problems such as occlusion, image blur, rain and fog.

In the field of autonomous driving, Occupancy Network provides a new perspective to handle perception tasks, especially when dealing with long-tail obstacles and objects of unknown categories. With the continuous development of technology, Occupancy Network is expected to become an indispensable part of autonomous driving perception systems.