DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Clouds

Existing offboard 3D detectors always follow a modular pipeline design totake advantage of unlimited sequential point clouds. We have found that thefull potential of offboard 3D detectors is not explored mainly due to tworeasons: (1) the onboard multi-object tracker cannot generate sufficientcomplete object trajectories, and (2) the motion state of objects poses aninevitable challenge for the object-centric refining stage in leveraging thelong-term temporal context representation. To tackle these problems, we proposea novel paradigm of offboard 3D object detection, named DetZero. Concretely, anoffline tracker coupled with a multi-frame detector is proposed to focus on thecompleteness of generated object tracks. An attention-mechanism refining moduleis proposed to strengthen contextual information interaction across long-termsequential point clouds for object refining with decomposed regression methods.Extensive experiments on Waymo Open Dataset show our DetZero outperforms allstate-of-the-art onboard and offboard 3D detection methods. Notably, DetZeroranks 1st place on Waymo 3D object detection leaderboard with 85.15 mAPH (L2)detection performance. Further experiments validate the application of takingthe place of human labels with such high-quality results. Our empirical studyleads to rethinking conventions and interesting findings that can guide futureresearch on offboard 3D object detection.