EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View

Multi-view aggregation promises to overcome the occlusion and misseddetection challenge in multi-object detection and tracking. Recent approachesin multi-view detection and 3D object detection made a huge performance leap byprojecting all views to the ground plane and performing the detection in theBird's Eye View (BEV). In this paper, we investigate if tracking in the BEV canalso bring the next performance breakthrough in Multi-Target Multi-Camera(MTMC) tracking. Most current approaches in multi-view tracking perform thedetection and tracking task in each view and use graph-based approaches toperform the association of the pedestrian across each view. This spatialassociation is already solved by detecting each pedestrian once in the BEV,leaving only the problem of temporal association. For the temporal association,we show how to learn strong Re-Identification (re-ID) features for eachdetection. The results show that early-fusion in the BEV achieves high accuracyfor both detection and tracking. EarlyBird outperforms the state-of-the-artmethods and improves the current state-of-the-art on Wildtrack by +4.6 MOTA and+5.6 IDF1.