PointBeV: A Sparse Approach to BeV Predictions

Bird's-eye View (BeV) representations have emerged as the de-facto sharedspace in driving applications, offering a unified space for sensor data fusionand supporting various downstream tasks. However, conventional models use gridswith fixed resolution and range and face computational inefficiencies due tothe uniform allocation of resources across all cells. To address this, wepropose PointBeV, a novel sparse BeV segmentation model operating on sparse BeVcells instead of dense grids. This approach offers precise control over memoryusage, enabling the use of long temporal contexts and accommodatingmemory-constrained platforms. PointBeV employs an efficient two-pass strategyfor training, enabling focused computation on regions of interest. At inferencetime, it can be used with various memory/performance trade-offs and flexiblyadjusts to new specific use cases. PointBeV achieves state-of-the-art resultson the nuScenes dataset for vehicle, pedestrian, and lane segmentation,showcasing superior performance in static and temporal settings despite beingtrained solely with sparse signals. We will release our code along with two newefficient modules used in the architecture: Sparse Feature Pulling, designedfor the effective extraction of features from images to BeV, and SubmanifoldAttention, which enables efficient temporal modeling. Our code is available athttps://github.com/valeoai/PointBeV.