Planning-oriented Autonomous Driving

Modern autonomous driving system is characterized as modular tasks insequential order, i.e., perception, prediction, and planning. In order toperform a wide diversity of tasks and achieve advanced-level intelligence,contemporary approaches either deploy standalone models for individual tasks,or design a multi-task paradigm with separate heads. However, they might sufferfrom accumulative errors or deficient task coordination. Instead, we argue thata favorable framework should be devised and optimized in pursuit of theultimate goal, i.e., planning of the self-driving car. Oriented at this, werevisit the key components within perception and prediction, and prioritize thetasks such that all these tasks contribute to planning. We introduce UnifiedAutonomous Driving (UniAD), a comprehensive framework up-to-date thatincorporates full-stack driving tasks in one network. It is exquisitely devisedto leverage advantages of each module, and provide complementary featureabstractions for agent interaction from a global perspective. Tasks arecommunicated with unified query interfaces to facilitate each other towardplanning. We instantiate UniAD on the challenging nuScenes benchmark. Withextensive ablations, the effectiveness of using such a philosophy is proven bysubstantially outperforming previous state-of-the-arts in all aspects. Code andmodels are public.