Hidden Biases of End-to-End Driving Models

End-to-end driving systems have recently made rapid progress, in particularon CARLA. Independent of their major contribution, they introduce changes tominor system components. Consequently, the source of improvements is unclear.We identify two biases that recur in nearly all state-of-the-art methods andare critical for the observed progress on CARLA: (1) lateral recovery via astrong inductive bias towards target point following, and (2) longitudinalaveraging of multimodal waypoint predictions for slowing down. We investigatethe drawbacks of these biases and identify principled alternatives. Byincorporating our insights, we develop TF++, a simple end-to-end method thatranks first on the Longest6 and LAV benchmarks, gaining 11 driving score overthe best prior work on Longest6.