Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints

Detecting 3D lanes from the camera is a rising problem for autonomousvehicles. In this task, the correct camera pose is the key to generatingaccurate lanes, which can transform an image from perspective-view to thetop-view. With this transformation, we can get rid of the perspective effectsso that 3D lanes would look similar and can accurately be fitted by low-orderpolynomials. However, mainstream 3D lane detectors rely on perfect camera posesprovided by other sensors, which is expensive and encounters multi-sensorcalibration issues. To overcome this problem, we propose to predict 3D lanes byestimating camera pose from a single image with a two-stage framework. Thefirst stage aims at the camera pose task from perspective-view images. Toimprove pose estimation, we introduce an auxiliary 3D lane task and geometryconstraints to benefit from multi-task learning, which enhances consistenciesbetween 3D and 2D, as well as compatibility in the above two tasks. The secondstage targets the 3D lane task. It uses previously estimated pose to generatetop-view images containing distance-invariant lane appearances for predictingaccurate 3D lanes. Experiments demonstrate that, without ground truth camerapose, our method outperforms the state-of-the-art perfect-camera-pose-basedmethods and has the fewest parameters and computations. Codes are available athttps://github.com/liuruijin17/CLGo.