HybridNets: End-to-End Perception Network

End-to-end Network has become increasingly important in multi-tasking. Oneprominent example of this is the growing significance of a driving perceptionsystem in autonomous driving. This paper systematically studies an end-to-endperception network for multi-tasking and proposes several key optimizations toimprove accuracy. First, the paper proposes efficient segmentation head andbox/class prediction networks based on weighted bidirectional feature network.Second, the paper proposes automatically customized anchor for each level inthe weighted bidirectional feature network. Third, the paper proposes anefficient training loss function and training strategy to balance and optimizenetwork. Based on these optimizations, we have developed an end-to-endperception network to perform multi-tasking, including traffic objectdetection, drivable area segmentation and lane detection simultaneously, calledHybridNets, which achieves better accuracy than prior art. In particular,HybridNets achieves 77.3 mean Average Precision on Berkeley DeepDrive Dataset,outperforms lane detection with 31.6 mean Intersection Over Union with 12.83million parameters and 15.6 billion floating-point operations. In addition, itcan perform visual perception tasks in real-time and thus is a practical andaccurate solution to the multi-tasking problem. Code is available athttps://github.com/datvuthanh/HybridNets.