Exploring Classification Equilibrium in Long-Tailed Object Detection

The conventional detectors tend to make imbalanced classification and sufferperformance drop, when the distribution of the training data is severelyskewed. In this paper, we propose to use the mean classification score toindicate the classification accuracy for each category during training. Basedon this indicator, we balance the classification via an Equilibrium Loss (EBL)and a Memory-augmented Feature Sampling (MFS) method. Specifically, EBLincreases the intensity of the adjustment of the decision boundary for the weakclasses by a designed score-guided loss margin between any two classes. On theother hand, MFS improves the frequency and accuracy of the adjustment of thedecision boundary for the weak classes through over-sampling the instancefeatures of those classes. Therefore, EBL and MFS work collaboratively forfinding the classification equilibrium in long-tailed detection, anddramatically improve the performance of tail classes while maintaining or evenimproving the performance of head classes. We conduct experiments on LVIS usingMask R-CNN with various backbones including ResNet-50-FPN and ResNet-101-FPN toshow the superiority of the proposed method. It improves the detectionperformance of tail classes by 15.6 AP, and outperforms the most recentlong-tailed object detectors by more than 1 AP. Code is available athttps://github.com/fcjian/LOCE.