Shared Coupling-bridge for Weakly Supervised Local Feature Learning

Sparse local feature extraction is usually believed to be of importantsignificance in typical vision tasks such as simultaneous localization andmapping, image matching and 3D reconstruction. At present, it still has somedeficiencies needing further improvement, mainly including the discriminationpower of extracted local descriptors, the localization accuracy of detectedkeypoints, and the efficiency of local feature learning. This paper focuses onpromoting the currently popular sparse local feature learning with camera posesupervision. Therefore, it pertinently proposes a Shared Coupling-bridge schemewith four light-weight yet effective improvements for weakly-supervised localfeature (SCFeat) learning. It mainly contains: i) the\emph{Feature-Fusion-ResUNet Backbone} (F2R-Backbone) for local descriptorslearning, ii) a shared coupling-bridge normalization to improve the decouplingtraining of description network and detection network, iii) an improveddetection network with peakiness measurement to detect keypoints and iv) thefundamental matrix error as a reward factor to further optimize featuredetection training. Extensive experiments prove that our SCFeat improvement iseffective. It could often obtain a state-of-the-art performance on classicimage matching and visual localization. In terms of 3D reconstruction, it couldstill achieve competitive results. For sharing and communication, our sourcecodes are available at https://github.com/sunjiayuanro/SCFeat.git.