Cross-modal Local Shortest Path and Global Enhancement for Visible-Thermal Person Re-Identification

In addition to considering the recognition difficulty caused by human postureand occlusion, it is also necessary to solve the modal differences caused bydifferent imaging systems in the Visible-Thermal cross-modal personre-identification (VT-ReID) task. In this paper,we propose the Cross-modalLocal Shortest Path and Global Enhancement (CM-LSP-GE) modules,a two-streamnetwork based on joint learning of local and global features. The core idea ofour paper is to use local feature alignment to solve occlusion problem, and tosolve modal difference by strengthening global feature. Firstly,Attention-based two-stream ResNet network is designed to extract dual-modalityfeatures and map to a unified feature space. Then, to solve the cross-modalperson pose and occlusion problems, the image are cut horizontally into severalequal parts to obtain local features and the shortest path in local featuresbetween two graphs is used to achieve the fine-grained local feature alignment.Thirdly, a batch normalization enhancement module applies global features toenhance strategy, resulting in difference enhancement between differentclasses. The multi granularity loss fusion strategy further improves theperformance of the algorithm. Finally, joint learning mechanism of local andglobal features is used to improve cross-modal person re-identificationaccuracy. The experimental results on two typical datasets show that our modelis obviously superior to the most state-of-the-art methods. Especially, onSYSU-MM01 datasets, our model can achieve a gain of 2.89%and 7.96% in allsearch term of Rank-1 and mAP. The source code will be released soon.