GaitSTR: Gait Recognition with Sequential Two-stream Refinement

Gait recognition aims to identify a person based on their walking sequences,serving as a useful biometric modality as it can be observed from longdistances without requiring cooperation from the subject. In representing aperson's walking sequence, silhouettes and skeletons are the two primarymodalities used. Silhouette sequences lack detailed part information whenoverlapping occurs between different body segments and are affected by carriedobjects and clothing. Skeletons, comprising joints and bones connecting thejoints, provide more accurate part information for different segments; however,they are sensitive to occlusions and low-quality images, causinginconsistencies in frame-wise results within a sequence. In this paper, weexplore the use of a two-stream representation of skeletons for gaitrecognition, alongside silhouettes. By fusing the combined data of silhouettesand skeletons, we refine the two-stream skeletons, joints, and bones throughself-correction in graph convolution, along with cross-modal correction withtemporal consistency from silhouettes. We demonstrate that with refinedskeletons, the performance of the gait recognition model can achieve furtherimprovement on public gait recognition datasets compared with state-of-the-artmethods without extra annotations.