Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification

Online multi-object tracking is a fundamental problem in time-critical videoanalysis applications. A major challenge in the popular tracking-by-detectionframework is how to associate unreliable detection results with existingtracks. In this paper, we propose to handle unreliable detection by collectingcandidates from outputs of both detection and tracking. The intuition behindgenerating redundant candidates is that detection and tracks can complementeach other in different scenarios. Detection results of high confidence preventtracking drifts in the long term, and predictions of tracks can handle noisydetection caused by occlusion. In order to apply optimal selection from aconsiderable amount of candidates in real-time, we present a novel scoringfunction based on a fully convolutional neural network, that shares mostcomputations on the entire image. Moreover, we adopt a deeply learnedappearance representation, which is trained on large-scale personre-identification datasets, to improve the identification ability of ourtracker. Extensive experiments show that our tracker achieves real-time andstate-of-the-art performance on a widely used people tracking benchmark.