PoseTrack: Joint Multi-Person Pose Estimation and Tracking

In this work, we introduce the challenging problem of joint multi-person poseestimation and tracking of an unknown number of persons in unconstrainedvideos. Existing methods for multi-person pose estimation in images cannot beapplied directly to this problem, since it also requires to solve the problemof person association over time in addition to the pose estimation for eachperson. We therefore propose a novel method that jointly models multi-personpose estimation and tracking in a single formulation. To this end, we representbody joint detections in a video by a spatio-temporal graph and solve aninteger linear program to partition the graph into sub-graphs that correspondto plausible body pose trajectories for each person. The proposed approachimplicitly handles occlusion and truncation of persons. Since the problem hasnot been addressed quantitatively in the literature, we introduce a challenging"Multi-Person PoseTrack" dataset, and also propose a completely unconstrainedevaluation protocol that does not make any assumptions about the scale, size,location or the number of persons. Finally, we evaluate the proposed approachand several baseline methods on our new dataset.