Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

Throughout the history of art, the pose, as the holistic abstraction of thehuman body's expression, has proven to be a constant in numerous studies.However, due to the enormous amount of data that so far had to be processed byhand, its crucial role to the formulaic recapitulation of art-historical motifssince antiquity could only be highlighted selectively. This is true even forthe now automated estimation of human poses, as domain-specific, sufficientlylarge data sets required for training computational models are either notpublicly available or not indexed at a fine enough granularity. With the Posesof People in Art data set, we introduce the first openly licensed data set forestimating human poses in art and validating human pose estimators. It consistsof 2,454 images from 22 art-historical depiction styles, including those thathave increasingly turned away from lifelike representations of the body sincethe 19th century. A total of 10,749 human figures are precisely enclosed byrectangular bounding boxes, with a maximum of four per image labeled by up to17 keypoints; among these are mainly joints such as elbows and knees. Formachine learning purposes, the data set is divided into three subsets,training, validation, and testing, that follow the established JSON-basedMicrosoft COCO format, respectively. Each image annotation, in addition tomandatory fields, provides metadata from the art-historical online encyclopediaWikiArt. With this paper, we elaborate on the acquisition and constitution ofthe data set, address various application scenarios, and discuss prospects fora digitally supported art history. We show that the data set enables theinvestigation of body phenomena in art, whether at the level of individualfigures, which can be captured in their subtleties, or entire figureconstellations, whose position, distance, or proximity to one another isconsidered.