Detecting Human-Object Interaction via Fabricated Compositional Learning

Human-Object Interaction (HOI) detection, inferring the relationships betweenhuman and objects from images/videos, is a fundamental task for high-levelscene understanding. However, HOI detection usually suffers from the openlong-tailed nature of interactions with objects, while human has extremelypowerful compositional perception ability to cognize rare or unseen HOIsamples. Inspired by this, we devise a novel HOI compositional learningframework, termed as Fabricated Compositional Learning (FCL), to address theproblem of open long-tailed HOI detection. Specifically, we introduce an objectfabricator to generate effective object representations, and then combine verbsand fabricated objects to compose new HOI samples. With the proposed objectfabricator, we are able to generate large-scale HOI samples for rare and unseencategories to alleviate the open long-tailed issues in HOI detection. Extensiveexperiments on the most popular HOI detection dataset, HICO-DET, demonstratethe effectiveness of the proposed method for imbalanced HOI detection andsignificantly improve the state-of-the-art performance on rare and unseen HOIcategories. Code is available at https://github.com/zhihou7/HOI-CL.