Universal Instance Perception as Object Discovery and Retrieval

All instance perception tasks aim at finding certain objects specified bysome queries such as category names, language expressions, and targetannotations, but this complete field has been split into multiple independentsubtasks. In this work, we present a universal instance perception model of thenext generation, termed UNINEXT. UNINEXT reformulates diverse instanceperception tasks into a unified object discovery and retrieval paradigm and canflexibly perceive different types of objects by simply changing the inputprompts. This unified formulation brings the following benefits: (1) enormousdata from different tasks and label vocabularies can be exploited for jointlytraining general instance-level representations, which is especially beneficialfor tasks lacking in training data. (2) the unified model isparameter-efficient and can save redundant computation when handling multipletasks simultaneously. UNINEXT shows superior performance on 20 challengingbenchmarks from 10 instance-level tasks including classical image-level tasks(object detection and instance segmentation), vision-and-language tasks(referring expression comprehension and segmentation), and six video-levelobject tracking tasks. Code is available athttps://github.com/MasterBin-IIAU/UNINEXT.