UGainS: Uncertainty Guided Anomaly Instance Segmentation

A single unexpected object on the road can cause an accident or may lead toinjuries. To prevent this, we need a reliable mechanism for finding anomalousobjects on the road. This task, called anomaly segmentation, can be a steppingstone to safe and reliable autonomous driving. Current approaches tackleanomaly segmentation by assigning an anomaly score to each pixel and bygrouping anomalous regions using simple heuristics. However, pixel grouping isa limiting factor when it comes to evaluating the segmentation performance ofindividual anomalous objects. To address the issue of grouping multiple anomalyinstances into one, we propose an approach that produces accurate anomalyinstance masks. Our approach centers on an out-of-distribution segmentationmodel for identifying uncertain regions and a strong generalist segmentationmodel for anomaly instances segmentation. We investigate ways to use uncertainregions to guide such a segmentation model to perform segmentation of anomalousinstances. By incorporating strong object priors from a generalist model weadditionally improve the per-pixel anomaly segmentation performance. Ourapproach outperforms current pixel-level anomaly segmentation methods,achieving an AP of 80.08% and 88.98% on the Fishyscapes Lost and Found and theRoadAnomaly validation sets respectively. Project page:https://vision.rwth-aachen.de/ugains