HyperAIHyperAI

Command Palette

Search for a command to run...

Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance

Phuc Nguyen Tuan Duc Ngo Evangelos Kalogerakis Chuang Gan Anh Tran Cuong Pham Khoi Nguyen

Abstract

We introduce Open3DIS, a novel solution designed to tackle the problem ofOpen-Vocabulary Instance Segmentation within 3D scenes. Objects within 3Denvironments exhibit diverse shapes, scales, and colors, making preciseinstance-level identification a challenging task. Recent advancements inOpen-Vocabulary scene understanding have made significant strides in this areaby employing class-agnostic 3D instance proposal networks for objectlocalization and learning queryable features for each 3D mask. While thesemethods produce high-quality instance proposals, they struggle with identifyingsmall-scale and geometrically ambiguous objects. The key idea of our method isa new module that aggregates 2D instance masks across frames and maps them togeometrically coherent point cloud regions as high-quality object proposalsaddressing the above limitations. These are then combined with 3Dclass-agnostic instance proposals to include a wide range of objects in thereal world. To validate our approach, we conducted experiments on threeprominent datasets, including ScanNet200, S3DIS, and Replica, demonstratingsignificant performance gains in segmenting objects with diverse categoriesover the state-of-the-art approaches.


Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp