HyperAIHyperAI

Command Palette

Search for a command to run...

InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling

Muhammad Gohar Javed Chuan Guo Li Cheng Xingyu Li

Abstract

Generating realistic 3D human-human interactions from textual descriptionsremains a challenging task. Existing approaches, typically based on diffusionmodels, often produce results lacking realism and fidelity. In this work, weintroduce InterMask, a novel framework for generating human interactions usingcollaborative masked modeling in discrete space. InterMask first employs aVQ-VAE to transform each motion sequence into a 2D discrete motion token map.Unlike traditional 1D VQ token maps, it better preserves fine-grainedspatio-temporal details and promotes spatial awareness within each token.Building on this representation, InterMask utilizes a generative maskedmodeling framework to collaboratively model the tokens of two interactingindividuals. This is achieved by employing a transformer architecturespecifically designed to capture complex spatio-temporal inter-dependencies.During training, it randomly masks the motion tokens of both individuals andlearns to predict them. For inference, starting from fully masked sequences, itprogressively fills in the tokens for both individuals. With its enhancedmotion representation, dedicated architecture, and effective learning strategy,InterMask achieves state-of-the-art results, producing high-fidelity anddiverse human interactions. It outperforms previous methods, achieving an FIDof 5.1545.1545.154 (vs 5.5355.5355.535 of in2IN) on the InterHuman dataset and 0.3990.3990.399 (vs5.2075.2075.207 of InterGen) on the InterX dataset. Additionally, InterMask seamlesslysupports reaction generation without the need for model redesign orfine-tuning.


Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp