HyperAIHyperAI

Command Palette

Search for a command to run...

InternVideo2: Scaling Foundation Models for Multimodal Video Understanding

Abstract

We introduce InternVideo2, a new family of video foundation models (ViFM)that achieve the state-of-the-art results in video recognition, video-texttasks, and video-centric dialogue. Our core design is a progressive trainingapproach that unifies the masked video modeling, crossmodal contrastivelearning, and next token prediction, scaling up the video encoder size to 6Bparameters. At the data level, we prioritize spatiotemporal consistency bysemantically segmenting videos and generating video-audio-speech captions. Thisimproves the alignment between video and text. Through extensive experiments,we validate our designs and demonstrate superior performance on over 60 videoand audio tasks. Notably, our model outperforms others on various video-relateddialogue and long video understanding benchmarks, highlighting its ability toreason and comprehend longer contexts. Code and models are available athttps://github.com/OpenGVLab/InternVideo/tree/main/InternVideo2/.


Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing

HyperAI Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp