HyperAIHyperAI

Command Palette

Search for a command to run...

2 months ago

Wan-Animate: Unified Character Animation and Replacement with Holistic Replication

Wan-Animate: Unified Character Animation and Replacement with Holistic
  Replication

Abstract

We introduce Wan-Animate, a unified framework for character animation andreplacement. Given a character image and a reference video, Wan-Animate cananimate the character by precisely replicating the expressions and movements ofthe character in the video to generate high-fidelity character videos.Alternatively, it can integrate the animated character into the reference videoto replace the original character, replicating the scene's lighting and colortone to achieve seamless environmental integration. Wan-Animate is built uponthe Wan model. To adapt it for character animation tasks, we employ a modifiedinput paradigm to differentiate between reference conditions and regions forgeneration. This design unifies multiple tasks into a common symbolicrepresentation. We use spatially-aligned skeleton signals to replicate bodymotion and implicit facial features extracted from source images to reenactexpressions, enabling the generation of character videos with highcontrollability and expressiveness. Furthermore, to enhance environmentalintegration during character replacement, we develop an auxiliary RelightingLoRA. This module preserves the character's appearance consistency whileapplying the appropriate environmental lighting and color tone. Experimentalresults demonstrate that Wan-Animate achieves state-of-the-art performance. Weare committed to open-sourcing the model weights and its source code.

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp
Wan-Animate: Unified Character Animation and Replacement with Holistic Replication | Papers | HyperAI