Wan-Animate-2 – The Next-Generation Character Animation Model Open-Sourced by WanXiang Team

1. What is Wan-Animate-2

Wan-Animate-2 is the next-generation character animation model open-sourced by the WanXiang team, representing a major architectural upgrade from Wan-Animate. It employs an end-to-end dual-branch Diffusion Transformer (DiT) architecture, directly capturing motion priors from reference videos, and eliminates the need for explicit pose skeletons and auxiliary pose extraction networks. This model supports text-driven perspective control, offering real-time streaming variants at up to 24fps, with significant improvements in motion fidelity and identity consistency.

Image source: Official article
Image source: official article

Technical positioning and domain: Belongs to the field of computer vision and generative AI, focusing on character animation generation, specifically on motion transfer tasks based on reference images and driving videos. Its unique positioning lies in using a purely end-to-end approach, without relying on intermediate pose estimation, directly learning motion priors, and integrating text-controlled perspective, providing a more efficient technical pathway for digital humans and virtual character creation.

Development background: Developed by the WanXiang team (Wan-Video), which has deep expertise in video generation and has previously launched the Wan series of video generation models. The motivation for development was to address issues of motion extraction error accumulation and insufficient identity preservation in existing character animation methods, pushing the field toward more efficient and higher-quality solutions, while also promoting community ecosystem growth through open-sourcing.

Core value: Solves the problem of unnatural motion transfer and poor identity consistency caused by reliance on explicit pose skeletons in traditional methods. By leveraging the end-to-end DiT architecture and reference token injection, it achieves high-fidelity motion transfer and strong identity preservation. Additionally, it provides distilled and Lite versions, advancing the application of real-time streaming digital humans, reducing the threshold for digital content creation, and enhancing creative flexibility and production efficiency.

Technical features: Key technical advantages include the dual-branch DiT architecture that directly consumes driving videos, eliminating errors from intermediate representations; text normalization control enables perspective decoupling, enhancing creative controllability; the distilled version supports 10-step inference, significantly accelerating generation; and the Lite version employs teacher forcing and other techniques to pursue real-time performance. Overall, the architecture demonstrates differentiated capabilities in motion fidelity, identity consistency, and inference speed.

2. Key Features

• End-to-end Dual-branch DiT Architecture: The redesigned Diffusion Transformer directly consumes driving videos without going through an intermediate motion extractor, reducing error accumulation from the source. The dual branches process the reference image and driving video separately, and fuse them at the attention layer, simultaneously enhancing motion fidelity and identity consistency.

• Identity/Appearance Preservation Mechanism: By incorporating reference image tokens and reference video tokens into attention calculations, the appearance details of the reference character (such as facial features and clothing textures) are injected into the generation process. Compared to traditional methods, this significantly reduces issues like "the motion looks right but the face is off" or style drift, resulting in more stable outputs.

• Text-driven Perspective Control: Supports using natural language text to decouple the output camera perspective from the driving video. Users can adjust the camera position and lens angle by modifying the prompt, without altering the motion sequence, providing editing materials for scenarios such as short videos and MVs with "the same set of motions, different perspectives," greatly enhancing creative flexibility.

• Text Normalization Control (LLM Caption Preprocessing): Before inference, a fixed-format caption is generated using an LLM (e.g., Qwen3.7-Plus), describing only "character appearance + background," without mentioning actions or inferring emotions. This preprocessing normalization ensures prompt consistency and stability, facilitating batch generation and reproducibility, and reducing the need for user prompt engineering.

• Distilled Low-step Inference Mode: Provides distilled weights, supporting inference with as few as 10 steps, guidance_scale=1.0, Euler solver, and no CFG. Compared to the Base version's 40-step inference, this offers approximately 4 times faster speed while maintaining acceptable quality, suitable for quick rendering, low-latency verification, and resource-constrained scenarios.

• Lite Real-time Streaming Architecture (Preview Feature): The paper introduces Wan-Animate-2-Lite, which employs optimization techniques such as teacher forcing, error buffer, Self-Forcing distillation, and chunk-wise backpropagation to compress latency within real-time thresholds, targeting streaming character animation applications (e.g., virtual anchors in live broadcasts). This approach clearly defines the technical direction for the model's evolution into real-time digital human applications.

• Configurable Parallel Generation: Supports multi-GPU parallel inference. By default, it is tuned for 8×A800, 720P configurations, and has also been tested with 2×A800, 480P setups. Users can modify the YAML parallel configuration file to adapt to different hardware environments, balancing quality and resource efficiency while lowering deployment barriers.

3. How to Use

1. Environment Preparation and Repository Cloning: Prioritize planning according to the official default hardware specifications of 8×A800 and 720P. The official team has also tested a configuration with 2×A800 and 480P. It is not recommended to run the default parameters directly on consumer-grade single GPUs. Execute the following command to clone the repository: git clone --recursive https://github.com/Wan-Video/Wan-Animate-2.git.

2. Build the Runtime Environment: Use Python 3.11 and install torch 2.7.0 / torchvision 0.22.0 / torchaudio 2.7.0, along with CUDA 12.6. Then install requirements.txt and flash-attn, and finally execute pip install -e . to complete the environment setup. Ensure that the CUDA and PyTorch versions are compatible to avoid compatibility issues.

3. Download Model Weights: The weights are hosted on HuggingFace and ModelScope. For HuggingFace, use the command: huggingface-cli download Wan-AI/Wan2.2-Animate-2-14B --local-dir ./ckpts/; for ModelScope, use the command: modelscope download --model Wan-AI/Wan2.2-Animate-2-14B --local_dir ./ckpts/. The weight files are large (14B parameters), so ensure a stable network connection and sufficient disk space.

4. Generate Normalized Caption: Use an LLM such as Qwen3.7-Plus and write the prompt according to the official Chinese format: only describe "character appearance + background," strictly avoiding any description of actions, emotions, or evaluations. For example: "A young woman wearing a red Hanfu, with a classical-style courtyard as the background." This step is crucial for generation quality and directly affects the accurate reconstruction of the character's appearance.

5. Run Base Version Inference: Navigate to the infer directory and execute python wan_animate_2_demo.py, passing the parameters --prompt (caption text), --refer-img-file (reference image path), --refer-video-file (driver video path), and --config ./wan_animate_2.yaml. The default num_inference_steps=40 prioritizes quality.

6. Run Distilled Version Inference: Instead, use the wan_animate_2_distillation.yaml configuration file and add the parameters --sample_guide_scale 1.0 --step 10. On the Diffusers side, set guidance_scale=1.0 and flow_solver="euler". The distilled version offers faster inference speed, making it suitable for quick validation and low-latency scenarios.

7. Adjust Parallel Configuration: If the number of GPUs and VRAM differs from the official default, first modify the parallel configuration in the YAML file, then check for OOM (out of memory) errors or unusual speed. It is recommended to start testing with a lower resolution (480P) and gradually adjust to the target configuration.

8. Quick Experience (Gradio / Online Demo): Run wan_animate_2_gradio.py or wan_animate_2_gradio_distillation.py locally to launch the Gradio interface. If you prefer not to deploy locally, you can directly access the online demo on ModelScope Studio: https://www.modelscope.cn/studios/Wan-AI/Wan2.2-Animate.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For professional users seeking the highest motion fidelity and identity consistency, Wan-Animate-2's end-to-end architecture and text-based perspective control capabilities offer greater advantages, especially for short video production and digital human content requiring multi-camera generation. If hardware resources are limited or if quick integration into an existing ComfyUI workflow is needed, MusePose provides a more mature ecosystem, though users should be aware of its limitations in facial detail stability. MagicAnimate, as an early solution, can still function in simple scenarios, but its outdated technical approach is not recommended for new projects.

For Real-time Streaming Scenarios: If the goal is virtual live streaming or real-time digital human interaction, Wan-Animate-2's Lite route is currently the most promising direction, though it awaits an official release. In the interim, the distilled version can be used for low-latency verification, or consider switching to other solutions specifically designed for real-time performance (e.g., audio-driven models).

6. Editor's Summary

Wan-Animate-2 demonstrates a clear direction of technological innovation in the field of character animation. Its core value lies in completely abandoning the explicit pose skeleton as an intermediate representation, instead learning motion priors directly from driving videos through a dual-branch DiT. This design reduces information loss and error accumulation at the architectural level, making motion transfer more natural and fluid. At the same time, the reference token injection mechanism effectively alleviates the long-standing issue of identity loss, maintaining the stability of character facial and clothing details even under complex motions. The text-based perspective control function further expands the creative space, allowing users to adjust the camera as flexibly as a director, which holds practical value for scenarios requiring multi-camera materials, such as short videos and music videos.

From a practical standpoint, the model offers two inference modes: Base and Distilled, balancing quality and speed. The Lite version's technical vision clearly outlines a path toward real-time streaming evolution, pointing the way for low-latency applications such as digital human live streaming. The Apache-2.0 license also lowers the barrier for commercial integration. However, the current version has high hardware requirements, with the default configuration needing multiple A800 GPUs; consumer users may need to reduce resolution or wait for optimizations. The community ecosystem is still in its early stages, and the absence of mainstream tools like ComfyUI limits its adoption speed.

The target audience mainly includes: professional video creators, digital human developers, AI content research teams, and individuals in film previsualization and game development who have high demands for character animation quality. For general enthusiasts, it is recommended to first experience the online demo before deciding whether to deploy it locally.

In terms of future development potential, with the official release of the Lite version and the maturation of the community ecosystem, Wan-Animate-2 is expected to play a significant role in areas such as real-time digital humans, virtual live streaming, and batch production of short videos. Its end-to-end design philosophy may also inspire the architectural evolution of subsequent character animation models. Overall, this is a technically advanced, practical, and forward-looking open-source character animation tool.

7. Application Scenarios

• Virtual Hosts / Digital Human Broadcasting: The Lite route clearly targets real-time thresholds and streaming character animation, making it ideal for building low-latency live streaming setups with "real-person-driven virtual avatars." The host uses a webcam to drive the virtual character, achieving synchronized expressions and movements. Combined with text-based perspective control, camera angles can be switched, enhancing the interactivity of the live stream.

• Short Video / Short Drama Character Replacement: Use an actor's performance video as the driving video, applying it to character illustrations or reference images to quickly generate performance clips of fantastical characters or IP figures. The end-to-end motion transfer and identity preservation capabilities ensure natural character movements and stable facial expressions, making it suitable for low-cost character replacement shots and short drama series production.

• Dance, MV, and Gesture Dance Remixes: Use a dance template to drive different character avatars, enabling multi-character dance covers, simultaneous costume changes in the same frame, or stylized MVs. Text-based perspective control can also generate "same motion, different camera angles" materials, enriching editing options and significantly reducing the cost of repeated filming.

• E-commerce and Brand Content: Enable brand mascots or virtual models to showcase products using a unified motion template, suitable for product launch short videos, promotional page videos, and social media content. By standardizing caption control for appearance, different versions with various clothing and backgrounds can be generated in bulk, improving the efficiency of marketing content production.

• Game NPCs / Cutscene Previews: Game designers can first use real-person performances to quickly drive character prototypes, validating motion pacing, camera angles, and emotional expression before deciding whether to proceed with formal motion capture and rendering pipelines. This process can shorten the iteration cycle in game development and reduce early-stage production costs.

8. FAQ

Q: What are the main differences between Wan-Animate-2 and the first-generation Wan-Animate?
A: Wan-Animate-2 represents a major architectural upgrade, abandoning explicit pose skeletons and auxiliary pose extraction networks in favor of an end-to-end dual-branch DiT that directly consumes driving videos, reducing error accumulation from intermediate representations. It also introduces new features such as text-driven perspective control, distilled low-step inference, and Lite real-time streaming architecture, resulting in significant improvements in motion fidelity, identity consistency, and speed.

Q: What hardware configuration is required to run Wan-Animate-2?
A: The official recommendation is 8×A800 (80GB) GPUs and 720P resolution for inference. The official team has also tested configurations with 2×A800 and 480P resolution. Consumer-grade single GPUs (such as RTX 4090 24GB) may not be able to run the default parameters directly and would require reduced resolution or the use of the distilled version, along with adjustments to the YAML parallel configuration. It is recommended to use GPUs with at least 24GB of VRAM and to be prepared for OOM (out of memory) troubleshooting.

Q: How should one correctly write a caption (prompt)?
A: Use an LLM (e.g., Qwen3.7-Plus) to generate captions following the official fixed format: only include "character appearance description + background description," without describing actions or inferring emotions. For example: "A young woman wearing a white dress, in a park under a cherry blossom tree." It is strictly forbidden to include action or emotional descriptions such as "dancing" or "smiling," as they may interfere with the generation results.

Q: How should one choose between the Base version and the distilled version?
A: The Base version uses 40 steps for inference, prioritizing quality and is suitable for final outputs that require high detail and smoothness. The distilled version only requires 10 steps for inference, offering a speed improvement of approximately 4 times, with a slight quality trade-off. It is ideal for quick rendering, low-latency verification, and resource-constrained scenarios. It is recommended to use the distilled version for initial testing in a project and the Base version for final output.

Q: Does Wan-Animate-2 support ComfyUI or DiffSynth-Studio?
A: Currently, the official support includes a Gradio local interface and Diffusers source code integration. Integration with ComfyUI and DiffSynth-Studio is still on the todo list and has not been officially released. Community users can follow updates on the GitHub repository or develop their own plugins. Online experiences can be accessed via ModelScope Studio.

Q: What legal considerations should be taken into account for commercial use?
A: The model is licensed under Apache-2.0, allowing commercial use of the code and weights. However, the materials involved in the generated content (such as reference images and driving videos) must ensure that they have legal copyright or authorization. Use of human portraits requires consent from the individual. Additionally, platform rules (such as labeling requirements for AI-generated content on live streaming platforms) must be followed. It is recommended to consult a legal professional before commercial use.

9. Project Links

1. Product Official Website: https://humanaigc.github.io/wan-animate-2
2. Official Experience Entry (ModelScope Studio): https://www.modelscope.cn/studios/Wan-AI/Wan2.2-Animate
3. GitHub Repository: https://github.com/Wan-Video/Wan-Animate-2
4. ModelScope Model Weights: https://modelscope.cn/models/Wan-AI/Wan2.2-Animate-2-14B
5. arXiv Technical Paper: https://arxiv.org/pdf/2608.06009