Xiaomi-Robotics-U0 – Xiaomi's Unified Embodied Synthesis Model

1. What is Xiaomi-Robotics-U0

Xiaomi-Robotics-U0 is Xiaomi's unified embodied synthesis model with 38 billion parameters, trained continuously on the world foundation model, and jointly optimized for five major tasks: text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This model is the first high-quality multi-view scene generation model that supports cross-multiple robot forms. It outperforms GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, ranks first on the World Arena embodied video generation leaderboard, and increases the out-of-distribution success rate of π0.5 in real-world control tasks from 36.9% to 63.2%.

Image source: Official article
Image source: official article

Technical positioning and domain: Xiaomi-Robotics-U0 belongs to the intersection of embodied intelligence and multimodal generation, positioning itself as a unified embodied synthesis world model. Unlike traditional end-to-end robot control models (such as π0.5), this model focuses on generating high-quality, diverse synthetic operation data for downstream policy learning, addressing the core pain points of high data collection costs and limited scenarios in real robots.

Development background: This model was developed by Xiaomi's robotics team, initialized based on the EMU3.5 architecture. The motivation for its development stems from the common data bottleneck in the field of embodied intelligence—real-world robot operation data collection is time-consuming, annotation costs are high, and scenario diversity is insufficient, limiting the generalization ability of policy models. Xiaomi-Robotics-U0 aims to provide an expandable synthetic data engine for robot training through generative methods.

Core value: This model is the first to unify basic image/video generation capabilities with embodied generation capabilities within a single autoregressive framework, avoiding catastrophic forgetting of visual knowledge that occurs in traditional two-stage training. The generated embodied scenes and videos can be directly used to enhance the out-of-distribution generalization ability of downstream policy models. Testing has shown that it improves the success rate of π0.5 in unseen scenarios by 26.3 percentage points.

Technical features: It employs a unified autoregressive framework, where all modalities (text, image, video) share a discrete vocabulary for next-token prediction. It introduces FlashAR+ inference acceleration technology, increasing image generation speed by 82.9 times at 1024×1024 resolution. The structured control decomposition mechanism decouples scenes into five independent dimensions, supporting fine-grained, scalable editing.

2. Key Features

• Text-to-Image Generation: Supports high-quality text-to-image synthesis, preserving the visual knowledge and semantic understanding capabilities of the base world model (e.g., EMU3.5). Users can input natural language descriptions to generate high-fidelity images with a resolution of 1024×1024, suitable for visual previsualization and conceptual design in robotic scenarios.

• Image Editing: Provides Any-to-Image editing capabilities, supporting fine-grained operations such as camera perspective control, timeline control, and structure extraction. Users can provide one to three reference images along with text instructions to make precise modifications to the generated results, without needing to regenerate the entire scene.

• Embodied Scene Generation: Generates initial observation frames that strictly satisfy multi-view consistency and geometric coherence based on robot morphology descriptions (e.g., single-arm, dual-arm, wheeled) and structured scene text. This is the first high-quality multi-view scene generation feature that supports multiple robot morphologies across scenarios.

• Embodied Scene Transfer: Enables structured, controllable transfer across scenes while maintaining multi-view consistency and interactive dynamics. Users can specify target scene descriptions (e.g., changing workbench layout, lighting conditions, or background environment), and the model automatically adjusts scene elements without compromising the physical plausibility of the interaction.

• Embodied Video Generation: Supports zero-shot multi-view embodied video generation, transforming static scenes into temporally coherent robotic operation videos with 15 to 25 frames. This feature does not require task-specific fine-tuning and can directly generate dynamic sequences that include the complete operation process.

• Data Engine Application: Generated embodied scenes or video sequences can be directly fed into downstream robotic policy networks (e.g., π0.5) to enhance out-of-distribution generalization for real-world manipulation tasks. Experiments show that synthetic data generated using this model can increase policy success rates from 36.9% to 63.2%.

• Subtask-Subgoal Interleaved Learning: Decomposes robotic trajectories into subtasks using HDBSCAN clustering, generating interleaved image-text sequences that capture long-term task progress and fine-grained interactive dynamics. This feature allows the model to understand phased objectives within complex operational tasks.

• Inference Acceleration (FlashAR+): Introduces a vertical prediction head with diagonal parallel decoding, combined with vLLM optimization, achieving an 82.9× speedup in image generation at 1024×1024 resolution. This feature significantly reduces computational costs for large-scale generation tasks, making real-time or near-real-time generation feasible.

3. How to Use

1. Environment Setup: Visit the Xiaomi Robotics official website to download the model weights and inference code. The hardware environment should be configured with a high-performance GPU that supports CUDA (recommended: NVIDIA A100 or H100 series), with a minimum recommended VRAM of 80GB. The software environment requires Python 3.8+, PyTorch 2.0+, and the vLLM inference framework. It is recommended to use Ubuntu 20.04 or a higher version as the operating system.

2. Model Loading: Initialize and load the 380 billion parameter checkpoint of Xiaomi-Robotics-U0 based on the EMU3.5 architecture. Use the IBQ Tokenizer to process multimodal input sequences. This tokenizer supports 16×16 spatial compression and can efficiently encode high-resolution images into a discrete vocabulary. Example loading command:

   python load_model.py --checkpoint_path /path/to/checkpoint --tokenizer ibq
   

3. Text-to-Image Generation: Input a structured textual description, and the model directly generates high-quality images with a resolution of 1024×1024 through standard autoregressive next-token prediction. It is recommended to use detailed descriptions to improve generation accuracy, such as "A robotic arm in an industrial setting grasping a silver gear on a metal workbench, with a gray wall in the background and a light source on the left."

4. Embodied Scene Generation and Transfer: Input a description of the robot's form (e.g., "UR5e single-arm robotic arm") and a structured scene text. The model outputs an initial observation image that is consistent across multiple perspectives. For transfer, provide the current multi-perspective observation images and a target scene description. The model then generates multi-perspective RGB images after the transfer while maintaining interactive dynamics. It is recommended to use 3–5 perspectives as input to achieve the best consistency.

5. Embodied Video Generation: Provide an initial observation frame and a task instruction (e.g., "Move the red block from position A to position B"), and the model generates a temporally coherent sequence of 15 to 25 operation frames through autoregressive expansion. The diversity of the generated content can be controlled by adjusting the temperature parameter (recommended range: 0.7–0.9).

6. Inference Acceleration Configuration: To enable FlashAR+ vertical prediction heads and vLLM optimization, add the parameters --use_flashar --enable_vllm to the inference script. This configuration can achieve a speedup of 82.9 times at 1024×1024 resolution and is recommended to be enabled by default in batch generation scenarios.

7. Data Engine Application: Save the generated embodied scenes or video sequences in standard formats (e.g., MP4 or image sequences) and directly input them into downstream robot policy networks. It is recommended to perform automated quality screening on the generated data (e.g., checking multi-perspective consistency scores) to further improve training effectiveness.

Notes: Model inference requires a high amount of VRAM. It is recommended to use multi-GPU distributed inference. The first time loading the model weights will take a long time (approximately 10–15 minutes), and subsequent inferences can be accelerated using caching. The number of video frames generated can be adjusted using the --num_frames parameter, but it is recommended not to exceed 25 frames to ensure temporal coherence.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For research teams that need to generate synthetic data for robot policy training, Xiaomi-Robotics-U0 is currently the most targeted choice. Its unified framework retains general visual knowledge while providing multi-perspective consistency and structured control capabilities, making it particularly suitable for cross-form scenario transfer and out-of-distribution data enhancement. π0.5 is more appropriate for end-to-end control tasks where sufficient real-world data is already available, with its advantage lying in directly generating action instructions rather than synthetic data. GPT-Image-2.0 excels in general image generation quality but lacks support for embodied scene generation and multi-perspective consistency, making it unsuitable for robot data synthesis scenarios. For teams requiring both general image generation and embodied data generation, Xiaomi-Robotics-U0's joint training features make it a more efficient choice.

6. Editor's Summary

Xiaomi-Robotics-U0 demonstrates significant technological innovation in the field of embodied intelligence. Its core contribution lies in unifying foundational generative models with embodied generative tasks within a single autoregressive framework. By employing a joint continuous training strategy, it effectively avoids catastrophic forgetting of visual knowledge that traditionally occurs in two-stage methods. From a technical performance standpoint, the model maintains high scores on the VLM benchmark (ERQA 40.8, SEED 78.6), while its embodied scene generation capability surpasses that of GPT-Image-2.0, validating the feasibility of the unified framework. The FlashAR+ inference acceleration technology increases the image generation speed at 1024×1024 resolution by 82.9 times, addressing the efficiency bottleneck of large-scale autoregressive models in practical deployment.

From a practical value perspective, Xiaomi-Robotics-U0's role as a data engine is clearly defined. The experimental results showing that π0.5's out-of-distribution success rate in real-world manipulation tasks increased from 36.9% to 63.2% directly prove the practical value of generative data in robotic learning. The structured control decomposition mechanism (five dimensions including workspace, task object, lighting, etc.) provides fine-grained control capabilities for data augmentation, a feature that is relatively rare in existing open-source solutions.

This model is primarily targeted at embodied intelligence research teams, robotic algorithm engineers, and AI laboratories requiring large-scale synthetic training data. For individual developers or small teams, the hardware requirements imposed by its 38 billion parameters may pose a usage barrier. In the future, if Xiaomi can release a lightweight version or offer cloud-based inference services, it will significantly expand the model's user base. In terms of technical development direction, extending the video generation frame count to longer sequences (such as 50–100 frames) and supporting more complex multi-robot collaboration scene generation will be key breakthroughs in enhancing the model's practicality.

7. Application Scenarios

• Robot Data Synthesis: Generate large-scale, diverse synthetic operation trajectories for training real robots, addressing the high cost of collecting real data and the limitations of real-world scenarios. Research teams can input robot morphology descriptions and task instructions to batch generate synthetic datasets containing multi-perspective observations and sequential operation videos, which can be used for pre-training or data augmentation.

• Cross-Morphology Scene Transfer: Transfer the same operational task from one robot morphology (e.g., single-arm) to another (e.g., dual-arm), maintaining scene consistency and interactive dynamics. Robot manufacturers can use this feature to quickly generate training data compatible with different morphologies when developing various product models, reducing the cost of repeated data collection.

• Simulation Environment Construction: Rapidly generate high-quality multi-perspective robot operation scenes for training and strategy validation in simulation platforms. Developers can directly import the generated scenes into simulation environments such as MuJoCo and Isaac Gym, accelerating the iteration and testing cycle of strategy models.

• Operational Strategy Enhancement: Utilize generated out-of-distribution data to improve the generalization capability of robot strategies. Experiments have shown that this model can increase the success rate of π0.5 by 26.3 percentage points in unseen scenarios. In practical applications, it can be used to generate edge-case data where the strategy model performs poorly, enabling targeted enhancement.

• Embodied Intelligence Research: As a world-class foundational model, it supports basic research in long-term task planning, subtask decomposition, and interactive dynamic prediction. Researchers can leverage the model's capability for interleaved learning of subtasks and subgoals to analyze the phased goal structures in complex operational tasks, promoting theoretical advancements in embodied intelligence.

8. FAQ

Q: What is the difference between Xiaomi-Robotics-U0 and end-to-end control models (such as π0.5)?
A: Xiaomi-Robotics-U0 is a generative world model, focusing on generating synthetic training data for downstream policy models rather than directly controlling robots. The multi-perspective scenarios and operation videos it generates can be used as input for policy models like π0.5, enhancing their generalization capabilities in unseen environments. The two models are complementary rather than substitutable.

Q: What are the specific hardware requirements for the model?
A: Due to its 38 billion parameter scale, it is recommended to use NVIDIA A100 (80GB) or H100 series GPUs, with a minimum of 80GB of VRAM. Multi-GPU distributed inference can reduce the load on individual GPUs. Consumer-grade GPUs (such as the RTX 4090 with 24GB VRAM) cannot fully load the model. It is recommended to use FlashAR+ acceleration during inference to reduce computational overhead.

Q: What is the quality of the generated embodied videos, and can they be used directly for policy training?
A: The model supports generating temporally coherent operation videos with 15 to 25 frames and ranks first on the embodied video generation leaderboard in World Arena. Experiments show that directly inputting the generated video data into π0.5 can increase the out-of-distribution success rate from 36.9% to 63.2%, verifying its effectiveness as training data. It is recommended to perform automated quality filtering on the generated data to further improve results.

Q: Does the model support Chinese input?
A: The official documentation and demonstrations are currently based on English input. Since the model is built on the EMU3.5 architecture, it theoretically supports multiple languages. However, the quality and consistency of Chinese input have not been systematically validated yet. It is recommended to use English descriptions for the best results.

Q: How can the structured control decomposition feature be used?
A: This feature decouples the scene into five independent dimensions: workspace, task object, irrelevant object, lighting, and background. When using it, you must explicitly specify the parameters for each dimension in the input (such as lighting intensity, background type, etc.), and the model will generate a scene that meets all constraints. The usage documentation for this feature is currently incomplete; it is recommended to refer to the experimental setup in the technical paper.

9. Project Links

• Project Website: https://robotics.xiaomi.com/xiaomi-robotics-u0.html
• HuggingFace Model Library: https://huggingface.co/collections/XiaomiRobotics/xiaomi-robotics-u0
• arXiv Technical Paper: https://arxiv.org/pdf/2607.11643