LingBot-VA 2.0 – AntWorld's Native World Action Model for Embodied Intelligence

1. What is LingBot-VA 2.0

LingBot-VA 2.0 is AntWorld's industry-first native world action model for embodied intelligence, pre-trained from scratch based on an autoregressive architecture, enabling robots to possess general-purpose control capabilities that allow them to "reason and act simultaneously." The model employs a causal DiT and sparse MoE architecture, with a total of 15.3B parameters but only activates 2.5B during inference. It achieves a 93.6% success rate for dual-arm tasks on the RoboTwin 2.0 benchmark, with a single-GPU inference speed of 150Hz, reaching up to 225Hz in asynchronous mode. This model is specifically designed for the physical world, from architecture, data collection to training objectives, rather than being fine-tuned from digital video generation models. It represents a significant breakthrough in real-time performance, generalization, and data efficiency.

Image source: Official article
Image source: official article

Technical positioning and domain: LingBot-VA 2.0 belongs to the category of native world action models (World Action Model) in embodied intelligence, focusing on providing robots with general and real-time visual-action control capabilities. Unlike traditional two-stage methods (first pre-training a video generation model and then fine-tuning it for control), this model directly learns the joint evolution of actions and visual data from physical interaction data, positioning itself as the core inference engine for robotic closed-loop control.

Development background: AntWorld is a research team under Ant Group specializing in robot technology and embodied intelligence. Previously, they have launched a series of models including LingBot-Depth, LingBot-VLA, and LingBot-Video, building a complete technical stack from perception to execution. The motivation behind developing LingBot-VA 2.0 is to address the pain points of existing embodied models, such as reliance on pre-trained digital video models, high inference latency, and weak generalization capabilities. The goal is to create a truly physics-world-oriented, real-time deployable general action model.

Core value: This model achieves the first-ever native pre-training from scratch for embodied intelligence, completely eliminating the dependency on large-scale internet video generation models. Through a causal autoregressive architecture and the Foresight Reasoning asynchronous inference mechanism, robots can precompute the next step while executing the current action, significantly reducing control latency. Additionally, the semantic visual-action tokenizer allows for the extraction of action supervision signals from unlabeled online videos, improving training data efficiency by 2.3 times and greatly reducing reliance on costly robot-labeled data.

Technical features: The core architecture combines causal DiT (Causal Diffusion Transformer) with sparse MoE, maintaining a total of 15.3B parameters while only activating 2.5B per inference step, balancing model capacity with inference efficiency. The proprietary Foresight Reasoning asynchronous inference mechanism increases control frequency up to 225Hz, eliminating serial delays. The semantic visual-action tokenizer implicitly extracts action information from continuous frames using inverse and forward dynamics models, enabling unlabeled videos to provide effective training signals. Furthermore, the MCP multi-step prediction training objective accelerates convergence, while consistency distillation and low-precision compilation further reduce inference latency.

2. Key Features

• Visual-Action Joint Prediction: The model simultaneously generates predictions of future world states and the next robot action, enabling "anticipatory control." Unlike traditional pipeline approaches that first predict visual frames and then plan actions, this feature unifies visual evolution and action decision-making within a single autoregressive framework, avoiding information loss in intermediate representations and allowing robots to plan ahead in dynamic environments, thereby enhancing operational smoothness.

• Long-Horizon Task Planning: With an integrated high-dimensional Planner module, the model can automatically decompose complex multi-step goals (e.g., "tidy the desk") into structured subtask sequences and supports parallel execution with both arms and state memory. This feature enables the robot to independently complete long-range household tasks, such as arranging dishes and returning items to their proper places, without relying on external task planners.

• Real-Time Closed-Loop Control: Based on the Foresight Reasoning asynchronous inference mechanism, the model predicts the next state and action in parallel while executing the current action segment, and immediately calibrates upon receiving real observations, forming a closed loop of prediction-execution-correction. This eliminates the latency introduced by traditional serial reasoning, achieving a control frequency of 150Hz (225Hz in asynchronous mode), meeting the requirements for high-speed dynamic grasping.

• Cross-Domain Generalization: The model maintains a task success rate of over 93% in both clean environments and domain-randomized scenarios, demonstrating strong robustness to variations in lighting, background, and object material. This capability stems from embodiment-native pretraining from scratch, allowing the model to truly learn general dynamic rules of the physical world rather than overfitting to specific scenarios.

• Fine-Grained Manipulation: Supports high-precision force-controlled grasping of fragile items such as chips, thin sheets, and breakable objects. The model integrates fine-grained visual servoing with force feedback, enabling it to control the gripper's force while maintaining high speed, thus preventing damage to the target object. This is applicable to scenarios such as precision assembly and food handling.

• Semantic Visual-Action Tokenizer: Unlike traditional VAEs that focus solely on pixel reconstruction, this tokenizer enforces alignment between semantic and action information during visual compression. By implicitly extracting action supervision from continuous frames using an inverse dynamics model and verifying it through a forward dynamics model, the model can effectively utilize large-scale unlabeled web videos for training. This feature significantly reduces reliance on manually annotated robot data and is a core enabler of the model's data efficiency.

3. How to Use

1. Access the official website for resources: Visit the project's official website at https://technology.robbyant.com/lingbot-va-v2 to learn about the model architecture, watch demonstration videos, and review technical details. The website provides a full model description and performance data, making it the first stop for getting started.

2. Obtain code and model weights: Download the open-source code and technical reports from the GitHub repository. The repository can be accessed via the official website or directly at https://github.com/Robbyant/lingbot-va. The code includes the causal DiT implementation, sparse MoE configuration, and the asynchronous inference pipeline for Foresight Reasoning. Model weight files must be downloaded separately; it is recommended to use Git LFS or the official Baidu Netdisk link (depending on the release channel).

3. Environment deployment and dependency installation: It is recommended to run the model on a server equipped with an NVIDIA A100 (80GB) or higher-performance GPU. The operating system should be Ubuntu 22.04, and CUDA 12.1 and PyTorch 2.1 or higher versions must be installed. Create a virtual environment using conda and install dependencies via pip install -r requirements.txt, including core libraries such as transformers, diffusers, and flash-attn. If using low-precision compilation for acceleration, also install TensorRT or ONNX Runtime.

4. Model weight loading and configuration: After downloading, place the weight files in the designated directory and initialize the model using the provided loading script. Key configuration parameters include: inference precision (recommended FP16 or INT8), number of MoE experts (default 64 experts, 2 activated), and chunk size (affects latency and throughput). It is advised to use the default configuration for the first run, then adjust settings after verifying inference accuracy.

5. Hardware integration and sensor configuration: Deploy the model onto the robot's main unit and connect the camera (recommended RGB-D camera, such as RealSense D435) and robotic arm/gripper actuator. Ensure the camera stream can provide real-time input at a minimum of 30fps, and that the actuator supports high-frequency position/torque commands. You will need to write a ROS 2 node or a custom bridge program to convert observational data into the model's input format (recommended image resolution: 224×224 or 256×256).

6. Configure the asynchronous inference pipeline: Enable the Foresight Reasoning mechanism by setting use_async=True in the code and configuring the precomputation step size (typically 2–4 steps). The model will perform the current action chunk while simultaneously computing the next step in parallel, with a separate thread managing the synchronization between prediction and observation. Note that memory usage increases in asynchronous mode; it is recommended to adjust the chunk size based on the available GPU memory.

7. Closed-loop calibration and task adaptation: Set up a real observation feedback channel so that the model recalibrates its internal prediction state with each new observation, preventing physical drift. To execute a new task, collect a small amount of (approximately 50–100) robotic operation data and use the provided fine-tuning script to perform efficient fine-tuning based on the pre-trained weights (both LoRA and full-parameter fine-tuning are supported, with LoRA recommended to maintain generalization).

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For dual-arm operation tasks requiring extreme real-time performance and generalization capabilities, LingBot-VA 2.0 is the optimal choice due to its native action architecture and asynchronous inference mechanism, especially suitable for high-frequency scenarios such as industrial dynamic grasping and precision assembly. ACE-Ego has unique advantages in single-arm operations and human video pre-training, making it ideal for teams needing rapid deployment with limited hardware resources. LingBot-VLA 2.0 is more suitable for complex task planning scenarios requiring natural language interaction, but it is less real-time than VA models. RT-2, as a general-purpose VLA baseline, excels in large model inference and language understanding, but its high inference latency makes it unsuitable for high-frequency control. Developers should choose the appropriate model based on their task requirements for real-time performance, generalization, and interaction complexity.

6. Editor's Take

LingBot-VA 2.0 demonstrates a significant paradigm shift in the field of embodied intelligence with its technological innovation. It completely abandons the traditional approach of fine-tuning digital video generation models and instead builds a native world action model from scratch. This design choice fundamentally addresses the semantic gap between video generation models and physical control. The combination of Causal DiT and sparse MoE achieves an excellent balance between model capacity and inference efficiency—15.3B total parameters with only 2.5B activated, paired with Foresight Reasoning asynchronous inference, enables a single GPU to achieve a control frequency of 150Hz, placing it at the forefront of similar models. The 93.6% success rate on dual-arm tasks in the RoboTwin 2.0 benchmark, with only a 0.4% drop in accuracy under domain randomization scenarios, fully validates its generalization robustness.

In terms of practical value, this model directly targets the core challenges of real-time closed-loop control in robotics, offering a complete solution from pre-training to deployment. The semantic visual-action tokenizer significantly reduces reliance on annotated data, allowing more teams to train using unlabeled videos and lowering the development threshold for embodied intelligence. The full-stack collaborative capabilities (with LingBot-Depth, LingBot-VLA, etc.) further enhance its integration value in complex systems.

The primary users include robotics algorithm researchers, embodied intelligence developers, and industrial automation engineers. For academic researchers, the open-source code and technical reports provide valuable references for studying causal action modeling and asynchronous inference. For industrial users, its high real-time performance and generalization capabilities can be directly applied to dynamic grasping and precision assembly lines.

The future development potential is substantial. As the asynchronous inference mechanism matures and the sparse MoE scales up, the model is expected to maintain real-time performance at larger parameter scales and gradually expand into more scenarios such as mobile manipulation and human-robot collaboration. If the community ecosystem is further refined (e.g., standardized fine-tuning tools, ROS 2 integration packages), its influence will continue to grow.

Recommendation Index: ★★★★★ (5/5)
Rating Basis: It achieves industry-leading levels in core dimensions such as embodied-native architecture, real-time inference efficiency, data efficiency, and generalization capability. Additionally, it is open-source and open, making it one of the most groundbreaking action models in the field of embodied intelligence today.

7. Application Scenarios

• Home Service Robots: Perform long-horizon household tasks such as desktop organization, object placement, and table setting. The model leverages a long-horizon task planning module to decompose complex goals into sub-task sequences, and completes multi-step operations through dual-arm coordination and state memory capabilities. For example, the robot can autonomously identify cluttered items on a table, plan the grasping sequence, and return each item to its designated location—all without human intervention.

• Industrial Dynamic Grasping: Real-time grasping of moving targets on conveyor belts and assembly lines. The model uses vision-action joint prediction to calculate the future position of objects in advance, and synchronously adjusts the robotic arm’s trajectory to achieve dynamic grasping that matches the conveyor belt speed. Compared to traditional photoelectric trigger solutions, this model adapts to variations in object shape and speed, significantly improving grasping success rates.

• Precision Assembly Tasks: High-precision force-controlled grasping and assembly of chips, thin sheets, and fragile components. The model employs fine-grained visual servoing to adjust gripper position and force in real time, maintaining non-destructive handling of delicate objects during high-speed motion. For instance, on an electronic component assembly line, the robot can accurately pick up a 0.5mm-thick ceramic substrate and place it at a designated location, with force control precision reaching 0.1N.

• Human-Robot Interactive Entertainment: Supports high-frequency interactive scenarios such as air hockey and tabletop games that require real-time prediction of opponent actions. The model responds to opponent movements at a millisecond level with an asynchronous control frequency of 225Hz, predicting the opponent’s next position and planning interception or striking strategies to deliver a smooth competitive experience. This scenario fully demonstrates the model’s potential in dynamic game-playing.

• Warehouse Logistics Sorting: Sorting, transporting, and palletizing goods in dynamically changing warehouse environments. The model adapts to packages of varying sizes, shapes, and materials, maintaining stable grasping under changing lighting conditions and occlusion through cross-domain generalization capabilities. For example, in a parcel sorting center, the robot can identify and grasp target packages one by one from an unsorted pile and place them onto the corresponding pallet.

8. FAQ

Q: Is LingBot-VA 2.0 fully open source? Where can I obtain the code and model weights?
A: The model code and technical report have been open-sourced on GitHub (link to be updated after official release) under the 2.0 license. Model weight files can be downloaded via the project's official website or the GitHub Release page. Some large files may require Git LFS. The exact open-source scope is subject to the official repository's description.

Q: What hardware configuration is required to run LingBot-VA 2.0?
A: It is recommended to use an NVIDIA A100 (80GB) or a GPU with equivalent performance, with at least 24GB of VRAM. Consumer-grade GPUs like the RTX 4090 (24GB) can run the model, but the inference speed will drop to approximately 30-50Hz, and asynchronous mode cannot be enabled. CPU inference is not feasible due to the model's heavy reliance on parallel computation. It is recommended to have at least 64GB of RAM and to reserve over 100GB of disk space for storing weights and code.

Q: What robot hardware does the model support? How to integrate it?
A: The model is decoupled from hardware and supports any robotic arm or gripper that provides a ROS 2 interface or custom API. It is recommended to use collaborative robots with force control capabilities (e.g., UR5e, Franka Emika Panda), and RGB-D cameras (e.g., RealSense D435, Azure Kinect) are advised. Integration requires writing a bridge program to convert sensor data into the model's input format and sending the output action commands to the actuator.

Q: What is the difference between LingBot-VA 2.0 and LingBot-VLA 2.0? How to choose between them?
A: LingBot-VA 2.0 is a native world action model, focusing on visual-action joint prediction and high-frequency real-time control, making it suitable for tasks requiring fast response and precise manipulation (e.g., dynamic grasping, assembly). LingBot-VLA 2.0 is a visual-language-action model, emphasizing natural language instruction understanding and complex task planning, making it suitable for scenarios involving human-robot interaction and long-term reasoning. The two models can be used in conjunction: VLA handles high-level task decomposition, while VA handles low-level action execution.

Q: How can I fine-tune the model for custom tasks? How much data is needed?
A: The model supports fine-tuning based on pre-trained weights, and it is recommended to use LoRA to maintain generalization. For simple tasks (e.g., grasping a specific object), around 50-100 demonstration data samples can yield good results; for complex tasks (e.g., multi-step assembly), 200-500 samples may be required. The data format should include continuous RGB images and corresponding robot action sequences (joint angles or end-effector poses). Fine-tuning scripts and example datasets will be available in the official repository.

Q: What is the model's inference latency? Can it meet real-time control requirements?
A: On the A100, the inference latency per chunk is approximately 142ms (corresponding to a raw inference frequency of about 7Hz). However, with the Foresight Reasoning asynchronous mechanism, the actual control frequency can reach up to 150Hz (synchronous mode) or even 225Hz (asynchronous mode). This means the model can output a new action command every 6.7ms, fully meeting the real-time control requirements for high-speed dynamic tasks (e.g., conveyor belt grasping, table tennis play).

9. Project Links

• Project Website: https://technology.robbyant.com/lingbot-va-v2
• Technical Paper and Open Source Code: https://github.com/Robbyant/lingbot-va/blob/main/LingBot_VA2_paper.pdf
• Official Organization Homepage (LingBot VA): https://github.com/Robbyant (For more model information, please follow this organization)