Orbis – Visko's Real-Time World Model That Makes AI Video Streaming Like Live Broadcasting

1. What is Orbis

Orbis is Visko's first Live Model (real-time world model), fundamentally changing the way AI videos are generated—no longer requiring minutes or even tens of minutes of offline rendering, but instead transforming video generation into a continuous streaming output similar to live broadcasting. Users can input new instructions at any time during the generation process, and the visuals will respond in real time, altering the direction dynamically, achieving true "interactive generation." The model supports real-time streaming generation at 4K@24FPS, maintains long-term consistency across hours using a multi-scale memory mechanism, and ensures natural and realistic motion through causal temporal decomposition and physics-aware reward mechanisms. It is applicable to multiple cutting-edge scenarios such as virtual live streaming, interactive storytelling, and robot simulation.

Image source: Official article
Image source: official article

Technical positioning and domain: Orbis belongs to the field of real-time video generation and interactive world models, serving as a significant representative of the evolution of generative AI from "offline creation tools" to "real-time interactive systems." It breaks through the traditional asynchronous paradigm of "input prompt - wait - output video," compressing latency to within seconds.

Development background: The Visko team focuses on real-time interactive video generation technology. Their motivation for developing Orbis stems from insights into the bottleneck of "waiting for the final output" in scenarios such as virtual live streaming, interactive entertainment, and robot simulation. By redefining video generation as an initial value problem of physical evolution, Orbis has achieved a fundamental innovation in architecture compared to traditional bidirectional denoising methods.

Core value: This tool addresses three major core pain points in traditional video generation: the inability of users to intervene in real time, quality degradation in long video generation, and unrealistic motion physics. It transforms AI video from a "one-way output of generated content" into a "virtual world that can be interacted with in real time," providing content creators and developers with a brand-new human-computer interaction paradigm.

2. Key Features

• Real-time Streaming Generation: This is Orbis's most core capability. The output is continuous and uninterrupted, like a live stream, supporting real-time delivery at 4K@24FPS. Unlike traditional video generation tools, Orbis breaks through the limitations of "batch generation," enabling continuous video stream evolution with unlimited length.

• Interactive Generation: Users can append or modify prompts at any time during the generation process, and new instructions take effect on subsequent frames within an average of 1 second. This means creators don't need to regenerate the entire video to adjust the direction of the scene, character actions, or stylistic elements in real time, greatly enhancing creative flexibility and immediate feedback experience.

• Multimodal Input Support: Orbis supports three ways to initiate world generation: text-to-video, image-to-video, and video continuation. Users can start from a text description, a static image, or an existing video clip, all of which are handled by the same causal continuation framework, reducing the adaptation barriers between different input types.

• Long-term Consistency Maintenance: Through a bounded multi-scale memory mechanism, Orbis can maintain consistency over hours of continuous generation without noticeable quality degradation or color drift. Recent history is preserved at native latent resolution, while older history is progressively compressed, and information beyond the window is integrated into a fixed-capacity learning state, decoupling memory cost from rollout length.

• Enhanced Physical Realism: The model is based on causal decomposition and world model rewards to ensure that object movement and interaction follow physical laws. The physical perception reward alignment module introduces a frozen V-JEPA 2 latent world model to evaluate the predictive consistency of candidate frames with historical states, and jointly optimizes with visual quality rewards to guide the generation of content that aligns with real dynamics.

• Streaming Inference Optimization: On the engineering level, Orbis implements state reuse, compiled Transformer execution, multi-GPU sequence parallelism, and a progressive decoding-super-resolution-transmission pipeline. This series of inference optimizations makes real-time delivery of 4K video possible, going beyond theoretical concepts.

3. How to Use

1. Start a Session: To initiate a real-time session, visit the Playground on the Visko website or call the API. You can start a session by providing a text description, uploading an image, or pasting a video prefix. This is the starting point for interacting with Orbis, and different input methods will influence the initial visual generation direction and quality.

2. Launch Real-Time Generation: In the session interface, click the Live button to start streaming generation. At this point, the visuals will continuously evolve in real-time, like a live broadcast, without needing to wait for the full video rendering. The system automatically maintains visual consistency and physical plausibility during generation.

3. Real-Time Intervention in Visuals: During generation, users can input new prompts or voice commands at any time. New instructions will typically take effect within one second, dynamically altering the direction of subsequent visuals. This feature is ideal for scenarios requiring dynamic content adjustments, such as changing visuals in a live stream based on audience feedback or gradually refining the style during the creative process.

4. Seamless Scene Switching: Use the "Morph the scene" function to describe scene transitions. New instructions will be seamlessly integrated at the boundary of the next chunk, avoiding abrupt changes in the visuals. This mechanism relies on the design of causal temporal decomposition, ensuring that predictions for each time segment are based only on the submitted historical information.

5. Select Output Quality: Choose between 1080p, 2K, or 4K output resolutions based on your usage needs. The system delivers high-quality visuals in real-time using streaming super-resolution technology. Higher resolutions require more computational resources and bandwidth, so it is recommended to select based on the actual performance of your device.

6. Capture and Save Clips: Click the Clip button to capture and save the currently generated video segment at any time for later editing or sharing. This provides a convenient tool for material accumulation and content review, especially suitable for live stream recordings and short video creation scenarios.

7. Integrate with Your Own Applications: Developers can obtain an API Key to integrate Orbis's real-time generation capabilities into their own applications, live streaming platforms, or robot simulation workflows. The API interface offers flexible integration options, supporting real-time generation, instruction intervention, and quality output features within third-party systems.

8. Notes and Best Practices: It is recommended to plan computational resources reasonably during use, as 4K real-time generation consumes significant GPU resources. For long-generation tasks, it is advisable to plan instructions in segments, leveraging the characteristics of the "bounded multi-scale memory" mechanism, and providing new instructions at key points to guide the generation direction.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For applications requiring real-time feedback and long-running operations, such as live streaming and virtual anchors, Orbis stands out with its 4K real-time generation and hour-level consistency. Its streaming interaction capabilities are well-suited for real-time content creation directed at audiences. In contrast, Matrix Game 3.0 is better suited for game-like interactive scenarios involving keyboard/mouse control, offering higher interaction response rates, though with relatively limited visual resolution.

For offline generation tasks where visual quality and creative expression are the primary concerns, such as high-quality short films and advertising concept demonstrations, Sora and Runway Gen-3 remain the main choices. These tools do not prioritize real-time performance and instead focus more on the artistic effect and detail representation of each generated frame, making them ideal for non-time-sensitive creative workflows. Developers should select the most appropriate tool based on the specific requirements of their application regarding real-time performance, resolution, and interaction method.

6. Editor's Summary

Orbis represents a significant technological direction in the evolution of AI video generation, shifting from "asynchronous generation" to "real-time interaction." Its core innovation lies in redefining video generation as an initial value problem of physical evolution, circumventing the inherent flaws of bidirectional denoising—where "future noise contaminates current decisions"—through a causal time decomposition mechanism. This design choice directly enables real-time interaction during the generation process. The introduction of the bounded multi-scale memory mechanism is also noteworthy, as it decouples memory and computational costs from the generation length, making hour-long continuous generation feasible from an engineering standpoint, rather than simply extending the context window.

In terms of practical value, Orbis reduces video generation interaction latency from "minute-level waiting" to "second-level response." This change is not merely quantitative but represents a qualitative shift in the interaction paradigm. Previously, applications such as virtual live streaming, interactive storytelling, and robot simulation were constrained by generation speed, but now they have the technical foundation for real-time feedback. For content creators, Orbis offers a more intuitive experience akin to "directing a live performance." For developers, the API interface provides an effective pathway for integrating real-time generation capabilities.

Orbis is better suited for applications with strict real-time requirements, long continuous generation needs, and a focus on physical realism, such as building virtual live streaming systems, developing AI virtual anchors, and constructing robot simulation training platforms. For offline creation tasks that prioritize high-quality output in a single generation, traditional video generation tools remain more mature options. It is important to note that real-time generation technology requires continuous computational resources, which means its deployment and operational costs will be higher than those of offline generation solutions. This factor must be carefully considered when choosing a technical approach.

As the first real-time world model launched by Visko, Orbis demonstrates a high level of completeness in both its technical architecture and engineering implementation. However, the developer ecosystem around the model, the completeness of its toolchain, and large-scale commercial use cases still require time to mature. Its technical approach holds directional significance for the entire AI video generation field, and its future development is worth watching.

7. Application Scenarios

• Interactive Entertainment and Storytelling: Creators and audiences can choose plot branches or narrate stories in real-time, with visuals evolving instantly based on instructions, transforming traditional one-way movie-watching into an "interactive film." Suitable for interactive series, narrative segments in role-playing games, and other entertainment scenarios requiring audience participation.

• AI Live Streaming and Virtual Anchors: Orbis can serve as a real-time visual engine for virtual anchors, continuously generating live stream visuals and responding to chat comments or voice commands, without the need for pre-recorded content. In scenarios such as live e-commerce and virtual idol operations, it can dynamically adjust performance content based on audience interaction, reducing content production costs.

• Robot Simulation and Training: Train and test robot behavior in a high-fidelity virtual environment generated by Orbis, using physical perception and reward mechanisms to ensure object interactions in the virtual environment follow real-world physics. This reduces trial-and-error costs before real-world deployment, making it ideal for embodied intelligence research and autonomous driving simulation testing.

• Immersive Education and Training: Visualize abstract knowledge or historical scenarios in real-time, allowing students to instantly change the content of demonstrations through voice questions. For example, in physics education, real-time simulation of different parameter configurations for mechanics experiments, or in history classes, dynamically switching the observational perspective of historical scenes.

• E-commerce Virtual Display: Virtual influencers can dynamically showcase products in real-time generated environments, with users able to instantly request changes in style, angle, or usage context. For example, real-time display of the same clothing item in different scenarios, or adjusting the product's display angle and lighting conditions according to user preferences.

8. FAQ

Q: What is the fundamental difference between Orbis and traditional AI video generation tools?
A: Traditional tools use offline asynchronous generation methods, where users must wait for the entire video to be rendered after inputting prompts. Orbis, on the other hand, enables real-time streaming generation, with continuous output that can be intervened in real-time. This difference stems from its architecture design based on causal time decomposition, defining the generation process as an initial value problem with segment-by-segment prediction, rather than a one-time denoising process.

Q: What kind of hardware configuration is required for real-time generation at 4K@24FPS?
A: Real-time streaming generation at 4K resolution typically requires a multi-GPU parallel computing environment. Visko officially provides optimization measures for streaming inference engineering, including state reuse, compiled Transformer execution, and multi-GPU sequence parallelism. Specific hardware configuration requirements should be referenced in the official documentation. Consumer-grade GPUs may still achieve usable performance at lower resolution tiers, such as 1080p.

Q: How can long-duration generation be achieved without quality degradation?
A: Orbis maintains consistency over hours using a bounded multi-scale memory mechanism. Recent history is preserved at native latent resolution, while older history is progressively compressed. Information beyond the window is integrated into a fixed-capacity learned state, thereby decoupling memory and computational costs from the generation length. This architectural approach resolves common issues of quality degradation and content drift in long video generation.

Q: What input formats are supported? Can it generate videos based on images or videos?
A: Orbis supports three initialization methods: text-to-video, image-to-video, and video continuation. Users can provide initial input in the form of text descriptions, static images, or existing video clips. All three methods are handled by the same causal continuation framework, ensuring uniform mechanisms for continuity in the generated output.

Q: How can Orbis be integrated into your own application?
A: Developers can integrate Orbis' real-time generation capabilities into their own applications, live streaming platforms, or robot simulation workflows by obtaining an API Key. The API provides functional interfaces for session initiation, real-time generation, instruction intervention, quality selection, and clip extraction.

Q: What is the instruction response latency of Orbis?
A: The official documentation states that the average time for a new instruction to take effect on subsequent frames is approximately 1 second. This latency mainly depends on the chunk boundary mechanism and the pipeline efficiency of streaming inference. New instructions are applied to the next generated chunk, enabling real-time adjustment of the visual output.

9. Project Links

• Product Website: https://www.visko.ai/models
• Technical Paper: https://www.visko.ai/blog/papers/files/Visko_Orbis_1_0__A_Live_Model_for_Real_TimeInteractive_Long_Video_Generation.pdf