Step Edge – The Edge Model Suite from StepFusion

1. What is Step Edge

Step Edge is StepFusion's edge model suite, comprising four core components: Base, Audio, GUI, and Gen, designed for mobile phones and automotive terminals. Through an edge-cloud collaborative architecture, the model enables agents to achieve sub-0.1-second response times locally, while ensuring full-modal data privacy. It has achieved first place in 29 key evaluations and is optimized for terminal inference with the self-developed Step Inference NPU engine.

Image source: Official article
Image source: official article

Technical positioning and domain: Step Edge belongs to the field of edge-side multi-modal large models, focusing on providing comprehensive modal understanding, generation, and automation capabilities on terminal devices such as mobile phones and cars. It differs from traditional cloud-based large models by emphasizing local inference, ultra-low latency, and data privacy protection. By integrating multi-modal understanding, audio processing, GUI agents, and image generation into a unified edge-side framework, it represents a pioneering "suite" design in the industry. This product is targeted at IoT, smart automotive, and smartphone scenarios, aiming to bring AI Agent capabilities from the cloud down to the terminal, enabling real-time, secure, and low-cost intelligent interactions.

Development background: Developed by the StepFusion team, the company has long focused on innovation in large model technologies and has accumulated substantial technical expertise in natural language processing and multi-modal domains. The launch of Step Edge stems from a deep understanding of the challenges in terminal scenarios, such as high latency of cloud models, significant privacy risks, and uncontrollable costs. It aims to promote the large-scale deployment of AI Agents on the edge. The team has self-developed the Step Inference NPU inference engine, which is fully optimized for terminal hardware from low-level operators to memory management, demonstrating their full-stack capabilities from algorithm to engineering.

Core value: Step Edge achieves sub-0.1-second local responses through its edge-cloud collaborative architecture, while ensuring the privacy and security of text, image, and audio data across all modalities. Its comprehensive suite design covers understanding, auditory, visual interaction, and image generation, offering a one-stop AI solution for terminal devices and eliminating the complexity of integrating multiple models for developers. It has ranked first in 29 key evaluations, spanning multiple dimensions such as visual question answering, speech recognition, GUI operations, and image generation, validating its leading overall performance.

2. Key Features

• Step Edge Base Model: As the core of the full-stack solution, this model supports end-to-end understanding and reasoning across multiple modalities on the device side, including text, image, video, OCR, spatial comprehension, and tool calling. It can perform tasks such as document analysis, scene recognition, and object detection offline on a smartphone, providing a general intelligent foundation for upper-layer applications, and enabling interaction with local apps through tool calling.

• Step Edge Audio: Specifically designed for on-device audio scenarios, this model integrates speech recognition (ASR) and audio understanding capabilities, supporting multiple languages such as Chinese and English. It enables voice wake-up, command recognition, and audio content Q&A in offline mode, with extremely low latency, making it ideal for automotive voice assistants and smart home devices, without relying on cloud-based voice services.

• Step Edge GUI: On-device GUI Agent, supporting visual perception, element localization, and automated operations for mobile and desktop applications. This model can understand UI elements in screenshots and perform actions such as clicking, swiping, and inputting. It has achieved leading results in evaluations such as OSWorld, and can complete high-frequency tasks like ordering takeout, sending messages, and browsing the photo album offline.

• Step Edge Gen: On-device image generation and editing model, supporting text-to-image, image-to-image, and local editing. It performs exceptionally well in evaluations such as OneIG and DPG-Bench, capable of generating high-quality images locally on a smartphone without uploading photos to the cloud, meeting users' needs for instant creation and privacy protection.

• Edge-Cloud Collaboration Architecture: Simple and high-frequency tasks are processed locally on the device, while complex, long-chain reasoning is offloaded to the cloud, achieving a balanced integration of speed, capability, and cost. Developers can configure collaboration strategies based on the scenario, ensuring real-time responsiveness while selectively invoking the powerful capabilities of cloud-based large models, reducing network dependency and bandwidth costs.

• Full-Modality Local Inference: Multi-modal data such as text, vision, and speech is processed locally on the terminal, ensuring sensitive information never leaves the device. This design is particularly important for privacy-sensitive scenarios such as finance, healthcare, and automotive, eliminating data leakage risks at the physical level.

• Step Inference NPU Engine: A self-developed on-device inference engine optimized for operators and memory usage in hardware such as smartphones and cars, reducing end-to-end latency for input modalities like text, vision, and speech. The engine supports mixed-precision inference, operator fusion, and memory reuse, achieving near-theoretical peak utilization on mainstream NPU chips, and serves as the core technology underpinning the low-latency characteristics of Step Edge.

• Low-Latency Toolcall Execution: On-device tool calling latency is as low as 0.1 seconds, supporting real-time responsiveness in high-frequency interactive scenarios. This capability allows the Agent to quickly invoke system tools such as calendar, maps, and SMS locally, completing complex task chains and reducing reliance on cloud-based links, thereby enhancing the smoothness of user experience.

3. How to Use

1. Understand Product Capabilities: Visit the official Step Edge blog (link to be updated after official release).

2. Prepare Hardware Environment: Step Edge is optimized for terminal devices equipped with NPUs. Developers must ensure that the target device (such as smartphones, vehicle infotainment systems, IoT gateways) is fitted with an NPU chip that supports the Step Inference engine, such as Qualcomm Hexagon, MediaTek APU, Huawei Ascend, etc. The specific compatibility list will be provided in the official documentation to be released later.

3. Follow Open Updates: Currently, Step Edge is in the launch and demonstration phase and has not yet opened public testing or API downloads. Developers can follow the official organization of Step Edge on platforms such as GitCode and GitHub to get the latest model weights, SDKs, and development documentation. It is recommended to subscribe to the official blog's RSS feed or join the official community to receive update notifications.

4. Apply for Beta Testing or Collaboration: Enterprise users and developers can apply for beta testing access or commercial collaboration through the contact information provided on the official website (such as email or form). Step Edge may offer limited beta testing slots, allowing partners to integrate Step Edge capabilities into their products in advance for verification.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For scenarios requiring full-modal capabilities on the edge (including GUI Agent, audio understanding, and image generation) with strict privacy requirements, Step Edge is a more suitable choice, though its availability is pending. For general multimodal understanding tasks, Qwen3-VL and InternVL2 offer mature open-source solutions, and edge deployment can be achieved through quantization, though GUI and audio capabilities require additional integration. LLaVA-1.6 has an active community and comprehensive documentation, making it ideal for rapid prototyping and academic research, but it falls short of Step Edge in terms of edge optimization and functional completeness. If the project has strict requirements for real-time performance and privacy, and the terminal hardware is relatively modern, it is recommended to prioritize tracking the open-source progress of Step Edge. If the goal is to achieve quick deployment and flexibility, using Qwen3-VL or InternVL2 with external modules is a more reliable option.

6. Editor's Summary

From the perspective of technological innovation, Step Edge's edge-cloud collaborative architecture and full-modal on-device inference design demonstrate a high level of originality in the industry. It integrates multimodal understanding, audio processing, GUI agents, and image generation into a unified on-device framework, accompanied by a self-developed NPU inference engine. This creates a complete technical stack, ranging from low-level operator optimization to high-level application interfaces. This "all-in-one" approach changes the previous situation where on-device models had limited functionality and required multiple models to be combined, thereby reducing the complexity of integration for developers. Achieving first place in 29 core evaluations also demonstrates the reliability of its overall performance, particularly excelling in niche areas such as GUI Agent and image generation.

In terms of practical value, Step Edge targets the urgent need of terminal devices such as smartphones, cars, and IoT devices for real-time AI and privacy protection. Its response latency of 0.1 seconds and the ability to process data locally enable applications like voice assistants, automated operations, and image editing to run smoothly in offline environments while avoiding the upload of sensitive information to the cloud. This is of significant value in privacy-sensitive areas such as automotive scenarios, enterprise office environments, and medical imaging. Additionally, the edge-cloud collaborative design allows complex tasks to leverage cloud capabilities, balancing lightweight responsiveness with deep inference, thus offering strong practicality.

This product primarily targets terminal device manufacturers, application developers, and AI researchers. Device manufacturers can integrate it into smartphones, automotive systems, or smart home devices to quickly gain full-modal AI capabilities; developers can use its API to build third-party applications, such as automated testing tools or offline translation assistants; researchers can draw inspiration from its edge-cloud collaborative and NPU optimization concepts to explore more efficient on-device model deployment solutions.

In terms of future development potential, as terminal chip computing power continues to improve and NPU becomes more widespread, on-device models will become one of the mainstream forms for AI deployment. As a pioneer, Step Edge has the potential to secure a favorable position in the on-device AI market if it can quickly open its ecosystem, improve documentation, and strengthen community support. However, its success also depends on its ability to attract a sufficient number of developers and partners to form a positive feedback loop. Overall, Step Edge shows forward-thinking in both its technical direction and product design, and it is worth the attention of the industry.

7. Application Scenarios

• Smartphone Assistant: Integrate Step Edge into the mobile operating system to enable on-device real-time speech recognition and GUI automation. Users can perform high-frequency tasks such as ordering takeout, sending messages, checking their photo gallery, and setting reminders through voice commands even when offline, without requiring a network connection. The response latency is at the 0.1-second level, ensuring a smooth experience while protecting personal data privacy.

• In-Vehicle Intelligent Cockpit: Deploy Step Edge in the vehicle's central control system to locally process voice commands and in-vehicle visual perception (such as driver state monitoring and gesture recognition). All data is computed within the vehicle, safeguarding driving-related data privacy, and enabling zero-latency control of navigation, air conditioning, entertainment, and other functions. Meanwhile, through on-device and cloud collaboration, real-time traffic information and other cloud-based data can be accessed.

• On-Device Image Generation: Utilize the Step Edge Gen model to perform text-to-image, image-to-image, and local editing directly on the mobile device. Users can quickly generate social media images, modify photo backgrounds, or engage in creative design without uploading photos to the cloud, meeting immediate creative needs while preventing the leakage of private images.

• Industrial Terminal Inspection: Deploy Step Edge on inspection terminals in low-network environments such as factories or outdoor settings. The model can locally identify equipment failure images (such as cracks or overheating) and audio anomalies (such as unusual noises). Analysis is completed offline, with critical information reported in real time, reducing reliance on stable network connections and improving inspection efficiency and safety.

• IoT Edge Devices: Deploy Step Edge in IoT terminals such as smart home devices and wearable technology to locally handle voice wake-up, visual recognition, and simple decision-making. For example, a smart speaker can respond to voice commands offline, and a door lock can unlock using local facial recognition. This reduces cloud communication latency and bandwidth costs while ensuring the privacy of household data.

8. FAQ

Q: Is Step Edge open source?
A: The official has not yet announced the open source license. The model weights and code are not publicly available, and only a technical blog has been released. For specific open source plans, please follow the official announcements. If open sourced, it may adopt a license similar to Apache 2.0 or a custom license, supporting both commercial and academic use.

Q: Which hardware platforms does Step Edge support?
A: Step Edge is designed for end devices such as mobile phones and cars, and comes with the Step Inference NPU engine, supporting mainstream NPU chips (such as Qualcomm Hexagon, MediaTek APU, and Huawei Ascend). The exact compatibility list and minimum computational requirements will be announced by the official, and developers are advised to follow the subsequent hardware compatibility documentation.

Q: What is the difference between Step Edge and cloud-based large models?
A: Step Edge focuses on local on-device inference, achieving sub-0.1 second response times and full-modal data privacy protection. Cloud-based models have larger parameter sizes and stronger capabilities, but they suffer from network latency and data upload risks. Step Edge employs an edge-cloud collaborative architecture, handling simple tasks locally and complex tasks via the cloud, combining the strengths of both to balance speed, capability, and cost.

Q: How can developers obtain access to Step Edge?
A: Currently, Step Edge is in the launch and showcase phase and has not yet opened public testing or API access. Developers can visit the official blog for more details and follow the official organization on platforms such as GitCode and GitHub to receive updates on open source information as soon as possible. Enterprise users can apply for beta testing or commercial collaboration through the official contact information on the website.

Q: How capable is Step Edge's GUI Agent?
A: Step Edge includes a dedicated GUI model, supporting visual perception, element localization, and automated operations for mobile and desktop applications. It has achieved leading performance in evaluations such as OSWorld. It can complete tasks like ordering takeout, sending messages, and checking the photo gallery offline, without requiring cloud assistance, resulting in extremely low latency and making it suitable for high-frequency interaction scenarios.

Q: What measures does Step Edge take for privacy protection?
A: Step Edge ensures full-modal local inference, processing all text, image, and audio data on the terminal device, keeping sensitive information on the device. The edge-cloud collaborative architecture allows users to control which tasks are uploaded to the cloud, and the cloud only handles complex inference requests after de-identification, further reducing privacy risks. Additionally, the self-developed NPU engine performs all computations locally, avoiding data interception during transmission.

9. Project Links

• Official blog/project introduction page: https://static.stepfun.com/blog/step-edge/