MiniCPM5-2B – End-to-End Native Audio Modeling Architecture Open-Sourced by RWKV and Tsinghua University

1. What is MiniCPM5-2B

MiniCPM5-2B is an end-side language foundation model open-sourced jointly by RWKV, the OpenBMB community, and Tsinghua University. With approximately 2.5 billion parameters, it supports a context length of 128K and achieves an average score of 53.9 across 34 benchmarks, surpassing many 4B-level models and reaching the state-of-the-art (SOTA) for open-source 2B-level models. The model integrates Agent capabilities such as tool calling, deep search, and code generation into the edge side, enabling operation on devices with 6GB of memory. Additionally, it open-sources training data, the RL framework Meshy, and the JustRL II algorithm, making it a powerful "compact powerhouse" for edge computing.

Technical Positioning and Domain: This model belongs to the category of end-side language foundation models in the field of natural language processing, targeting edge computing scenarios such as mobile devices, PCs, and domestic chips. It does not merely pursue parameter scale but instead leverages comprehensive data governance and reinforcement learning techniques to achieve superior overall performance within a 2B parameter capacity. It is positioned as a general-purpose foundation for localized AI applications.

Development Background: Jointly developed by RWKV, the OpenBMB open-source community, and the Natural Language Processing Lab at Tsinghua University. RWKV has extensive experience in end-side model development, and the previous MiniCPM series has already been widely adopted on terminal devices. This collaborative open-sourcing effort aims to deeply integrate Agent capabilities with end-side models, addressing the capability limitations of small models in complex tasks.

Core Value: It resolves the contradiction between the limited capabilities of end-side models and the privacy risks associated with cloud-based models. With 2.5 billion parameters, it achieves an average score of 53.9 across 34 benchmarks, outperforming several 4B-level models. In Agent capability evaluations, it scores 20, ranking first among its peers. The model natively supports tool calling, deep search, and code generation, fully compressing the core capabilities of an agent into the edge side, enabling intelligent interaction without data leaving the local environment.

Technical Features: It employs a standard Dense Transformer architecture, specifically the LlamaForCausalLM structure used in 40B-level parameter models, combined with the GQA mechanism to reduce KV Cache overhead. It natively supports a context length of 128K. A full-chain Agent capability training approach injects prior knowledge of task decomposition and tool usage during the pre-training phase. This is further enhanced by UltraX function-calling data refinement and UltraData hierarchical data governance, achieving efficient utilization of data and computational resources.

2. Key Features

• On-device Dialogue Q&A: Run the language model directly on local devices to complete tasks such as daily Q&A, writing, and summarization, without the need to upload data to the cloud. Based on a standard architecture with 2B parameters, it can smoothly operate on smartphones or regular laptops with 6GB of memory, balancing response speed and privacy security.

• Long Text Understanding: Supports a 128K context window, enabling local processing of meeting minutes, contracts, and long documents while accurately extracting key information. Thanks to the optimized KV Cache management via the GQA mechanism, memory usage during long text inference is effectively controlled, maintaining stable output even when processing documents of up to 30,000 words.

• Code Generation and Repair: From code completion to real-world repository-level fixes, the model scores 46.4 on the SWE-bench Verified benchmark. During pre-training, the model adopted a progressive governance system from L0 to L3 for code data, accumulating solid code comprehension and generation capabilities, making it suitable as an offline coding assistant.

• Tool Calling: Natively supports function calling, enabling the model to invoke external tools as needed to perform queries, calculations, and database interactions. Through 500,000 SFT trajectories covering tool calling scenarios, the model has mastered standardized function calling formats and parameter passing logic, providing a foundation for Agent workflow construction.

• Deep Search: Equipped with Agent-based search capabilities, the model can autonomously plan multi-step retrieval and integrate answers. It dynamically adjusts retrieval strategies across multiple rounds of search, combining 128K context for comprehensive analysis of multi-source information, making it suitable for complex information collection scenarios such as research and price comparison.

• Hybrid Reasoning Mode: Switch between fast response mode and deep thinking mode using the enable_thinking toggle, balancing efficiency and complex problem-solving. The shallow mode is suitable for low-latency scenarios such as everyday conversations, while the deep mode triggers long reasoning chains, paired with JustRL II algorithm's token-level credit assignment, ensuring high-quality reasoning.

• On-device Batch Processing: Privacy-sensitive tasks such as resume screening, exam grading, and contract analysis can be processed in parallel at low cost on the device side. Thanks to the standard architecture's native compatibility with mainstream frameworks like vLLM and Ollama, developers can easily build local batch processing pipelines.

3. How to Use

1. Verify hardware requirements: The BF16 original weight is approximately 4.7GiB, and the GGUF quantized version (about 1.2GB) can run on mobile phones or regular laptops with 6GB of memory. The GPU environment needs to support CUDA or ROCm, while the CPU environment relies on the AVX2 instruction set to achieve acceptable inference speed.

2. Obtain model weights: Download the required version from HuggingFace or ModelScope community, choosing from formats such as BF16, GGUF, MLX, GPTQ, SFT, etc., based on your needs. It is recommended to select the appropriate format according to the target device's memory capacity and the inference framework's support.

3. Install the inference framework: For cloud or high-concurrency scenarios, use vLLM (≥0.21.0 natively supports it, with a single card TP=1 sufficient). For local lightweight deployment, use llama.cpp, Ollama, or SGLang. For CPU environments, the Arclight framework developed by The Chinese Wall is available. The model follows the standard Llama architecture and does not require custom kernels.

4. Load and run the model: Load the model directly using the standard LlamaForCausalLM structure. No custom kernel is needed. Mainstream frameworks such as vLLM, llama.cpp, and Ollama can directly launch the service.

5. Switch thinking mode: Control between fast response and deep reasoning using the enable_thinking switch in the chat template. Choose the mode based on the task difficulty. For simple Q&A, disable this option to reduce latency. For complex reasoning tasks, enable it to obtain higher quality answers.

6. Integrate tool calling/Agent: Leverage the model's native function calling and deep search capabilities, and combine them with the accompanying deployment Cookbook and Agent Skills to build a local Agent workflow. You can refer to the example code provided by the OpenBMB community to achieve multi-tool collaboration.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: In scenarios requiring full Agent capabilities and strict device memory constraints, MiniCPM5-2B demonstrates outstanding overall performance. Its native tool calling and deep search capabilities are rare among models of similar scale, and the standard Llama architecture reduces the risk of deployment modifications. For developers who have already invested in Alibaba Cloud infrastructure and require deep integration with the Tongyi ecosystem, Qwen3.5-2B offers better ecosystem synergy.

In general conversation and code completion scenarios, Phi-4-mini still has a clear advantage in math and code-specific tasks, but its 3.8B parameter count demands higher memory on edge devices, and its context window is relatively small. Gemma 3 2B is more suitable as a standard mobile conversation model, but its Agent capabilities are relatively basic. Developers must carefully consider the specific task complexity, target hardware configuration, and privacy compliance requirements when making a choice.

6. Editor's Summary

MiniCPM5-2B demonstrates a new direction for the development of on-device models in terms of technical approach. Rather than solely pursuing expansion in parameter scale, it compensates for model capacity through the UltraData tiered data governance system and the UltraX function-calling style data refinement. Experimental data shows that training with 16B tokens of refined data achieves better performance than full training with 20B tokens of original data, validating the significant gains that high-quality data signals can bring to smaller models. The complete open-sourcing of this data engineering methodology provides the industry with a reproducible technical paradigm.

Innovations in the reinforcement learning domain are also noteworthy. The Meshy framework eliminates the need for a central controller and Ray dependency in traditional RL training, modeling training and inference as peer service nodes. This makes large-scale RL training more scalable in on-device scenarios. The JustRL II algorithm addresses two major pain points in long-chain RL for small models: "reward signals biasing the model" and "GRPO credit assignment being coarse." It introduces a Critic to enable token-level credit assignment, offering a refined optimization strategy that provides new solutions for reinforcement learning in small models.

In terms of practical value, this model fully compresses Agent capabilities previously requiring cloud computing power into on-device deployment, making intelligent applications feasible in privacy-sensitive scenarios. It supports Day0 compatibility with nine domestic chips, aligning with the diverse hardware ecosystem needs in the context of China's information innovation environment. For developers in privacy-sensitive industries, entrepreneurs in on-device AI applications, and researchers in the large model field, MiniCPM5-2B offers a foundational option that is both capable and adaptable. Continued attention is warranted for its subsequent version iterations and the development of its community ecosystem.

7. Application Scenarios

• Privacy-sensitive Document Processing: Question-answering and summarization of meeting minutes, contracts, and internal emails are completed locally, ensuring data never leaves the device. Legal and administrative teams within enterprises can process confidential documents offline, utilizing a 128K context window to handle long documents and accurately extract key clauses.

• Recruitment and Human Resources: Local batch screening of resumes, extraction of candidate information, and structured comparison, avoiding the need to upload resumes to the cloud. HR departments can build private initial screening pipelines, keeping candidate data stored on local servers, while leveraging the model's code generation capabilities to develop customized screening scripts.

• On-device Coding Assistant: Complete code completion, debugging suggestions, and even repository-level defect fixes offline, ensuring the security of code assets. Development teams working on classified projects can deploy a local coding assistant, with a SWE-bench Verified score of 46.4 demonstrating its practical value in real-world repository tasks.

• Local Deep Search: Automatically plan multi-step web searches and integrate answers, suitable for research, price comparison, and information gathering. Research institutions can build a local search Agent in isolated network environments, leveraging the model's native search planning capabilities to achieve multi-round retrieval and information integration without relying on cloud services.

• Education and Grading: Local batch processing of exam grading, homework review, and error analysis, reducing costs for schools and educational institutions. Schools can deploy the model on internal servers to process student homework and exam data in bulk, avoiding the leakage of student privacy information while providing personalized feedback.

8. FAQ

Q: How much memory is required to run MiniCPM5-2B specifically?
A: The BF16 original weight is approximately 4.7GiB, and it is recommended to run on devices with at least 8GB of memory. When using the GGUF quantized version in INT4, it requires about 1.2GB, theoretically allowing it to run on phones or laptops with 6GB of memory. Actual deployment must also consider the impact of context length and batch size on KV Cache, and it is advised to dynamically adjust using frameworks such as Ollama.

Q: What quantization formats does the model support?
A: The official release includes multiple formats such as BF16, GGUF, MLX, GPTQ, SFT, and Base. Among these, GGUF is suitable for local deployment with llama.cpp and Ollama, MLX is optimized for Apple Silicon, GPTQ is tailored for GPU quantization scenarios, and Base is the unaligned base version.

Q: How can one switch between fast response mode and deep thinking mode?
A: This is controlled via the enable_thinking boolean switch in the chat template. When set to false, the model responds quickly, suitable for everyday conversations. When set to true, it triggers long reasoning chains, improving accuracy on complex tasks. Both modes can be dynamically switched within the same deployment instance using request parameters.

Q: What inference frameworks and chip platforms are supported?
A: For cloud deployment, vLLM (≥0.21.0 natively supported) is recommended. For local deployment, llama.cpp, Ollama, and SGLang are available. The Arclight framework, developed by the model's creators, supports CPU environments. Chip compatibility includes AMD, Intel, MediaTek, Qualcomm, and domestic platforms such as Ascend, Hygon, Kunlun, and Moor Computing.

Q: How can one obtain the training data and RL framework?
A: The training data is released through the UltraData series of datasets. The RL framework Meshy and algorithm JustRL II are both open source, along with the training recipe. Developers can view the reproduction guide and configuration files in the OpenBMB/MiniCPM project on the GitHub repository.

Q: What advantages does the model have compared to Qwen3.5-2B?
A: It demonstrates significant advantages in Agent capabilities, leading the Agent leaderboard by 20 points over comparable models. The standard Llama architecture makes deployment and adaptation simpler than the Qwen series. Additionally, full-stack open source (including data and training framework) provides greater flexibility for secondary development.

Q: Can it be used for commercial projects?
A: The model is licensed under the Apache 2.0 protocol, allowing free commercial use and modification. However, note that some third-party data included in the training data still needs to comply with the original data license agreements. Before commercial use, developers should verify the specific authorization terms of the data sources.

9. Project Links

1. GitHub Repository: https://github.com/OpenBMB/MiniCPM
2. HuggingFace Model Library: https://huggingface.co/collections/openbmb/minicpm5