VibeThinker-3B – Weibo's Open-Source 3-Billion-Parameter Dense Reasoning Model

1. What is VibeThinker-3B

VibeThinker-3B is a 3-billion-parameter dense reasoning model open-sourced by the AI team at Weibo. Built upon the Qwen2.5-Coder-3B base model, it undergoes an enhanced Spectrum-to-Signal post-training process, achieving performance levels comparable to leading large models such as Gemini 3 Pro and Claude Opus 4.5 in verifiable reasoning tasks like mathematics and programming. This model explores the performance boundaries of small models with minimal parameter size, complementing the traditional Scaling Law and offering a new path for high-performance reasoning in low-computational environments.

Technical Positioning and Domain: VibeThinker-3B belongs to the domain of natural language processing and is a reasoning-dense small model, specifically designed for verifiable reasoning tasks such as mathematics, programming, and STEM. Its core positioning is "achieving cutting-edge reasoning performance with minimal parameters," offering a differentiated approach from the mainstream large models that rely on increasing parameter scale to improve performance. The model employs a dense Transformer architecture, with no changes to the base structure; all performance improvements come from extreme optimization of the post-training technology stack.

Development Background: Developed primarily by the AI team at Weibo, which has deep expertise in natural language processing and recommendation systems. The motivation for its development stems from a reflection on the limits of the Scaling Law—while increasing parameter scale leads to rising training and deployment costs, small models still hold significant untapped potential in specific tasks. VibeThinker-3B aims to demonstrate that through refined training strategies, small models can rival large models in verifiable reasoning domains, thereby reducing the barriers to AI application.

Core Value: Addresses the critical issue of the "upper limit of small model reasoning capabilities." Traditional views suggest that reasoning ability increases monotonically with parameter scale, but VibeThinker-3B achieves a score of 94.3 in the AIME’26 mathematics competition and a Pass@1 rate of 80.2 in the LiveCodeBench programming test with just 3B parameters, outperforming many hundred-billion-parameter models. Its value lies in providing a high-performance reasoning engine for resource-constrained scenarios (such as mobile phones, IoT devices, and offline environments), while also offering a benchmark for academia to explore the reasoning limits of small models.

Technical Features: Utilizes the Spectrum-to-Signal post-training paradigm, dividing training into two stages: the "Spectrum Phase" (SFT for expanding capability coverage) and the "Signal Phase" (RLVR for precise optimization of verifiable tasks). It innovatively integrates two-phase curriculum SFT, diversity exploration distillation, MGPO reinforcement learning, and multi-domain sequential RL (Math → Code → STEM), and supports the test-time scaling (CLR) strategy, which further enhances answer reliability during inference.

2. Key Features

• Advanced Mathematical Reasoning: Achieved a score of 94.3 in the AIME’26 competition, 89.3 in HMMT’25, and 93.8 in BruMO’25. The model is capable of handling complex mathematical competition problems, generating solutions with complete reasoning chains (CoT), and supporting multi-path exploration. Its mathematical abilities are approaching or surpassing those of some human gold medalists, far exceeding models of the same parameter scale.

• Competitive Programming: Reached 80.2 Pass@1 in the LiveCodeBench v6 test, with a pass rate of up to 96.1% in the latest LeetCode weekly and biweekly contests. The model not only generates correct code but also provides detailed problem-solving strategies and complexity analysis. Its performance in programming is comparable to that of the closed-source large model Claude Opus 4.5.

• STEM Reasoning: Scored 76.4 on the IMO-AnswerBench, increasing to 80.6 when combined with the CLR strategy. It supports step-by-step derivation for scientific problems in physics, chemistry, and biology, and can handle multi-step calculations and logical reasoning. This feature is particularly important for STEM education scenarios.

• Instruction Following: Achieved a score of 74.5 on the IFBench test, with capabilities in format sensitivity and constraint checking. The model accurately understands complex instruction requirements (such as output format, length limits, and specific terminology usage), demonstrating strong controllability in both dialogue and task execution.

• Test-Time Scaling (CLR): Introduces a Claim-Level Reliability evaluation strategy, which allows for verification of each intermediate reasoning step during inference, selecting the most reliable answer path. This feature does not require retraining and only slightly increases computational overhead during inference to improve accuracy. It is especially suitable for scenarios requiring precise results, such as mathematics.

• Multi-Domain Sequential Reinforcement Learning: Conducts reinforcement learning strictly in the order of Math → Code → STEM, and experiments have shown this sequence yields the best overall performance. The model independently optimizes for each domain, avoiding interference between different tasks, while retaining a full 64K long context reasoning trajectory to ensure deep reasoning capabilities.

• Offline Self-Distillation: Filters high-quality reasoning trajectories for unified distillation, prioritizing the learning of correct paths that the student model has not yet mastered. This technique avoids the issue in traditional distillation where the teacher model's output may contain errors, improving the model's performance on challenging samples through self-comparison.

3. How to Use

1. Environment Requirements: The model is based on the Qwen2.5-Coder-3B architecture. It is recommended to use an NVIDIA GPU that supports CUDA 12.0+ (e.g., RTX 3090/4090 or A100), with a memory requirement of approximately 8GB (FP16) or 16GB (FP32). The operating system should be Linux (Ubuntu 20.04+) or Windows 10+, and Python 3.9+ and PyTorch 2.0+ must be installed.

2. Download the Model: Access the HuggingFace model repository (link to be updated once officially released). The model is published in Transformers format and can be directly loaded via the transformers library. It is recommended to use git lfs to download large files.

3. Local Deployment: Use the following Python code to load the model and start inference:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   model = AutoModelForCausalLM.from_pretrained("WeiboAI/VibeThinker-3B", torch_dtype="auto", device_map="auto")
   tokenizer = AutoTokenizer.from_pretrained("WeiboAI/VibeThinker-3B")
   

   The model supports running on consumer-grade GPUs (e.g., RTX 3090 24GB can run smoothly), and it can also perform inference on a CPU, although the speed will be slower.

4. Invoke Inference: Input mathematical, programming, or STEM-related questions, and the model will output answers with a complete reasoning chain (CoT). Example:

   prompt = "Solve: Let x be a positive integer. Find the smallest x such that x^2 + 5x + 6 is a perfect square."
   inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
   outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.9)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

5. Enable CLR (Optional): For mathematical tasks, you can enable Claim-Level reliability evaluation. This requires loading the additional CLR verification module (see clr.py in the GitHub repository), which scores the reliability of each reasoning step during inference and ultimately selects the answer path with the highest score. This feature can further improve accuracy by approximately 3-5 percentage points, but it will increase inference time by about 20%.

6. Notes: The model is optimized for verifiable reasoning tasks and may perform less effectively on unstructured tasks such as open-ended conversations or creative writing. It is recommended to use a temperature parameter of 0.6-0.8 to balance diversity and accuracy, with 0.6 for mathematical problems and 0.8 for programming problems. For long-context reasoning (more than 4096 tokens), ensure that the GPU has sufficient memory, or use gradient checkpointing techniques.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For users requiring local deployment, low computational resources, and focusing on verifiable reasoning tasks such as mathematics and programming, VibeThinker-3B is the optimal choice. Its fully open-source nature and ability to run on consumer-grade GPUs make it an ideal engine for educational, research, and edge inference scenarios. In contrast, Claude Opus 4.5 and Gemini 3 Pro excel in general-purpose tasks and long-context processing, but they rely entirely on cloud APIs, which can be costly and subject to network limitations. DeepSeek-R1-1.5B has a smaller parameter size but significantly lags in performance, making it suitable only for extremely low-computational scenarios where inference quality is not a priority. If users need to handle multimodal tasks (such as images and videos), Gemini 3 Pro is more appropriate. Overall, VibeThinker-3B stands out with a unique advantage at the intersection of "small parameters + high performance."

6. Editor's Take

VibeThinker-3B is impressive in terms of technological innovation. Its Spectrum-to-Signal post-training paradigm splits training into two distinct phases: the "spectrum phase" (coverage of capabilities) and the "signal phase" (precise optimization), offering a clear and reproducible logic. The combined use of techniques such as two-phase course SFT, diversity exploration distillation, and MGPO reinforcement learning demonstrates the team's deep understanding of training strategies. Particularly noteworthy is the experimental discovery from multi-domain sequential RL (Math → Code → STEM), which provides valuable insights for future small model training. The scaling strategy during CLR testing is a masterstroke, enabling the model to dynamically optimize during the inference phase. This "training + inference" dual-layer optimization approach holds significant academic value.

In terms of practical value, VibeThinker-3B directly lowers the barrier to using high-performance inference models. With 3B parameters, it can run on consumer-grade GPUs costing less than a thousand dollars (e.g., RTX 3060 12GB), and even some mobile SoCs (e.g., Qualcomm Snapdragon 8 Gen3) can achieve local inference through quantization. This is of great significance for scenarios such as education, programming competitions, and edge AI. However, the model's shortcomings in general tasks limit its potential as a "versatile assistant," and users must clearly understand its applicable boundaries.

The target audience includes: AI researchers (exploring the upper limits of small model inference), algorithm competition participants (assisting in problem-solving), programming learners (for problem practice guidance), STEM educators (as an intelligent tutoring tool), and developers who need to deploy inference capabilities in offline or low-computational environments.

Its future development potential is substantial. As the Spectrum-to-Signal paradigm matures, the team may expand it to more base models and a wider range of tasks. If the model can enhance its general conversation capabilities while maintaining inference performance, the VibeThinker series could become a benchmark in the small model domain. Additionally, integrating the CLR strategy with technologies such as dynamic quantization and model pruning could further improve the practicality and efficiency of small models.

7. Application Scenarios

• Algorithm Competition Preparation: Participants can use VibeThinker-3B to assist in solving math competition problems such as AIME, HMMT, and IMO. The model can output multi-path reasoning references, helping contestants understand different solution approaches and improve problem-solving efficiency. During preparation, it can serve as an intelligent sparring partner for repeated practice.

• Programming Interview Training: When job seekers practice programming problems on platforms like LeetCode and LiveCodeBench, they can use the model to obtain correct solutions with detailed explanations. The model not only provides code but also analyzes time/space complexity, edge cases, and test cases, helping users deeply understand the essence of algorithms.

• STEM Education Tutoring: Teachers or students can ask the model science questions in physics, chemistry, biology, etc., and receive step-by-step problem-solving processes. The model supports multi-step calculations and formula derivations, making it suitable as an after-school tutoring tool, especially for self-study scenarios.

• Edge Inference Deployment: With only 3B parameters, the model can be compressed to about 1.5GB through quantization (e.g., INT4), making it suitable for local operation on low-compute environments such as phones, IoT devices, and Raspberry Pi. Developers can embed it into products like educational apps and offline programming assistants, providing high-quality inference services without an internet connection.

• Reasoning Capability Research: The academic community can use VibeThinker-3B as a benchmark for exploring the upper limit of small model reasoning and study alternative paths to Scaling Law. Its fully open-source technical report and training code provide a reproducible experimental foundation for subsequent research.

8. FAQ

Q: How much performance improvement does VibeThinker-3B have compared to Qwen2.5-Coder-3B?
A: At the AIME’26 math competition, Qwen2.5-Coder-3B scored around 40 points, while VibeThinker-3B achieved 94.3 points, an improvement of over 50 points. On LiveCodeBench v6, the base model's Pass@1 is approximately 30%, whereas VibeThinker-3B reaches 80.2%. This improvement mainly comes from the extreme optimization of the post-training process in the Spectrum-to-Signal framework, including two-stage SFT, diversity distillation, and multi-domain reinforcement learning.

Q: Can the model be used for general conversation or content generation?
A: VibeThinker-3B is primarily optimized for verifiable reasoning tasks and performs generally on unstructured tasks such as open-ended dialogue and creative writing. For general-purpose capabilities, it is recommended to use Qwen2.5-3B-Instruct or similar general models. However, the model performs well in instruction-following (74.5 on IFBench) and can handle simple formatting constraint tasks.

Q: How does CLR (Claim-Level Reliability Evaluation) work specifically?
A: CLR evaluates the reliability of each reasoning step (Claim) generated by the model during the inference phase. It uses a lightweight validator (based on a fine-tuned version of the same model) to score the correctness of each step, then selects the highest-scoring complete reasoning path as the final answer. This is equivalent to "self-correction" during inference, improving accuracy by approximately 3–5 percentage points without retraining.

Q: How much GPU memory does the model require? Can it run on consumer-grade GPUs?
A: It requires approximately 8GB of memory under FP16 precision and 16GB under FP32. It can run smoothly on RTX 3090 (24GB) and RTX 4090 (24GB), and on RTX 3060 (12GB) with 4-bit quantization (about 3GB memory). CPU inference is supported, but it is significantly slower (about 30 seconds to generate 100 tokens). The official recommendation is to use NVIDIA GPUs, while AMD GPUs can be supported via ROCm.

Q: How can I reproduce the training results presented in the paper?
A: The paper and code are fully open-sourced, including the Spectrum-to-Signal training process, two-stage SFT scripts, and MGPO RL code. Users need at least four A100 80GB GPUs (or equivalent computational power) and should follow the instructions in the README file under the train/ directory in the GitHub repository. Some of the training data comes from public datasets (such as MATH, GSM8K, CodeContests, etc.), while some are internally constructed. However, the team has provided data generation scripts.

Q: Does the model support Chinese input?
A: The base model Qwen2.5-Coder-3B supports mixed Chinese and English input, and VibeThinker-3B retains this capability. It can normally process Chinese math and programming problems, but since the training data is primarily in English, its performance on Chinese tasks is slightly lower. It is recommended that users input questions in English for the best results.

9. Project Links

• GitHub repository: https://github.com/WeiboAI/VibeThinker
• HuggingFace model library: https://huggingface.co/WeiboAI/VibeThinker-3B
• arXiv technical paper: https://arxiv.org/pdf/2606.16140
• Weibo AI official organization: https://github.com/WeiboAI