Leanstral 1.5 – Mistral AI's Open-Source Formal Verification Large Model

1. What is Leanstral 1.5

Leanstral 1.5 is an open-source formal verification large model from Mistral AI, deeply optimized for Lean 4 automated theorem proving. The model adopts a sparse mixture of experts (MoE) architecture with 119B parameters, activating only 6.5B parameters per token, significantly reducing inference costs while maintaining strong reasoning capabilities. It supports a 256k ultra-long context window and multimodal image-text input, achieving state-of-the-art (SOTA) performance on mathematical proof benchmarks such as miniF2F, PutnamBench, and FATE-H. The release of Leanstral 1.5 marks a critical step forward in automating formal mathematical verification and code correctness proof.

Image source: Official article
Image source: official article

Technical positioning and domain: It belongs to the intersection of natural language processing and formal verification, specifically applied to automated theorem proving (ATP) and program verification. Its unique positioning lies in deeply integrating the natural language understanding capabilities of large language models with the Lean 4 formal proof system, enabling end-to-end automatic generation from natural language propositions to verifiable formal code, filling the capability gap of general-purpose large models in strict logical verification scenarios.

R&D background: Developed by the French AI company Mistral AI, the team has accumulated deep technical expertise in efficient large model architectures (such as the Mixtral series) and open-source ecosystem building. The R&D motivation stems from the long-standing pain points in formal verification—low efficiency and high barriers to manually writing proofs—aiming to lower the threshold for using formal methods through the automation capabilities of large models, promoting their large-scale application in fields such as mathematical research and critical software verification.

Core value: It addresses two major challenges in formal proof automation: the search space explosion of complex theorems and the maintenance of long-chain logical consistency. The MoE architecture enables efficient inference, allowing ordinary researchers to leverage large models for proving difficult theorems; the 256k context window supports, for the first time, full formal verification of entire textbooks or large-scale software specifications. Its innovative value lies in elevating AI from "assisted conjecture" to the rigorous level of "verifiable proof," providing auditable and reproducible automation tools for mathematics and software engineering.

Technical features: The core advantage lies in the combination of extreme inference efficiency from the MoE architecture (only 6.5B activated parameters per token) and the 256k ultra-long context, achieving over 80% reduction in inference cost while maintaining performance comparable to a 671B-parameter dense model. Additionally, the model natively integrates a multimodal encoder, supporting joint reasoning over charts and formula images—a first in formal verification models.

2. Key Features

• Automated Theorem Proving: Automatically generates complete formal mathematical proofs based on the Lean 4 language, covering advanced mathematical fields such as algebra, geometry, number theory, and analysis. The model can directly convert mathematical propositions expressed in natural language into proof code verifiable by the Lean compiler, completing the entire process from proposition understanding to proof construction without human intervention.

• Formal Verification: Performs rigorous logical verification of software code and mathematical propositions, ensuring every step of reasoning conforms to Lean 4's type system and axiom system. This feature can be applied to correctness proofs of critical software systems, such as operating system kernels, encryption protocols, and aerospace control software, replacing time-consuming and error-prone parts of traditional manual auditing.

• Joint Image-Text Reasoning: Supports parsing and understanding multimodal inputs such as mathematical diagrams, geometric illustrations, and LaTeX formula screenshots during the proof process. By integrating a visual encoder, the model converts image features into a unified embedding representation with text, enabling cross-modal joint reasoning, particularly suitable for complex proof scenarios requiring geometric intuition.

• Long-Document Proof Generation: With a 256k context window, it can handle proof tasks at the level of entire mathematics textbooks or lengthy papers. The model maintains logical consistency across chapters and lemmas, automatically referencing previously defined definitions and proven theorems to avoid dependency breaks in long-chain reasoning.

• Agent-Based Interactive Proof: Adopts a code agent mode to autonomously invoke the Lean 4 compiler for compilation checks, iteratively refine proof strategies based on error messages, and retrieve known theorems and lemmas from mathematical libraries (e.g., mathlib4). This mode achieves an end-to-end automated proof process; users only need to input the target proposition, and the agent automatically completes the entire process of search, compilation, correction, and verification.

• Comprehensive Leadership Across Benchmarks: Achieves 100% saturation on the miniF2F benchmark (i.e., all test cases are successfully proven), solves 587/672 problems (87.4%) on PutnamBench, and attains 87% accuracy on FATE-H, all representing current public SOTA. This performance makes Leanstral 1.5 one of the strongest publicly available formal verification models to date.

3. How to Use

1. Obtain Model Weights: Visit the HuggingFace model repository (link to be updated after official release). It uses the Apache 2.0 open-source license, so you can download it directly without applying. It is recommended to use git lfs for downloading large files.

2. Configure Lean 4 Runtime Environment: Install the Lean 4 compiler (version ≥ 4.0.0) and the mathlib4 dependency library locally. It is recommended to use the elan tool to manage Lean versions and initialize the project via the lake build system. Ensure the environment variables are correctly configured so that the Lean code generated by the model can be properly parsed and verified.

3. Load the Model and Connect to the Proof Interface: Load the model weights using the Transformers library (≥4.45.0) or vLLM (≥0.6.0). It is recommended to use vLLM for inference deployment to achieve optimal performance. Convert natural language propositions into Lean 4 code via API or local scripts. It is recommended to set max_tokens to 8192 or higher to support complex proofs.

4. Launch the Automated Proof Agent: Call the model to generate an initial proof strategy. The model will automatically perform a compile-fix iteration loop. The Agent will invoke the Lean compiler to check code correctness, adjust the proof path based on error messages, and retrieve known theorems from the mathlib4 library until a proof complete status is achieved or a preset attempt limit (recommended to be set to 50) is reached.

5. Quick Online Experience: Log in to the Mistral AI Console (https://console.mistral.ai/), select the labs-leanstral-1-5 endpoint, and directly enter natural language propositions in the input box for verification. The online version supports rapid prototype verification and is suitable for lightweight usage scenarios that do not require local deployment.

Notes: Due to the large model size (119B), it is recommended to use at least 4 NVIDIA A100 (80GB) or equivalent GPUs for local inference. Using FP16 precision can reduce memory requirements to approximately 240GB. For users with consumer-grade graphics cards, it is recommended to prioritize using the online API service.

4. Pros and Cons Analysis

5. Comparison of Similar Tools

Selection Recommendations: For users seeking ultimate proof performance and long-document verification capabilities, Leanstral 1.5 is currently the optimal choice. Its 256k context and saturated miniF2F performance offer irreplaceable advantages in scenarios such as mathematical research and large-scale software verification. If budget is limited or deployment on consumer-grade hardware is required, although DeepSeek-Prover-V2 has more activated parameters, its 671B MoE architecture demands higher hardware requirements. In this case, it is recommended to prioritize the online API version of Leanstral 1.5.

For scenarios requiring only simple proof assistance or occasional verification, GPT-4o, despite its lower performance compared to specialized models, is more suitable for rapid prototype verification and informal verification needs due to its strong general capabilities and comprehensive API ecosystem. Small models like Copra are suitable for academic research and teaching scenarios, but limited by the 8k context and lower proof success rate, they are not suitable for production-grade applications.

6. Editor's Take

The emergence of Leanstral 1.5 marks a significant turning point in the field of formal verification AI. From a technological innovation perspective, Mistral AI has successfully pushed the advantages of the MoE architecture to the extreme—with a total of 119B parameters and 6.5B activated parameters, achieving 100% saturation on miniF2F. This result not only surpasses DeepSeek-Prover-V2 but also proves the feasibility of the "small activation + large capacity" approach. The introduction of a 256k context window is particularly strategic, elevating formal verification from "single theorem proving" to the level of "whole-book proving," which was previously unimaginable.

In terms of practical value, the core contribution of Leanstral 1.5 lies in lowering the barrier to formal verification. Traditionally, theorem proving with Lean 4 required deep mathematical expertise and programming experience. However, Leanstral 1.5's agent-based interactive proof mode allows users to describe propositions in natural language, with the model automatically handling the entire process of search, compilation, and correction. This means mathematicians, software engineers, and even students can use this tool for formal verification without needing to become Lean experts.

Regarding target users, Leanstral 1.5 is best suited for advanced mathematical researchers, formal verification engineers, and developers of critical software systems. For mathematical researchers, it can accelerate conjecture verification and lemma generation; for software engineers, it can replace some manual code auditing tasks. However, for ordinary developers who only need to occasionally verify simple propositions, the online API version is sufficient, and local deployment is unnecessary.

In terms of future development potential, Leanstral 1.5 opens up new directions for multimodal formal verification. The introduction of joint image-text reasoning capabilities provides new solutions for traditional challenges such as geometric proofs and diagram-assisted reasoning. As the precision of visual encoders improves and mathematical libraries continue to expand, the Leanstral series is expected to become a standard tool in the formal verification field. However, the current lack of support for other formal systems like Coq and Isabelle is a notable shortcoming, and future versions are expected to expand language support.

7. Application Scenarios

• Mathematical Research Assistance: Mathematicians can input unproven conjectures in natural language into Leanstral 1.5, which automatically generates formal proof drafts and validates them using the Lean compiler. In fields such as algebraic geometry and number theory, this tool can reduce the proof exploration cycle from weeks to hours, accelerating frontier mathematical discoveries.

• Competition Training and Problem Solving: Automatically solves high-level mathematical competition problems such as IMO and Putnam, providing complete, machine-verifiable Lean 4 formal solutions. Educational institutions can use this tool to generate standard answers and multiple proof paths, assisting competition participants in training and teaching method innovation.

• Software Correctness Verification: Generates formal specifications and verification proofs for critical software in aerospace, financial trading systems, medical device control, and other domains. Developers only need to provide a natural language description of the software specification, and Leanstral 1.5 automatically generates the corresponding Lean 4 verification code, replacing part of the manual auditing work and reducing the risk of human error.

• Automated Grading in Education: Automatically verifies student-submitted Lean 4 proof assignments in higher education, providing step-by-step feedback. The model can not only determine whether a proof is correct but also identify logical gaps, redundant steps, and areas for optimization, offering targeted improvement suggestions, significantly reducing the grading burden on instructors.

• AI Safety Alignment Verification: Used to verify the logical consistency, policy safety, and decision correctness of complex AI systems. By converting AI system behavioral specifications into Lean 4 propositions, Leanstral 1.5 can formally prove that the system will not produce harmful behaviors under given constraints, providing an auditable verification tool for the AI safety field.

8. FAQ

Q: What hardware configuration is required to run Leanstral 1.5?
A: It is recommended to use at least 4 NVIDIA A100 (80GB) or equivalent GPUs (such as H100, AMD MI250) for FP16 inference. The model has a total of 119B parameters, with FP16 weights requiring approximately 238GB of VRAM. Including KV Cache and intermediate activations, the actual requirement is around 320GB. Using INT8 quantization can reduce this to about 160GB, but it may slightly affect proof quality.

Q: How does Leanstral 1.5 compare to DeepSeek-Prover-V2?
A: On mathematical proof benchmarks, Leanstral 1.5 leads comprehensively: it achieves 100% saturation on miniF2F (DeepSeek-Prover-V2 has not disclosed saturation data) and solves 87.4% of problems on PutnamBench. More importantly, Leanstral 1.5 has lower inference costs (6.5B vs. 37B activated parameters) and supports 256k context and multimodal input, offering clear overall advantages.

Q: Does Leanstral 1.5 support input in Chinese mathematical propositions?
A: The model is primarily trained on mathematical propositions in English and LaTeX format. While it can process Chinese input, proof quality may degrade. For best results, it is recommended to input propositions in English or standard LaTeX format. For Chinese mathematics textbooks, translating them into English before input is advised.

Q: How can I verify whether a proof generated by Leanstral 1.5 is correct?
A: The Lean 4 code generated by the model must pass strict type checking and logical verification by the Lean compiler. A proof is considered correct only when the compiler outputs a proof complete status. Users do not need to understand the proof details; simply checking the compilation result is sufficient to determine correctness.

Q: Can Leanstral 1.5 be used to verify software code in non-mathematical domains?
A: Yes, but the behavior specifications of the software code must be translated into formal Lean 4 specifications. The model excels at generating Lean 4 code from natural language descriptions, so users only need to provide natural language specifications of software functionality (e.g., "a sorting function takes any list as input and returns a non-decreasing permutation"), and the model can generate the corresponding verification proof. For complex systems, it is recommended to verify modules step by step.

Q: How should the 256k context length of Leanstral 1.5 be allocated in practice?
A: It is recommended to use the first 128k tokens for inputting complete mathematical textbooks or software specification definitions, the middle 64k tokens for storing intermediate lemmas and proven theorems, and the last 64k tokens for generating proof code for the current target. This allocation maximizes the advantage of long context and ensures cross-chapter logical consistency.

9. Project Address

• Project Website: https://mistral.ai/news/leanstral-1-5/
• HuggingFace Model Library: https://huggingface.co/mistralai/Leanstral-1.5-119B-A6B
• Mistral AI Console (Online Experience): https://console.mistral.ai/
• Mistral AI Official Organization Page: https://huggingface.co/mistralai