Mistral AI has released Leanstral 1.5, an open-source model designed to verify mathematical proofs and check software correctness. The model achieved a 90% success rate on PutnamBench while using only one-seventh of the computing resources that Claude Opus 4.6 would have consumed.
TL;DR
- Mistral AI open-sources Leanstral 1.5, a model for verifying mathematical proofs and software correctness.
- Leanstral 1.5 achieved a 90% success rate on PutnamBench with significantly lower compute requirements than Claude Opus 4.6.
- The model is available under the Apache 2.0 license, allowing for commercial use and self-hosting.
What happened
Mistral AI has released Leanstral 1.5 as an open-source model under the Apache 2.0 license. This model is designed to verify mathematical proofs and check software correctness, and it is compatible with Lean 4, an interactive theorem prover.
Leanstral 1.5 uses a Mixture-of-Experts architecture with 119 billion total parameters, but only around 6 billion parameters are active during inference. This allows the model to use a smaller portion of its full architecture while processing each request.
According to Mistral, Leanstral 1.5 achieved a perfect score on both the validation and test portions of miniF2F, a benchmark used to evaluate formal mathematical reasoning. The model also solved 587 of the 672 problems included in PutnamBench, which tests complex reasoning and lengthy mathematical proofs.
Why it matters
Leanstral 1.5's ability to verify mathematical proofs and software correctness can significantly benefit developers and businesses, especially those with strict privacy, compliance, and data-governance requirements. The model's efficiency in computing resources makes it a cost-effective solution for complex reasoning tasks.
The open-source nature of Leanstral 1.5, combined with its permissive Apache 2.0 license, allows developers and organizations to use, modify, and deploy the model commercially. This can foster innovation and collaboration within the AI community.
The model's success in verifying Rust software automatically, as demonstrated by Mistral, highlights its potential for improving software safety and correctness. This can be particularly valuable for industries where software reliability is critical, such as aerospace, automotive, and healthcare.
Key facts
- Leanstral 1.5 is designed to work with Lean 4, an interactive theorem prover used to express and verify mathematical statements and software specifications.
- The model uses a Mixture-of-Experts architecture with 119 billion total parameters, but only around 6 billion parameters are active during inference.
- Leanstral 1.5 achieved a perfect score on both the validation and test portions of miniF2F, a benchmark used to evaluate formal mathematical reasoning.
- The model solved 587 of the 672 problems included in PutnamBench, which tests complex reasoning and lengthy mathematical proofs.
- Leanstral 1.5 required approximately one-seventh of the computing resources that Claude Opus 4.6 would have consumed while attempting the same PutnamBench workload.
- The model solved 87% of the graduate-level abstract algebra problems included in FATE-H and 34% of the doctoral-level problems in FATE-X.
- Mistral demonstrated a system that uses Leanstral 1.5 to verify Rust software automatically, detecting 47 violated properties and confirming 11 genuine software bugs across 57 open-source Rust repositories.
- Leanstral 1.5 is available under the Apache 2.0 license, allowing for commercial use and self-hosting.
Context
Mistral AI has previously released Mistral OCR 4, a document-intelligence and optical-character-recognition model. The release of Leanstral 1.5 further expands Mistral's portfolio of AI models designed to solve complex problems in various domains.
The open-source nature of Leanstral 1.5 aligns with Mistral's commitment to fostering innovation and collaboration within the AI community. By making the model freely available under the Apache 2.0 license, Mistral encourages developers and organizations to build upon and improve the model.
The success of Leanstral 1.5 in verifying mathematical proofs and software correctness highlights the growing importance of formal verification in AI. As AI models become more complex and their applications more critical, ensuring their correctness and safety becomes increasingly important.
