Leanabell-Prover-V2: Advancing Formal Theorem Proving with Verifier-Aware Reinforcement Learning
Table of Contents
Introduction
The Landscape of Formal Theorem Proving
What Makes Leanabell-Prover-V2 Unique
Background: From V1 to V2
Model Architecture and Core Components
Reinforcement Learning in Formal Proving
Verifier Feedback: Making the Model Self-Aware
Long Chain-of-Thoughts in Lean 4
Training Strategies and Feedback Token Masking
Simple but Effective Reward Mechanisms
Evaluation on MiniF2F
Comparison with Other Prover Models
Technical Innovations
Codebase and Open Source Accessibility
Challenges in Integrating with Lean 4
Real-World Use Cases
Implications for Education and Research
Future Directions
Conclusion
1. Introduction
Formal theorem proving is a foundational task in the intersection of logic, mathematics, and computer science. Traditional methods rely on human experts to write formally verified proofs using theorem provers like Lean, Coq, or Isabelle. While accurate, this process is time-consuming and not scalable.
Leanabell-Prover-V2 enters as a breakthrough solution: a 7B parameter large language model (LLM) designed specifically to generate verifiable Lean 4 proofs using advanced Reinforcement Learning (RL) with direct feedback from the Lean 4 verifier.
This article offers a deep dive into the architecture, innovations, and real-world significance of Leanabell-Prover-V2.
2. The Landscape of Formal Theorem Proving
Tools like Lean 4 are widely used in formalizing mathematics and verifying software correctness. But even with these tools, formalizing a simple theorem can require hours of manual effort.
Recently, large language models (LLMs) have shown potential in automating parts of this pipeline. However, prior models often hallucinate invalid proofs, and lack integration with the verifier — the key arbiter of correctness.
3. What Makes Leanabell-Prover-V2 Unique
Leanabell-Prover-V2 is one of the first large-scale models to be tightly coupled with a formal verifier in the training loop. Key innovations include:
Verifier-integrated Reinforcement Learning
Long Chain-of-Thought (CoT) reasoning generation
Multi-turn feedback-aware correction
Feedback token masking for stable training
Simple, interpretable reward design
4. Background: From V1 to V2
The first version, Leanabell-Prover-V1, introduced a new approach to post-train open-source LLMs (like LLaMA or Qwen) to produce formal proofs in Lean. However, it lacked dynamic feedback from the verifier during training.
V2 builds on this by upgrading the RL loop, incorporating Lean 4 feedback, and refining reward strategies. The result is a model that learns from its own mistakes, improves its proof generation capabilities, and closes the gap with top-tier models like DeepSeek-Prover-V2-7B.
5. Model Architecture and Core Components
Leanabell-Prover-V2 is based on a 7B parameter transformer architecture with the following components:
Token encoder/decoder: Learns syntax and semantics of Lean 4.
Chain-of-Thought (CoT) generator: Produces multi-line logical steps.
Verifier hook: Executes proofs and returns success/failure or error logs.
RL trainer: Optimizes policies using verifier feedback.
Feedback masking unit: Stabilizes the learning process.
6. Reinforcement Learning in Formal Proving
Traditional supervised learning (SFT) cannot fully capture the trial-and-error nature of proof construction. Hence, V2 uses RL to:
Explore new proof paths
Learn from failed or partially correct attempts
Align model behavior with verifiable success
The reward signal is directly tied to proof validity, as confirmed by the Lean 4 engine.
7. Verifier Feedback: Making the Model Self-Aware
This is the core innovation of V2.
Feedback Mechanism:
Model generates a full proof.
Lean 4 verifier runs the proof script.
If it fails, it returns:
Type errors
Tactic mismatches
Unresolved goals
The model then reflects on these errors and attempts to revise the proof.
This dynamic self-correction loop simulates how human mathematicians refine their work — a major step forward in LLM reasoning.
8. Long Chain-of-Thoughts in Lean 4
Unlike shallow models that guess one-liner proofs, Leanabell-Prover-V2 produces multi-turn reasoning sequences:
lean theorem add_zero (n : Nat) : n + 0 = n := by induction n with | zero => rfl | succ n ih => simp [Nat.add_succ, ih]
Each tactic, lemma, and step is part of a structured CoT, which significantly improves verifiability and readability.
9. Training Strategies and Feedback Token Masking
To stabilize RL training, the authors implemented feedback token masking:
Mask error tokens in the prompt
Guide the model to focus on revising faulty segments
Reduce gradient noise during training
This results in faster convergence and more robust proof generation.
10. Simple but Effective Reward Mechanisms
Unlike complex reward engineering, V2 uses binary and token-level rewards:
+1 for successful proof
0 for invalid or partial output
Token-level guidance from successful segments
This simplicity ensures interpretability and avoids overfitting.
11. Evaluation on MiniF2F
The MiniF2F dataset evaluates mathematical reasoning in Lean. V2 shows:
+3.2% improvement over Kimina-Prover-Distill-7B
+2.0% over DeepSeek-Prover-V2-7B
| Model | MiniF2F Pass@128 |
|---|---|
| Leanabell-Prover-V2 | +3.2%↑ |
| Kimina-Prover-Preview-7B | Baseline |
| DeepSeek-Prover-V2-7B | +2.0%↑ |
This confirms that verifier feedback significantly boosts performance.
12. Comparison with Other Prover Models
Leanabell-Prover-V2 outperforms many models in Lean proof generation due to:
Tighter coupling with Lean 4
Feedback-aware RL
Longer and more precise CoT sequences
It’s more efficient than autoformalization-only models and more accurate than prompt-only proof generators.
13. Technical Innovations
Some notable innovations include:
Verifier feedback as part of the RL environment
Feedback masking for stable optimization
Tactic-level action space
Multi-step trajectory evaluation
Minimal reliance on prompt engineering
14. Codebase and Open Source Accessibility
The full code, training scripts, and checkpoints are available at:
👉 https://github.com/Leanabell-LM/Leanabell-Prover-V2
Includes:
🧪 Training data (formal/informal pairs)
🧱 Dockerized Lean 4 environments
🧠 Reinforcement learning training loop
✅ Pretrained 7B checkpoint for fine-tuning
15. Challenges in Integrating with Lean 4
Training with a verifier-in-the-loop is non-trivial:
Latency from Lean 4 verification
Diverse error formats
The need to reset environments per proof
Debugging RL crashes during invalid episodes
But Leanabell’s team engineered robust sandboxing, error classifiers, and asynchronous queues to overcome this.
16. Real-World Use Cases
Leanabell-Prover-V2 has applications in:
🧮 Formal math education (auto-grading student proofs)
🛡️ Verified software development (proof assistants)
📚 Knowledge graph formalization
🧠 AI agents with provable reasoning
🧑🏫 Lean 4 learning tools and tutors
17. Implications for Education and Research
This model offers:
🏫 Curriculum tools for Lean 4 courses
🧑🎓 Research assistants for theorem formalization
🧠 Automated insights into proof strategies
🌐 Multilingual expansion possibilities (future work)
It makes formal logic accessible at scale — a huge leap for educational equity.
18. Future Directions
Planned improvements:
🔍 Integrate with Coq and Isabelle for multi-prover compatibility
🧩 Add symbolic reasoning tools (e.g., Lean tactics recommender)
📊 Improve pass@1 performance via smarter rewards
🌍 Expand multilingual prompts
🚀 Scale to 13B+ parameter checkpoints
19. Conclusion
Leanabell-Prover-V2 stands as a landmark achievement in formal theorem proving using LLMs. By embedding verifier feedback directly into RL training, it creates self-correcting, verifiable, and highly capable proof-generating models.
This work not only improves mathematical reasoning in machines but also offers a blueprint for combining LLMs with formal systems in any discipline — from math and code to law and science.
