Building an RL theorem-proving workflow on Modal | Modal Blog

What this post added

This post details the implementation of an RL theorem-proving workflow on Modal, leveraging its infrastructure for both GPU-intensive LLM inference (using vLLM) and CPU-bound Lean verification. It highlights the use of per-function images for distinct environments, `.map()` for parallel fan-out of ES perturbations, and Modal Sandboxes for isolated Lean verification. The post also describes a stateless checkpointing mechanism using seed histories and Volumes for base model storage, enabling efficient reconstruction of model states for ES updates.