Outperforming frontier models on emergency medicine documentation

This post details the development of a specialized AI model for emergency medicine documentation using iterative Supervised Fine-Tuning (SFT) and prompt mutation. It showcases how to build comprehensive evaluation frameworks with LLM-as-judge (Lumina) to address specific failure modes in complex medical text generation. The post highlights pipeline simplification by consolidating multiple LLM calls into fewer stages and replacing pattern-matching with deterministic code. It also demonstrates precise template modification techniques and the handling of high-risk diagnosis detection through rule-out criteria. The resulting model significantly outperforms general-purpose frontier models like gemini-2.5-pro in accuracy and latency for this domain.