From weeks to a day: how we made LLM evaluation fast enough to iterate on

This post introduces the concept of 'dual indeterminacy' in LLM evaluation, distinguishing between aleatoric (task ambiguity) and epistemic (model/judge limits) uncertainty. It details the implementation of a per-sample cache for references and judge scores to achieve deterministic evaluation. The core innovation is the 'micro adapter,' a low-rank LoRA patch trained in under an hour for specific bug fixes, layered on existing adapters. Lifecycle rules for managing these micro adapters (fuse co-triggering, retrain on accumulation, unload unused) are also described. Finally, it emphasizes the necessity of end-to-end validation at the seams of the LLM system, even with component-level confidence, to catch emergent behaviors.