Endpoint Candidates: evaluate and promote inference endpoints | Modal Blog

Introduced 'Endpoint Candidates' for staging and evaluating alternative inference endpoint deployments against production traffic. Implemented a traffic mirroring system using a modified Pingora-based proxy with per-candidate bounded buffers and a global body data retention budget to prevent slow mirrors from impacting production performance. This allows for testing new model checkpoints or models without custom infrastructure.