Canary rollouts: upgrade models in production without downtime

This post introduces and details the implementation of canary rollouts for LLM inference endpoints. It explains the mechanics of staged traffic migration, including scaling up the target deployment before shifting traffic, health checks, propagation waits, and metric gates (e.g., p95 latency regression). It also contrasts canary rollouts with hard swaps and DIY staged cutovers, and provides examples of API, CLI, and Python SDK usage for creating, controlling, and managing rollouts, including pause, resume, promote, and cancel operations. The post also details the anatomy of a canary step and the differences between canary, blue-green, and rolling strategies.