Serverless Inference Tiers

Serverless 2.0 introduces three distinct serving paths for inference: Standard, Priority, and Fast. Standard is the default, cost-efficient option suitable for development and workloads tolerating occasional throttling. Priority offers stronger admission during congestion, shedding last under load to improve reliability for production workloads. Fast provides a high-throughput path for supported models, optimizing for faster generated-token throughput. This evolution aims to provide more control over reliability and throughput without requiring reserved capacity, allowing teams to scale from shared serverless to dedicated deployments more gradually. The platform also clarifies error codes, distinguishing between account-level rate limits (429 Too Many Requests) and fleet saturation (503 Service Overloaded), enabling more appropriate retry logic and alerting.