Introducing Dedicated Container Inference: Delivering 2.6x faster inference for custom AI models

This post introduces Dedicated Container Inference (DCI) as a new capability for deploying custom generative media models. DCI is built around job orchestration for containers, enabling multiple independent queues, policy-driven traffic control, isolation between batch, real-time, and untrusted traffic, and predictable behavior during spikes. It supports deploying Docker containers with inference logic, managing deployment, autoscaling, queuing, and monitoring at the job level. Key architectural components include volume mounts for model weights, inference running as jobs, autoscaling driven by queue depth or custom metrics, explicit traffic policies, and built-in observability. The post highlights performance gains seen by customers, with up to 2.6x inference speedups for video generation models.