Baseten brings AI video to life on Nebius

This post details the technical implementation of running a custom text-to-video system on Nebius using the Baseten Inference Stack. It elaborates on optimizations at the Model Runtime layer, including modality-specific kernels, kernel fusion, attention kernels, continuous batching, request prioritization, and topology-aware parallelism. It also describes the Inference-optimized Infrastructure, focusing on intelligent request routing, geo-aware load balancing, SLA-aware autoscaling with fast cold starts, active-active reliability, and multi-cloud capacity management. The post discusses protocol selection (HTTPS, WebSockets, gRPC) for interactive workloads and outlines key performance metrics like P90/P99 latency and utilization. It details the request flow on Nebius and highlights the synergy between Nebius's capacity and Baseten's stack for predictable performance and uptime in video generation.