When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving | NVIDIA Technical Blog

This post introduces and details the encode-prefill-decode (EPD) disaggregation technique within NVIDIA Dynamo for accelerating multimodal model serving. It explains the rationale behind EPD by analyzing the latency breakdown in aggregated serving for multimodal requests, highlighting how vision encoding can become a bottleneck. The post describes three encoder placement topologies: aggregated, colocated encoder, and disaggregated encoder, and discusses the trade-offs and scenarios where each is most beneficial. It quantifies the performance gains (up to 5x faster TTFT, 7x faster E2E response) and identifies key factors influencing these gains, such as input media load, output sequence length, model size/precision, and traffic mix. Specific test environment details and experimental results are provided, demonstrating the impact of quantization and placement strategies on goodput.