DynoSim: Simulating the Pareto Frontier | NVIDIA Technical Blog

What this post added

Introduced DynoSim, a workload-driven discrete-event simulation of the NVIDIA Dynamo serving stack. DynoSim combines measured engine forward-pass timing, scheduler cores, router and planner behavior, and KV cache effects on a virtual timeline to rapidly explore the Pareto frontier of LLM serving configurations. It achieves high simulation speed (e.g., 1500x real time) by composing components as actors on a discrete-event timeline, enabling the screening of thousands of configurations before hardware validation. The simulation architecture includes a replay harness, single-engine simulation with scheduler fidelity (e.g., vLLM, SGLang) informed by AIConfigurator (AIC) for timing, and multi-engine simulation modeling system behaviors like routing and distributed caching. Experiments demonstrate the benefits of KV-aware routing over round-robin for improved prefix cache reuse and reduced TTFT.