Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo

This post details the evolution of Pinterest's VLM serving stack, focusing on the integration of NVIDIA Dynamo and Blackwell GPUs. It elaborates on specific challenges encountered, such as request payload complexity, expensive prefill, multi-turn workloads with evolving visual context, KV cache memory pressure, and custom model/payload support. The contribution highlights the implementation of disaggregated prefill/decode serving, KV cache offloading via LMCache, multimodal support in Dynamo frontend/vLLM, and the introduction of custom modality support for projection embeddings using PinCLIP to reduce serving costs.