TechPrufer
Get started
Pinterest
Pinterest logoPinterest · Feature trail

VLM Serving Stack

2 posts · Sep 2026

What this trail covers

Pinterest built a VLM serving stack on NVIDIA Dynamo and Blackwell GPUs to handle multimodal workloads. This stack addresses challenges like complex request payloads, expensive prefill, multi-turn interactions with evolving visual context, and KV cache pressure. Key features include disaggregated prefill/decode serving, KV cache offloading (via LMCache), multimodal support in Dynamo frontend/vLLM, and custom modality support for projection embeddings (using PinCLIP) to reduce the cost of image processing. The system was further optimized by integrating custom modality support for projection embeddings using PinCLIP, allowing the reuse of precomputed embeddings and avoiding expensive pixel-based image serving.

All Pinterest trails
  1. Building Pinterest’s VLM Serving Stack on NVIDIA DynamoThis post details the evolution of Pinterest's VLM serving stack, focusing on the integration of NVIDIA Dynamo and Blackwell GPUs. It elaborates on specific challenges encountered, such as request payload complexity, expensive prefill, multi-turn workloads with evolving visual context, KV cache memory pressure, and custom model/payload support. The contribution highlights the implementation of disaggregated prefill/decode serving, KV cache offloading via LMCache, multimodal support in Dynamo frontend/vLLM, and the introduction of custom modality support for projection embeddings using PinCLIP to reduce serving costs.
  2. Building Pinterest’s VLM Serving Stack on NVIDIA DynamoThis post details the evolution of Pinterest's VLM serving stack, focusing on the integration of NVIDIA Dynamo and Blackwell GPUs. It elaborates on specific challenges encountered, such as request payload complexity, expensive prefill, multi-turn workloads with evolving visual context, KV cache memory pressure, and the need for custom model and payload support. The contribution highlights the implementation of disaggregated prefill/decode serving, KV cache offloading via LMCache, multimodal support in Dynamo frontend/vLLM, and the introduction of custom modality support for projection embeddings using PinCLIP to reduce serving costs.