Heaps do lie: debugging a memory leak in vLLM. | Mistral AI

What this post added

This post details the debugging of a memory leak in vLLM within a disaggregated Prefill/Decode serving setup. The investigation methodically ruled out standard Python profiling tools and GDB, then utilized Heaptrack to identify that the leak was not in the traditional heap but in Resident Set Size (RSS). Further analysis with pmap revealed growing anonymous memory mappings, suggesting issues with mmap/mremap. The root cause was traced to the KV Cache transfer through NIXL, leading to unreleased memory regions. The post highlights the use of BPFtrace for kernel-level tracing to pinpoint the exact system calls responsible.