Modal + Mistral 3: 10x faster cold starts with GPU snapshotting

This post introduces and details the implementation of GPU memory snapshotting as an alpha feature within Modal's serverless GPU inference platform. It demonstrates a nearly 10x reduction in median cold start times for Mistral 3 3B models served via vLLM, reducing it from approximately 118 seconds to 12 seconds. The mechanism involves enabling Sleep Mode for the vLLM server and setting `experimental_options={'enable_gpu_snapshot': True}` to shift GPU memory to CPU memory for snapshotting, enabling faster restoration on subsequent starts.