Serverless GPU Inference
GPU Memory Snapshots: Supercharging sub-second startup

GPU Memory Snapshots: Supercharging sub-second startup

7/30/2025

What this post added

Introduced GPU memory snapshots, extending the existing memory snapshotting capability to include GPU state (device memory, CUDA kernels, streams, contexts). This is achieved by leveraging NVIDIA's CUDA checkpoint/restore APIs (`cuCheckpointProcessLock`, `cuCheckpointProcessCheckpoint`, `cuCheckpointProcessRestore`, `cuCheckpointProcessUnlock`). The implementation transparently captures and restores GPU memory, eliminating the need for manual GPU warm-up and recompilation after restore, leading to significantly faster cold starts for GPU-accelerated workloads.

Read the original post ↗