ModelExpress: Distributing Model Artifacts at the Speed of Light | NVIDIA Technical Blog

This post introduces NVIDIA ModelExpress (MX), a system designed to accelerate the model weight lifecycle by optimizing the transfer of model artifacts. MX prioritizes direct GPU-to-GPU P2P RDMA transfers via NIXL, bypassing object storage and host memory. It implements strategies for bootstrapping from storage (Model Streamer, GPUDirect Storage, Model Cache Service) and for scaling out via peer-to-peer transfers once a replica is serving. MX also optimizes NIXL memory registration overhead through techniques like VMM arena registration and efficient memory registration for P2P transfers.