TechPrufer
Get started
Baseten
Baseten logoBaseten · Feature trail

Baseten Delivery Network (BDN) for Model Weight Caching

1 post · Mar 2026 – Apr 2026

What this trail covers

Baseten's AI model deployment and serving platform now includes the Baseten Delivery Network (BDN), an opt-in weight delivery system. BDN mirrors model weights into Baseten-managed storage at deploy time, caching them across a three-tier hierarchy (node-local disk, in-cluster peer cache, mirrored origin). This significantly reduces cold start times, even during thundering-herd scale events, by minimizing reliance on upstream providers and optimizing weight transfer. BDN also reduces costs by ensuring only a single fetch per file is triggered across the cluster, preventing bandwidth saturation during burst scaling.

All Baseten trails
  1. How the Baseten Delivery Network (BDN) makes cold starts fastThis post introduces the Baseten Delivery Network (BDN), a new system designed to accelerate model cold starts by optimizing weight transfer. BDN addresses brittleness at scale by implementing a three-tier caching hierarchy: node-local disk, an in-cluster peer cache using a consistent hash ring, and a mirrored origin in Baseten-managed blob storage. Key technical contributions include push-time mirroring of weights to Baseten storage, content-hash-based deduplication, and a coordinated download mechanism across tiers. The system employs node-level single-flight downloads and a cluster-level hash ring for efficient fetch coordination, ensuring origin bandwidth is utilized 1x the model size and parallelized across the cluster. BDN also centralizes data transfer into an optimized chunk-transfer implementation.
  2. Introducing the Baseten Delivery Network: Fast cold starts for big modelsIntroduced the Baseten Delivery Network (BDN) to address cold start bottlenecks for large models. BDN implements a three-tier caching system (node-local, in-cluster peer, mirrored origin) and single-flight downloads to accelerate weight loading. This results in 2-3x faster cold starts, particularly during burst scaling events, by decoupling deployments from upstream storage dependencies and preventing thundering herd issues.