Multi-Plane Networking for Agentic AI: 5 Lessons | CoreWeave

This post details CoreWeave's experience building a multi-plane network fabric for agentic AI. Key technical contributions include: 1. Implementing a non-blocking network fabric (1:1 uplink capacity at every layer) to prevent oversubscription and ensure low, predictable per-flow latency for agent chains alongside training traffic. This leverages NVIDIA Spectrum-X Ethernet, BlueField-3 DPUs, and ConnectX-9 SuperNICs. 2. Designing a multi-plane and multi-rail topology to scale the fabric without adding hops, allowing for a flat, two-tier design to reach large GPU counts while maintaining short, predictable GPU-to-GPU paths. 3. Utilizing hardware-accelerated load balancing on ConnectX-9 SuperNICs and adaptive routing on Spectrum-X switches to dynamically spread flows across available paths based on real-time congestion telemetry, avoiding static hashing issues. 4. Emphasizing resilience across five layers (topology, hardware, software, orchestration, and agent-level) to handle agent chain failures, as they lack clean restart boundaries like training checkpoints. 5. Tuning congestion control mechanisms (Priority Flow Control, Explicit Congestion Notification) against actual traffic patterns, particularly for NCCL collectives, to maintain a lossless fabric under demanding conditions.