Modal

Modal Clusters are generally available

Oct 1, 2026Peyton Walters
Modal Clusters are generally available
SummaryAI, from the source post

Introduces Modal Clusters, a new primitive for multi-node workloads, accessible via the `@modal.clustered` decorator. This enables communication between nodes via InfiniBand verbs at up to 6.4 Tbps, automatically configured for PyTorch and NCCL. The post details the implementation of a new gang scheduler with an observe-plan-act loop for atomic N-node placement. It also describes the integration of RDMA support into gVisor, including upstreaming changes to the OSS project, to enable secure multi-tenant RDMA communication. The post highlights customer use cases for training and inference at scale.

More from the Inference Performance Optimization trail