Modal Clusters are generally available
Oct 1, 2026Peyton Walters

Summary
Introduces Modal Clusters, a new primitive for multi-node workloads, accessible via the `@modal.clustered` decorator. This enables communication between nodes via InfiniBand verbs at up to 6.4 Tbps, automatically configured for PyTorch and NCCL. The post details the implementation of a new gang scheduler with an observe-plan-act loop for atomic N-node placement. It also describes the integration of RDMA support into gVisor, including upstreaming changes to the OSS project, to enable secure multi-tenant RDMA communication. The post highlights customer use cases for training and inference at scale.
More from the Inference Performance Optimization trail
- Endpoint Candidates: evaluate and promote inference endpoints | Modal Blog
- TypeSafe AI scales Jev to trillions of tokens in three days on Modal | Modal Blog
- Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine | Modal Blog
- How to serve trillions of tokens for trillion-parameter coding agents | Modal Blog
- Kimi K3 by Moonshot now available on Modal | Modal Blog