NovelAI as Among the First to Have NVIDIA HGX H100 GPUs Online | CoreWeave

This post announces the general availability of NVIDIA HGX H100 GPUs on CoreWeave, highlighting NovelAI as an early adopter. It details CoreWeave's infrastructure for these GPUs, including a rail-optimized design, NVIDIA Quantum-2 InfiniBand networking (3.2Tbps per node), and the ability to scale to 16,384 H100 SXM5 GPUs. It also mentions the integration of NVIDIA BlueField DPUs for secure provisioning. The post emphasizes the benefits for large language model training and inference, enabling flexibility in model design, faster iteration, and serving models to millions of users.