Choosing the Right NVIDIA GPU for Running Inference | CoreWeave Blog

This post provides a detailed mapping of various NVIDIA GPU platforms (GB300 NVL72, GB200 NVL72, HGX B300, HGX B200, HGX H200, HGX H100, RTX PRO 6000 Blackwell Server Edition) to specific inference workload profiles. It outlines how factors like model size, context window, concurrency, batching behavior, latency SLOs, and deployment shape influence the choice of GPU. The post highlights the performance benefits and cost efficiencies of different platforms for use cases ranging from frontier-scale MoE models and high-concurrency chat endpoints to agentic AI inference and multimodal serving. It also introduces CoreWeave's unified inference offerings (CoreWeave Inference, Inference on CKS, Dedicated Inference, Serverless Inference) as solutions for managing these diverse inference needs.