H100 vs. H200 vs. B200: which GPU should you use?

This post analyzes the trade-offs between NVIDIA H100, H200, and B200 GPUs for AI inference. It details how SXM form factor and NVLink impact performance, the role of VRAM in fitting large models (e.g., GLM 5.2), and the benefits of MIG for smaller models. It also explains how FP4 precision in the Blackwell architecture (B200) increases inference throughput by reducing memory footprint and improving compute efficiency, and how asynchronous programming (Tensor Memory Accelerator, TMEM) keeps GPUs from waiting on data. The post provides guidance on choosing the right GPU based on model size, traffic volume, and budget, highlighting specific use cases for each GPU.