Comparing GPUs across architectures and tiers

This post provides a technical guide for comparing datacenter GPUs across architectures and tiers. It details key metrics for evaluating GPU performance and cost-effectiveness for ML workloads, including core count (CUDA, Tensor, RT cores), FLOPS (at various precisions), VRAM (capacity, bus size, clock speed, GDDR vs. HBM), and TDP. It emphasizes the importance of comparing FLOPS at the same precision and core type for accurate apples-to-apples comparisons and highlights VRAM capacity as a critical factor for model size limitations during inference. The post also references existing resources for deeper dives into specific GPU comparisons.