AI Model Co-Design: Hardware-Friendly LLM Design | NVIDIA Technical Blog

What this post added

This post introduces hardware-friendly LLM design principles, focusing on dimensioning linear layers (H, H') for optimal arithmetic intensity and GPU utilization. It details how near-square linear layer dimensions, alignment to GPU tile sizes, and a width-over-depth aspect ratio maximize throughput and interactivity. It also introduces NVFP4 quantization and its integration with TensorRT tooling for efficient inference, and discusses expert parallelism (EP) and hybrid parallel strategies for scaling large Mixture-of-Experts models across multi-node systems.