Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer | NVIDIA Technical Blog

This post details the application of Quantization-Aware Distillation (QAD) using NVIDIA Model Optimizer to develop the Nemotron 3.5 Lightning NVFP4 checkpoint. It describes the two-stage QAD process: first, post-training quantization (PTQ) to W4A16 for the student model, targeting a 95-99% median accuracy recovery to leave room for QAD. Second, quantization-aware distillation where the student is trained against a frozen BF16 teacher using KL divergence loss and simulated quantization. The post highlights that QAD consistently outperforms PTQ in accuracy recovery and agentic/coding benchmarks, enabling more aggressive quantization (e.g., NVFP4) that would otherwise cause unacceptable degradation. It also provides specific PTQ recipes and discusses the end-to-end workflow reproduction using NVIDIA Model Optimizer and Megatron-Bridge.