Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer | NVIDIA Technical Blog

What this post added

This post details the process of creating the NVIDIA Nemotron 3 Ultra NVFP4 checkpoint using NVIDIA Model Optimizer. It introduces NVFP4 quantization for NVIDIA Blackwell architecture, achieving higher inference throughput and reduced model size. The post highlights a mixed-precision approach for the Nemotron 3 Ultra checkpoint, using NVFP4, FP8, BF16, and FP16 based on layer sensitivity. It also explains how the checkpoint adapts to different hardware (Hopper and Blackwell) and delves into the challenges and solutions for FP4 quantization, including max scaling, MSE scaling, and a new four-over-six scaling method that optimizes for FP4 representable value gaps to improve accuracy.