NVIDIA Blackwell Sets STAC-AI Record for LLM Inference in Finance | NVIDIA Technical Blog

This post details the performance results of NVIDIA Blackwell platforms (HGX B200, RTX PRO 6000 Blackwell) on the STAC-AI LANG6 benchmark for LLM inference in financial use cases. It showcases significant speedups in batch and interactive modes, with up to 2.8x improvement on HGX B200 compared to GH200. The post highlights the use of TensorRT LLM with FP8 and NVFP4 quantization, and discusses the impact of chat templates and tokenization during inference. It also provides links to detailed STAC reports for HPE, Lambda, and Supermicro systems.