How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin | NVIDIA Technical Blog

This post introduces the NVIDIA Groq 3 LPX as an interactive AI inference accelerator for the NVIDIA Vera Rubin platform, enabling ultrafast interactivity at long context lengths (100K+ tokens) for agentic AI. It details how Groq 3 LPX achieves this through a deterministic execution model with compiler-scheduled chip-to-chip communication and computation-communication overlap, minimizing first-bit latency. Benchmarks show median speeds of 3,431 output tokens/second on Gemma 4 31B at 100K context and 4,767 tokens/second on the SPEED-Bench coding benchmark. The post also discusses configurations like prefill-decode disaggregation and attention-FFN disaggregation for optimizing performance.