NVIDIA Nemotron 3 Diarization: real-time speaker labels at a cent per audio hour

This post introduces the integration of NVIDIA Nemotron 3 Diarization into Baseten's platform, enabling real-time speaker labeling and diarized transcription. It details the model's transformer-based architecture built on NVIDIA Streaming Sortformer, its state management for unbounded audio, and its configurable algorithmic latencies (0.32s to 30.4s) for diverse use cases. Performance benchmarks show competitive Diarization Error Rates (DER) across various datasets and latency profiles. The post also highlights how speaker activity can be used for voice activity detection, end-of-turn detection, and turn-taking signals. Three specialized serving presets are provided: prerecorded, streaming, and real-time diarized transcription (which pairs Nemotron 3 Diarization with NVIDIA Parakeet V2 ASR). Performance testing on Baseten demonstrates over 500 concurrent streaming diarization streams per RTX PRO 6000 GPU and 1.35x higher batch processing throughput compared to NVIDIA's reference setup. The integration allows for direct conditioning of the ASR model on speaker activity features, enabling single-pass transcription of overlapped audio.