Introducing NVIDIA Nemotron 3.5 ASR Streaming

This post introduces the integration of NVIDIA Nemotron 3.5 ASR (English and multilingual variants) into the Baseten Model Library, enabling low-latency, production-ready speech recognition. It details the cache-aware FastConformer-RNNT architecture, performance benchmarks on a single H100 (up to 100 concurrent real-time streams over WebSocket, higher via gRPC), and comparative accuracy metrics (WER) on English and multilingual datasets. The post highlights the benefits of NVIDIA NIM for cache-aware inference and mentions the possibility of fine-tuning using Baseten Training.