Announcing the fastest inference for realtime voice AI agents

What this post added

This post introduces significant enhancements to Together AI's voice capabilities, focusing on real-time voice agents. Key contributions include: 1. Streaming Whisper STT: Optimized inference and system design (WebSocket streaming, VAD tuning) for up to 35% faster complete transcripts compared to alternatives, prioritizing time-to-complete-transcript for voice agents. 2. Serverless Open-Source TTS: Introduction of Orpheus (187ms TTFB, natural voice) and Kokoro (97ms TTFB, ultra-low latency) available via REST and WebSocket APIs, eliminating the need for dedicated infrastructure for sporadic TTS needs. 3. Voxtral Mini and Speaker Diarization: Addition of a premium multilingual transcription model optimized for European languages and challenging audio, offering measurably lower word error rates than standard Whisper, and speaker diarization for identifying and labeling different speakers in audio. Architectural decisions emphasize low latency (sub-200ms TTS, millisecond-accurate transcription), quality (accuracy and natural-sounding speech), and scale (consistent performance under load via optimized WebSocket multiplexing and auto-scaling).