TechPrufer
Get started
Together AI
Together AI logoTogether AI · Feature trail

Speech-to-Text (STT) APIs

8 posts · Jul 2025 – May 2026

What this trail covers

Together AI now offers native deployment of elite proprietary Speech-to-Text (STT) and Text-to-Speech (TTS) models, including Deepgram's Nova-3, Nova-3 Multilingual, Flux, and Aura-2, on dedicated infrastructure. This enables high-quality, low-latency voice generation and transcription with advanced features like WebSocket streaming for real-time audio processing, optimized voice activity detection, and speaker diarization. The integration allows for co-location with LLM and TTS workloads for unified pipelines, offering production-grade reliability with a 99.9% uptime SLA, SOC 2 Type II and HIPAA-ready support, and global regions with data residency options. The platform provides a single production surface for real-time voice agents, with STT, LLM, and TTS on one platform, simplifying operations and control over performance.

All Together AI trails
  1. How Together AI built the world’s fastest speech-to-text stackThis post details engineering changes to achieve world-leading ASR performance, including TensorRT multi-profile engines for encoder optimization, conditional CUDA graphs for GPU-side decoder control flow, lower-copy CPU paths using persistent Unix domain sockets and shared memory, evented streaming I/O with epoll for reduced scheduler pressure, and runtime GC control via `gc.freeze()` to eliminate tail latency spikes.
  2. Violin: An open-source video translation skill that breaks language barriersThis post introduces Violin, an open-source video translation skill that integrates Together API for ASR (Whisper V3), LLM translation (Deepseek V4 Pro), and TTS (Cartesia Sonic 3). It also includes a multimodal chat assistant powered by a vision-language model (Qwen3.5-397B-A17B) and offers web app, CLI, and agent skill interfaces. The contribution focuses on the orchestration of these components for video translation and interactive chat, highlighting the use of specific models and their capabilities.
  3. Deepgram speech-to-text and voice models now available natively on Together AIThis post announces the native deployment of Deepgram's Flux, Nova-3, Nova-3 Multilingual, and Aura-2 STT and TTS models on Together AI's Dedicated Model Inference platform. It highlights the integration of these models for real-time voice agents, emphasizing low latency and co-location with LLM and TTS workloads. Key technical contributions include the conversational STT capabilities of Flux with 250ms end-of-turn detection, Nova-3's robustness for real-world audio with vocabulary customization, and Aura-2's enterprise TTS for clarity and consistency. The infrastructure benefits include isolated GPU capacity, a 99.9% uptime SLA, and SOC 2 Type II/HIPAA-ready support. The developer experience is enhanced by unified SDKs, authentication, observability, and billing across the voice pipeline.
  4. Build real-time voice agents on Together AIThis post announces the native hosting of Deepgram (STT) and Cartesia (TTS) models on Together AI infrastructure, expanding the model choice for voice agents. It emphasizes the architectural benefit of co-locating STT, LLM, and TTS components within the same cluster to achieve end-to-end pipeline latency under 500ms. This co-location reduces inter-vendor network hops, improving speed, flexibility (access to intermediate transcript/response text), and reliability (single API, auth, billing, metrics). The post highlights the production use case of Decagon running concierge voice agents on this stack.
  5. How speech models fail where it matters the most and what to do about itThis post introduces the SF Streets and US Streets benchmarks to highlight the failure modes of state-of-the-art ASR models on named entities, particularly for non-English primary speakers, showing an average 39% transcription error rate and a 2.40-mile deviation for these speakers. It proposes and demonstrates a synthetic data generation technique called 'cross-lingual style transfer' using the XTTS model to create diverse phonetic pronunciations of English street names in foreign languages. Fine-tuning a base model on fewer than 1,000 synthetic utterances improved accuracy by 60% relative to the base model, with significant gains for non-English speakers. The post also discusses the economic impact of these errors and the release of the SF Streets and US Streets datasets for community research.
  7. Announcing the fastest inference for realtime voice AI agentsThis post details the engineering behind Together AI's expanded voice infrastructure, focusing on real-time performance and developer experience. Key contributions include: 1. **Streaming Whisper STT:** Optimized model inference and system design (WebSocket streaming, tuned VAD) for real-time audio processing, achieving faster transcript completion (up to 35% faster than alternatives) by prioritizing time-to-complete-transcript. 2. **Serverless Open-Source TTS:** Introduction of Orpheus (187ms TTFB, natural expressiveness) and Kokoro (97ms TTFB, ultra-low latency) available via REST and WebSocket, eliminating the need for dedicated instances for sporadic TTS needs. 3. **Voxtral Mini and Speaker Diarization:** Integration of a higher-accuracy multilingual transcription model (Voxtral Mini) with measurably lower word error rates, and speaker diarization for structured conversation analysis. Architectural decisions emphasize low latency (sub-200ms TTS, millisecond-accurate transcription), quality (multiple tiers for accuracy and naturalness), and scale (multiplexing conversations over single WebSocket connections for efficient handling of high concurrency).
  8. Together AI Launches Speech-to-Text: High-Performance Whisper APIsThis post announces the launch of Together AI's Speech-to-Text (STT) APIs, featuring a high-performance deployment of Whisper V3 Large. Key technical contributions include optimizations to the Whisper model for maximized GPU utilization, smart voice activity detection using Silero for precise audio segmentation, and intelligent chunking/batching strategies for handling long audio files. These optimizations result in transcription speeds 15x faster than OpenAI, with the ability to process files over 1GB and 30+ minute calls without chunking. The API offers superior word-level alignment, comprehensive language support (50+ languages), dedicated endpoints for sub-second processing, and batch processing capabilities. The post also highlights the cost-effectiveness of the offering at $0.0015 per audio minute and provides Python SDK examples for integration.