Build realtime voice agents on AI Gateway
Jun 29, 2026

Summary
This post introduces real-time voice, text-to-speech, and speech-to-text capabilities to Vercel's AI Gateway. It details how these audio modalities can be integrated using the AI SDK 7, including a `useRealtime` hook for browser-based conversational agents with features like barge-in and tool integration mid-conversation. It also covers `generateSpeech` for text-to-speech and `transcribe` for speech-to-text, demonstrating their usage with specific models from OpenAI and xAI. The post emphasizes that these audio capabilities are routed through AI Gateway with the same provider routing, observability, and spend controls as other modalities.