Vercel

Build realtime voice agents on AI Gateway

Jun 29, 2026
Build realtime voice agents on AI Gateway
SummaryAI, from the source post

This post introduces real-time voice, text-to-speech, and speech-to-text capabilities to Vercel's AI Gateway. It details how these audio modalities can be integrated using the AI SDK 7, including a `useRealtime` hook for browser-based conversational agents with features like barge-in and tool integration mid-conversation. It also covers `generateSpeech` for text-to-speech and `transcribe` for speech-to-text, demonstrating their usage with specific models from OpenAI and xAI. The post emphasizes that these audio capabilities are routed through AI Gateway with the same provider routing, observability, and spend controls as other modalities.

More from the AI Gateway and AI SDK Integration trail