The Transcript Looked Fine. The Call Wasn't. — Debugging Voice Agents, Arize

This post contributes to LLM observability by highlighting the specific challenges of debugging voice agents, where text transcripts alone are insufficient. It introduces the necessity of integrating audio, transcript, and trace data into a unified view for debugging. The talk advocates for OpenInference semantic conventions for real-time audio processing across different providers and proposes audio-native evaluations that assess metrics like sentiment, latency, interruptions, and task success directly from the audio. It also outlines an observe-evaluate-improve loop and agent experiments for self-healing voice agents, including monitors, investigation agents, and replayed fixes.