Latency Is a Budget. Humanlike Is the Goal. — Jesse Hall, LiveKit
Oct 11, 2026AI Engineer
Summary
This talk addresses the critical gap between measured latency and perceived latency in voice AI pipelines, arguing that humanlike interaction requires a focus on perceived latency and a 'latency budget.' It highlights how errors compound across pipeline stages (STT, LLM, TTS) and introduces audio-based turn detection for more natural conversation flow. The talk proposes a target of ~600ms for humanlike feel and introduces LiveKit's open-source benchmark and a voice-tuned Gemma model for optimizing these pipelines.
More from the Low-Latency Inference Engineering Playbook trail
- How Inference Keeps Up with Our Insatiable Need for Tokens — Cerebras & Gimlet Labs
- Taming the AI Hardware Cambrian Explosion — Abdul Dakkak, Modular
- Same Model, Different Speed: Why Your Inference Provider Matters — FriendliAI
- How MiniMax M3 Was Built: Sparse Attention and Native Multimodality — Olive Song
- From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model