TechPrufer
Get started
Decagon
Decagon logoDecagon · Feature trail

Audio-native Speaker Change Detection

1 post · Sep 2026

What this trail covers

Decagon is developing advanced audio-native semantic speaker-change detection capabilities. This system aims to reliably identify when a new participant joins a conversation, distinguishing relevant changes that require agent intervention (e.g., for security or fraud prevention) from irrelevant audio events (e.g., background noise, announcements). The system leverages speaker embeddings and multimodal LLMs, with a focus on constructing robust benchmarks and employing supervised fine-tuning to optimize for conversation-level false-positive rate and recall. This capability enhances the security and operational efficiency of voice agents.

All Decagon trails
  1. Audio-native semantic speaker-change detection | DecagonThis post details the development of an audio-native semantic speaker-change detection system. It introduces a novel benchmark construction focused on conversation-level false-positive rate (cFPR) and recall, moving beyond traditional diarization metrics. The solution explores speaker embeddings (TitaNet) with methodologies like mean dissimilarity and maxsplit, and employs multimodal LLMs post-trained on synthetic datasets. A key contribution is the supervised fine-tuning (SFT) objective derived from conversation-level probabilities, which directly optimizes for the desired operating point and achieves significant recall improvements over diarization models at a fixed cFPR.