TechPrufer
Sign inGet started

Blogs›Baseten›Voice Cloning Fine-tuning

Voice Cloning Fine-tuning

Voice Cloning Fine-tuning

1
posts
2026

This post introduces a new capability for fine-tuning Qwen3-TTS for high-quality voice cloning, detailing three approaches: In-Context Learning (ICL), speaker-embedding-only, and fine-tuning. It provides a technical deep-dive into the fine-tuning process, including dataset preparation using ASR with character-level timestamps, model training with talker and sub-talker loss, and speaker embedding centroid calculation. The post also discusses training optimizations like learning rate schedules and compares fine-tuning performance against zero-shot methods, highlighting improvements in expressiveness and operational benefits.

2026

Fine-tuning Qwen3-TTS for high-quality voice cloning

7/31/2026

This post details the technical process of fine-tuning Qwen3-TTS for high-quality voice cloning. It contrasts ICL and speaker-embedding-only methods with fine-tuning, explaining the inference overhead and speaker consistency trade-offs. The core contribution is a step-by-step guide to fine-tuning, covering dataset construction (using ASR and character-level timestamps), model training (multi-codebook approach, talker/sub-talker loss, centroid-based speaker embedding, learning rate scheduling), and evaluation. It also provides examples demonstrating improved expressiveness and naturalness compared to one-shot cloning.