Voice Cloning Fine-tuning

Baseten continues to optimize its AI model deployment and serving platform, focusing on specific model performance enhancements. This includes developing specialized APIs for models like GLM-5.3-Flash, DeepSeek V4 Pro 0813, and DeepSeek V4.1 Flash, such as a 'fast' API prioritizing latency for coding and agent tasks. Optimizations involve tuning parallelism, batching, and caching strategies, alongside improvements to speculative decoding and inference engines. The DeepSeek V4.1 Flash model intro. This post introduces a new capability for fine-tuning Qwen3-TTS for high-quality voice cloning, detailing three approaches: In-Context Learning (ICL), speaker-embedding-only, and fine-tuning. It provides a technical deep-dive into the fine-tuning process, including dataset preparation using ASR with character-level timestamps, model training with talker and sub-talker loss, and speaker embedding centroid calculation. The post also discusses training optimizations like learning rate schedules and improved speaker embedding derivation.