Baseten Feature Trails

Baseten continues to expand its AI model deployment and serving platform by integrating new state-of-the-art embedding models like NVIDIA Nemotron 3 Embed, optimizing for agentic workflows with LangChain Deep Agents, supporting the deployment of large multimodal models like Inkling, offering Step 3.7 Flash, a 198-billion-parameter sparse MoE vision-language model in a hardware-efficient configuration, and now providing the Whisper Large V3 model for highly performant audio transcription at up to 100 concurrent real-time streams. This includes the deployment of NVIDIA Nemotron 3.5 ASR Streaming, offering low-latency, production-ready speech recognition with both English and multilingual models, leveraging NVIDIA NIM for optimized streaming inference.

Baseten continues to optimize its AI model deployment and serving platform, focusing on specific model performance enhancements. This includes developing specialized APIs for models like GLM-5.2, such as a 'fast' API prioritizing latency for coding and agent tasks. Optimizations involve tuning parallelism, batching, and caching strategies, alongside improvements to speculative decoding and inference engines. Learnings from optimizing Kimi K3 are being applied to other open models. This post details the Baseten Inference Runtime, which offers frontier performance research in production for lowest latency and highest throughput.

This post details the use of Laguna S 2.1, an agentic coding model, to perform a repository-scale transformation of the Hypersomnia game. It highlights Laguna's ability to understand an unfamiliar codebase, coordinate changes across multiple files and specialized AI models (Step 3.7 Flash, Krea 2 Turbo, NVIDIA Cosmos 3 Nano), and continuously verify its work. The process involved transforming game assets, code, and text while preserving core gameplay mechanics. Laguna demonstrated judgment in selecting and orchestrating models, scaling generated assets, and making tradeoffs to ensure a stable, playable output. The capability extends to complex, long-horizon tasks requiring creative problem-solving and technical constraint adherence.

This post introduces a new capability for fine-tuning Qwen3-TTS for high-quality voice cloning, detailing three approaches: In-Context Learning (ICL), speaker-embedding-only, and fine-tuning. It provides a technical deep-dive into the fine-tuning process, including dataset preparation using ASR with character-level timestamps, model training with talker and sub-talker loss, and speaker embedding centroid calculation. The post also discusses training optimizations like learning rate schedules and compares fine-tuning performance against zero-shot methods, highlighting improvements in expressiveness and operational benefits.