Fine-tuning small open-source LLMs to outperform large closed-source models by 60% on specialized tasks

This post details how Baseten leverages its evaluation-first methodology and platform capabilities to fine-tune smaller open-source LLMs (e.g., Gemma 3 27B) to surpass the performance of larger proprietary models (e.g., Claude Sonnet 4) on specialized tasks like healthcare scribing. It introduces the concept of task-specific fine-tuning to exploit lower entropy output distributions, leading to 60% better accuracy and 50-80% lower inference costs. The post elaborates on Baseten's advanced evaluation methodology, which decomposes tasks into granular, objective checks to create a robust reward signal for Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). It also highlights the benefits of mechanistic interpretability for output attribution and self-improvement.