Fine-Tuning Small Open-Source LLMs to Outperform Large Closed-Source Models by 60% on Specialized Tasks

This post details how Parsed leveraged Together AI's fine-tuning platform to fine-tune a Gemma 3 27B model for a healthcare scribing task. It highlights the methodology of using a task-specific evaluation harness to drive both Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) optimization. The fine-tuned model achieved 60% better accuracy than Claude Sonnet 4 and significantly reduced inference costs, demonstrating the effectiveness of specialized optimization for smaller open-source models. The post also contrasts the performance of various open-source and proprietary models before and after fine-tuning.