Three Tests to Run Before You Switch from LoRA to FullFT

This post details experiments comparing LoRA and Full Parameter Fine-Tuning (FullFT) on Qwen3.5-9B, focusing on three key areas that influence performance: data coverage, optimization (learning rate tuning), and adapter capacity (rank). It provides a framework for engineers to test these factors before deciding to switch from LoRA to FullFT, demonstrating that data diversity and learning rate optimization can often close performance gaps previously attributed solely to adapter size. Specific experiments on 'Placement', 'Register Allocation', and 'Nexa VM' tasks illustrate these findings.