TechPrufer
Get started
Hugging Face
Hugging Face logoHugging Face · Feature trail

Parameter-Efficient Fine-Tuning (PEFT)

9 posts · Sep 2023 – Sep 2026

What this trail covers

The PEFT library continues to expand its support for parameter-efficient fine-tuning techniques. This post highlights the ongoing research and benchmarking efforts to compare various PEFT methods beyond LoRA, including their performance, memory usage, and checkpoint sizes. The goal is to provide users with more informed choices for fine-tuning models, moving beyond the default popularity of LoRA to explore techniques that offer better trade-offs for specific use cases. Benchmarks are being developed for Nemotron models, demonstrating their adaptability to specialized domains like competitive programming (IOI) and mathematical olympiads (IMO) through SFT and RL, paired with sophisticated inference systems. The Nemotron Labs IMO 2026 collection provides access to specialized checkpoints, training data, and benchmarks, while the IOI paper details the GenCorrect methodology and the Nemotron-3-Ultra-CC model is available on Hugging Face.

All Hugging Face trails
  1. Beyond LoRA: Can you beat the most popular fine-tuning technique?This post critically examines the overwhelming popularity of LoRA for parameter-efficient fine-tuning (PEFT) and argues for exploring alternative techniques. It details the challenges in relying solely on paper results for PEFT method selection due to publication bias and inconsistent benchmarking. The post introduces Hugging Face's ongoing efforts to establish objective benchmarks within the `PEFT` library, comparing techniques like LoRA against others on LLM math reasoning and image generation tasks. It emphasizes tracking metrics beyond test performance, such as VRAM usage, forgetting, runtime, and checkpoint size, to understand trade-offs. Initial findings suggest that while LoRA performs well, other PEFT methods can offer superior performance or memory efficiency, indicating that users should consider a broader range of techniques.
  2. One Model Family, Two Gold-Level Results: Fine-Tuning Nemotron for IOI and IMOThis post details the successful fine-tuning of Nemotron foundation models for specialized domains (IOI and IMO) using SFT and RL, achieving gold-medal level results. It introduces a reusable specialization recipe involving strong base models, curated domain-specific data and traces, standard post-training methods, and inference loops for generation, evaluation, and refinement. Specifics include the performance gains of Nemotron-3-Nano-CC and Nemotron-3-Ultra-CC on IOI tasks, and the complementary strengths of SFT and RL checkpoints for IMO. The post also highlights the co-design of models, data, and inference workflows, and makes the specialized models, datasets, and recipes available on Hugging Face.
  3. NeoMME: an efficient Multimodal-native and Multilingual EncoderThis post introduces NeoMME-260M, an efficient Multimodal-native and Multilingual Encoder for feature extraction. It details the model's architecture and performance, contributing to the ongoing development of parameter-efficient fine-tuning techniques by providing a new model for multimodal and multilingual tasks.
  4. Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO StepsThis post details the process of fine-tuning the LFM2.5-350M model using Group Relative Policy Optimization (GRPO) with the TRL library to improve its structured output compliance. It outlines data augmentation strategies to bridge gaps between training and evaluation datasets, the configuration of LoRA adapters targeting specific model modules, and the definition of three reward functions (JSON format, field count, schema validation) to guide the fine-tuning process. The post provides concrete training arguments and demonstrates the evaluation of the fine-tuned model against the IFStruct benchmark, showing a performance improvement from 22.6% to 29.7%. It also details the conversion of the merged model to GGUF format for local serving with llama.cpp.
  5. Training a coding model to paint watercolours with TRL and OpenEnvThis post details the open implementation of training a coding model to generate watercolour paintings using TRL (Transformer Reinforcement Learning) and OpenEnv. It reproduces an earlier project by Surya Narreddi, focusing on the engineering aspects. Key contributions include: an open-source RL environment that wraps the p5.brush JavaScript library, a system prompt restricting the model to 10 specific drawing methods, a hand-rated pool of reference images to define aesthetic taste, and the integration of two reward models (a pairwise judge and HPSv3) to guide the RL training. The post also details the training pipeline, including the use of Hugging Face Jobs for training, Spaces for the environment and scorer, and Inference Providers for the pairwise judge. Three different reward mixes were trained and compared, demonstrating that the hand-rated pool can steer the policy towards desired artistic outcomes.
  7. Making Knowledge Distillation Cheap Enough to Run at ScaleThis post introduces a novel and cost-effective knowledge distillation technique designed for large-scale deployment. It details a method to distill knowledge from large, expensive teacher models into smaller, more efficient student models. The core innovation lies in a new distillation strategy that significantly reduces the computational overhead and time required for the distillation process, making it feasible to distill knowledge from massive models into smaller ones for practical, scaled applications. The post likely outlines specific algorithmic improvements or architectural choices that enable this cost reduction and efficiency gain.
  8. Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization ProblemThis post introduces a novel method for pruning Large Language Models (LLMs) by treating block removal as a constrained binary optimization problem, drawing parallels to the Ising model in physics. The core idea is to map the weights of redundant blocks to binary variables and solve an optimization problem to identify which blocks can be removed with minimal impact on model performance. This approach aims to achieve significant model compression by systematically removing entire blocks of parameters, rather than fine-grained pruning, leading to potential inference speedups and reduced memory footprint.
  9. Direct Preference Optimization Beyond ChatbotsThis post introduces Direct Preference Optimization (DPO) as a method to align language models with human preferences, extending beyond chatbot applications to general-purpose models. DPO offers a simpler and more stable alternative to Reinforcement Learning from Human Feedback (RLHF) by directly optimizing a policy against a reference policy using a binary classification loss on preference pairs. The post details the mathematical formulation of DPO and provides an example of its application to a non-chatbot model, demonstrating its versatility.
  10. SetFit: Efficient Few-Shot Learning Without PromptsThis post introduces SetFit, a new parameter-efficient fine-tuning method for few-shot learning. SetFit utilizes a Siamese network architecture and contrastive learning to train models on small labeled datasets, achieving high accuracy without prompts. It involves a two-stage training process: first, training a Siamese network on unlabeled data to generate synthetic labels, and second, fine-tuning a classification model on these synthetic labels. The post provides benchmarks demonstrating SetFit's competitive performance against other few-shot learning methods.
  11. Parameter-Efficient Fine-Tuning using 🤗 PEFTThis post introduces the `mt0-large` model for text generation, updated on Sep 26, 2023. It also provides an example of using the PEFT library with this model for parameter-efficient fine-tuning.