TechPrufer
Get started
Modal
Modal logoModal · Feature trail

Fine-tuning Generative Models

7 posts · Dec 2023 – Dec 2024

What this trail covers

Modal's platform supports fine-tuning generative models on custom datasets, enabling users to prepare datasets, set up containerized environments with necessary dependencies, utilize cloud storage for model weights, and configure training hyperparameters. The process includes training scripts, hyperparameter configuration, and serving the fine-tuned model via a web UI. Ramp has utilized Modal to accelerate the development of their text-to-structured-JSON model for receipt management, driving down operational costs and improving accuracy. This post details an experiment to beat proprietary embedding models with a quick fine-tune, demonstrating that with just a handful of examples, a fine-tuned open-source embedding model can provide greater accuracy at a lower price than proprietary models like OpenAI's `text-embedding-3-small`.

All Modal trails
  1. What is LLM fine-tuning?This post provides a comprehensive overview of LLM fine-tuning, explaining its benefits (cost, efficiency, customization), common use cases (structured output, style adherence, domain-specific instruction), and where to fine-tune (OpenAI, Predibase, Modal, Colab, SageMaker). It also lists top frameworks (HuggingFace transformers, TRL, Axolotl) and outlines the steps for fine-tuning on Modal: choosing a base model, preparing the dataset (including prompt creation and optional special tokens), and using advanced strategies. The post links to a Modal tutorial for practical implementation.
  2. Create an infinite icon library by fine-tuning Stable DiffusionThis post details the process of fine-tuning Stable Diffusion on the Heroicons dataset to create an infinite icon library. It covers dataset preparation (SVG to PNG conversion, captioning), setting up Modal environments with Diffusers dependencies, using Modal Volumes for persistent storage of model weights, and configuring training hyperparameters. The post demonstrates the end-to-end workflow from data preparation to serving a Gradio UI for the fine-tuned model.
  3. Why Substack moved their AI and ML pipelines to ModalThis post details Substack's migration of their AI and ML pipelines from AWS SageMaker to Modal. It highlights the challenges faced with SageMaker, such as convoluted developer experience, difficulties in code collaboration, and slow container startup times. The post then describes how Substack leveraged Modal's features, including its natural iteration flow, autoscaling capabilities, and native storage primitives (network volumes and key-value stores), to build a faster and more flexible ML workflow. This includes a full fine-tune -> validation -> deployment process, enabling rapid development and deployment of recommendation models and other ML applications.
  4. Beating proprietary models with a quick fine-tuneThis post details an experiment demonstrating how to fine-tune an open-source embedding model to outperform proprietary models like OpenAI's `text-embedding-3-small` for a specific use case. It covers the process of selecting a base model (e.g., BAAI/bge-base-en-v1.5, sentence-transformers/all-mpnet-base-v2, jinaai/jina-embeddings-v2-small-en), preparing a dataset (using the Quora duplicates dataset), and acquiring training infrastructure using Modal's autoscaling GPU capabilities. The experiment involved a grid search over hyperparameters including model choice, dataset size (from 100 to 100,000+ samples), and embedding dimensions. The results indicate that fine-tuning can achieve superior accuracy at a lower cost, enabling the bootstrapping of a data flywheel.
  5. How Ramp automated receipt processing with fine-tuned LLMsThis post details how Ramp used Modal to fine-tune LLMs for receipt processing. Key contributions include parallel training of candidate models, persisting weights to Modal volumes, and serving inference endpoints. Modal's orchestration of GPU scaling and standardization of containerized environments were critical. Additionally, the post highlights using Modal for large-scale batch processing, specifically stripping PII from invoices, demonstrating a 20-minute completion time for a task that would have taken 3 days locally.
  7. Embedding English Wikipedia in under 15 minutesThis post details the implementation of a distributed GPU job on Modal to generate text embeddings for the entire English Wikipedia corpus. It covers setting up a Modal app with custom images for Hugging Face's Text Embedding Inference server, utilizing Modal Volumes for dataset persistence, and managing container lifecycles with Modal classes. The solution leverages concurrent requests to the inference server and dynamic batching to achieve efficient embedding generation, reducing the time from hours to under 15 minutes and costing under $15.
  8. How to fine-tune an LLM on ModalThis post details how to fine-tune a Large Language Model (LLM) on Modal. It covers preparing datasets, setting up containerized environments with necessary dependencies (e.g., PyTorch, Hugging Face Transformers), utilizing Modal's `modal.Image` for reproducible environments, and configuring training hyperparameters. The post demonstrates using Modal's `modal.Stub` and `@stub.function` decorators to orchestrate the fine-tuning process, including data loading, model training, and saving checkpoints to Modal's object storage. It also touches on serving the fine-tuned model.