Fine-tuned models now boot in less than one second
Sep 6, 2023

Summary
This post details a further optimization to the `torch.compile` caching mechanism, specifically targeting fine-tuned models. The new approach achieves sub-second cold boot times for fine-tuned language models (Llama 2 variants) and image models (SDXL). This improvement is available for new fine-tuned models created after the post date, with ongoing work to extend it to all models.