Running fine-tuned models on Workers AI with LoRAs

What this post added

This post introduces the open beta of fine-tuned inference with LoRAs on Workers AI, supporting Mistral, Gemma, and Llama 2 models. It details the technical concepts of fine-tuning and LoRA, explaining how LoRA adapters reduce trainable parameters and GPU memory requirements. The post also dives into the implementation details of multi-tenant LoRA serving on Cloudflare's platform, highlighting the use of the Punica CUDA kernel and global cache optimizations to manage GPU resources efficiently and achieve low inference latency.