Torch compile caching for inference speed

This post details the implementation of caching for `torch.compile` artifacts within Replicate's infrastructure. It explains how compiled code is cached across model container lifecycles, leading to 2-3x faster cold starts for PyTorch models. The caching mechanism involves keying cache files on model version, storing them near GPU nodes, and updating the cache upon graceful container shutdown. Performance improvements are quantified with specific examples of reduced boot times for models like flux-kontext-dev, flux-schnell, and flux.1-dev-lora.