Fluid compute: Evolving serverless for AI workloads

This post details the architectural mismatch between traditional serverless computing and LLM interactions, highlighting how LLM workloads involve extended execution times and periods of inactivity. It introduces Fluid compute as a new model that reuses existing resources before scaling new ones, processing multiple AI inference requests simultaneously within a single instance to avoid cold starts and reduce costs. Fluid compute also strategically places resources in regions close to data for optimized performance and consistency. The post emphasizes the security aspects of Fluid compute, including Vercel Firewall for Layer 7 protection and secure instance architecture with TCP tunnels, as well as enhanced reliability through automatic failover.