Serving four million Riffusion requests in two days

This post details how Baseten handled a surge of 4 million Riffusion requests in two days by scaling beyond default autoscaling limits. Key technical contributions include implementing a caching layer using a Postgres table to store and retrieve common model responses, thereby reducing unnecessary model invocations. It also highlights the strategic use of a mix of spot and on-demand instances for cost-effective scaling and discusses trade-offs accepted for a hobby project, such as slightly higher latency and error rates, to manage costs.