Scaling to 1 million concurrent sandboxes in seconds | Modal Blog

What this post added

Rebuilt the core sandbox platform to support millions of concurrent sandboxes and tens of thousands of sandbox creations per second. Eliminated central bottlenecks and global coordination from the critical path by implementing a horizontally scalable fleet of scheduling servers that directly communicate with workers via RPC. Workers publish state to Redis streams asynchronously, removing datastores from the critical path of sandbox creation. Optimized container scheduling and startup by simplifying the path to a layer of load balancers that create containers directly on the worker fleet. Addressed issues with the `rtnl` lock in the Linux kernel during high-volume container startup.