Serverless servers: Efficient serverless Node.js with in-function concurrency

This post introduces and details the implementation of in-function concurrency for Vercel Functions. It explains the historical limitations of the one-to-one invocation-to-instance model in serverless for interactive workloads, the architectural changes involving a streaming connection and a Rust-powered runtime, and the resulting efficiency gains (20%-50% reduction in compute usage and cost). It also discusses trade-offs such as potential latency increases for CPU-bound workloads and the backward-incompatible nature of the change.