How we built Pingora, the proxy that connects Cloudflare to the Internet

This post introduces Pingora, a new in-house HTTP proxy built with Rust, designed to replace NGINX and handle Cloudflare's massive scale of over a trillion requests per day. It addresses architectural limitations of NGINX, such as its worker process model and per-worker connection pools, which previously hindered performance and connection reuse. Pingora's multithreading and work-stealing approach, combined with a custom HTTP library and an event-based programmable interface, significantly improves performance (reducing TTFB and handshake time) and enables faster development of new features, including HTTP/2 upstream support and gRPC, which directly benefit Cloudflare's security and performance products.