TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

Proxy Framework Development

10 posts · Oct 2019 – Jun 2026

What this trail covers

Cloudflare is developing Oxy, a next-generation, Rust-based proxy framework designed to handle high-load scenarios and support a wide array of communication protocols. This includes exploring and implementing MASQUE (Multiplexing And Secure Communication over QUIC Express) to enable efficient tunneling of UDP traffic over QUIC, building on the DATAGRAM frame extension and extended CONNECT requests. This allows for proxying of protocols like HTTP/3 and other UDP-based applications, with consideration for kernel bypass techniques and efficient data handling. The Images service utilizes hyper, an open-source HTTP library for Rust, for managing socket connections and handling data flow between services. Debugging efforts focused on a race condition within hyper's buffer management when handling large responses, leading to data truncation. The investigation involved deep system-level tracing with strace to understand syscall behavior during successful and failing requests, ultimately identifying a timing-sensitive race condition in hyper's buffer flushing mechanism that occurred under high concurrency and large data loads. A four-line code fix was implemented to address the issue.

All Cloudflare trails
  1. How we found a bug in the hyper HTTP libraryThis post details the debugging of a race condition in the hyper HTTP library that caused intermittent data truncation for large image responses when using the Cloudflare Images binding. The bug manifested as a `200 OK` response with incomplete image data, traced to hyper's buffer management during high-concurrency socket writes. The investigation involved extensive use of `strace` to compare syscall behavior between successful and failing requests, revealing a timing-sensitive issue where hyper's outbound buffer could fill up before all data was flushed, leading to premature connection shutdown. The fix involved adjusting hyper's buffer flushing logic to ensure all data was written before shutting down the connection.
  2. Cloudflare just got faster and more secure, powered by RustThis post details the transition from the legacy NGINX/LuaJIT-based FL1 system to a new Rust-based implementation called FL2, built on the Oxy proxy framework. It highlights the architectural shift towards Rust for improved performance, safety (eliminating memory safety issues and data races), and modularity. Key improvements include the implementation of graceful restarts using systemd socket activation and custom coordination mechanisms (shellflip), and a strict module framework for FL2 that enforces no IO within product logic modules and defines explicit phases with defined inputs and outputs. The post also touches on the historical evolution of FL from its NGINX/PHP origins.
  3. Open sourcing Pingora: our Rust framework for building programmable network servicesThis post announces the open-sourcing of Pingora, a Rust async multithreaded framework used to build HTTP proxy services. It details Pingora's capabilities, including support for HTTP/1, HTTP/2, TLS, TCP/UDS, gRPC, and websocket proxying, with customizable load balancing and failover strategies. It highlights Pingora's focus on memory safety and performance, and provides an example of building a simple load balancer using its programmable API with filters and callbacks. The post also outlines future plans and contribution guidelines.
  4. How Oxy uses hooks for maximum extensibilityThis post introduces Oxy's dependency injection capabilities and hook system for extending proxy functionality programmatically. It details how developers can implement hook traits (e.g., EgressHook, HttpRequestHook) to modify connection handling, implement authorization schemes using opaque extensions for state management (e.g., PSK authorization), and add custom headers like 'Proxy-Status' using connection context. The implementation leverages Rust's generics and type erasure with the `Any` trait for framework-level flexibility without sacrificing type safety for application developers.
  5. Oxy: Fish/Bumblebee/Splicer subsystems to improve reliabilityThis post details the introduction of three specialized subsystems (Splicer, Bumblebee, and Fish) within the Oxy proxy framework to improve reliability and handle soft-unicast IP space. Splicer handles generic data piping between sockets, Bumblebee upgrades IP flows to TCP sockets using unshare and tun devices, and Fish manages layer 3 egress using soft-unicast IPs via netlink and packet rewriting. The post also elaborates on the use of Unix Domain Sockets with SCM_RIGHTS for inter-process communication and file descriptor passing, and discusses strategies for graceful restarts using Tokio's JoinSet and cancellation mechanisms.
  7. From IP packets to HTTP: the many faces of our Oxy frameworkThis post details how Cloudflare's Oxy framework handles traffic at different OSI layers, from raw IP packets to application protocols. It explains IP tunneling for multi-tenanted environments (Cloudflare One, WARP, Magic WAN) using connected and unconnected IP tunnels. It also describes IP flow tracking using the etherparse crate and how Oxy upgrades IP flows to TCP streams by leveraging Linux network namespaces and TUN interfaces for stateful Network Address Translation (NAT).
  8. Oxy is Cloudflare's Rust-based next generation proxy frameworkThis post introduces Oxy, Cloudflare's new Rust-based proxy framework. It details Oxy's architecture, capabilities, and extensibility model, highlighting its use in Zero Trust Gateway, iCloud Private Relay, and internal egress routing. Key features include multi-layer traffic processing (L3-L7), flexible on-ramps (HTTP/1/2/3, TCP, UDP, IP) and off-ramps, stateful tunneling, advanced HTTP tunneling (CONNECT-UDP, CONNECT-IP), dynamic TLS certificate generation, and integration with BoringSSL (including a FIPS-compliant version). Oxy also provides built-in support for configuration, asynchronous runtime, seccomp hardening, graceful restarts, comprehensive monitoring (Sentry, Prometheus, Kibana, tracing), audit logging, and an integration testing framework.
  9. How we built Pingora, the proxy that connects Cloudflare to the InternetCloudflare has built and deployed Pingora, a new in-house HTTP proxy written in Rust, to replace NGINX. Pingora handles over a trillion requests daily, offers improved performance and efficiency (one-third the CPU/memory of the previous infrastructure), and enables new features like HTTP/2 upstream support for gRPC. Key design decisions include using Rust for memory safety and performance, building a custom HTTP library to handle non-RFC compliant traffic, and employing a multithreaded, work-stealing scheduler (Tokio) for better resource sharing and connection reuse. Pingora's event-based programmable interface allows for clean separation of business logic and proxy logic, facilitating faster feature development and easier migration for engineers familiar with NGINX/OpenResty.
  10. Unlocking QUIC’s proxying potential with MASQUEThis post introduces MASQUE (Multiplexing And Secure Communication over QUIC Express) as a protocol for proxying UDP traffic over QUIC. It details the use of QUIC's DATAGRAM frame and extended HTTP/3 CONNECT requests to establish UDP tunnels. The post explains the encapsulation process, MTU considerations for tunneled UDP datagrams, and the potential for nested tunneling for enhanced privacy. Cloudflare's quiche library's support for the DATAGRAM frame is highlighted.
  11. A Primer on ProxiesThis post introduces the concept of forward proxying and contrasts it with reverse proxying. It details the transport basics of TCP, including its reliable byte stream nature and the issue of head-of-line blocking. The post then explores various proxy protocols like SOCKS, transparent proxies, Layer 4 proxies (Spectrum), and HTTP CONNECT. It provides in-depth explanations of how the HTTP CONNECT method works across HTTP/1.1, HTTP/2, and QUIC/HTTP/3, highlighting the differences in stream handling and connection management. The post emphasizes the use of HTTP as a substrate for proxying and mentions the ongoing development of MASQUE as a modern, efficient proxy protocol.
  12. Production ready eBPF, or how we fixed the BSD socket APIThis post introduces 'tubular', an open-source eBPF-based tooling for extending the Linux socket API. It addresses limitations of the BSD sockets API by enabling services to handle millions of IPs, co-locate multiple services on the same port across different addresses, and support listening on all 2^16 ports for products like Spectrum. Tubular uses eBPF maps for state persistence and a longest prefix match (LPM) trie for efficient binding resolution and precedence encoding. It also details methods for registering sockets with eBPF programs, including using `pidfd_getfd` to access sockets from foreign processes.
  13. Getting Cloudflare Tunnels to connect to the Cloudflare Network with QUICThis post details the debugging process for enabling Cloudflare Tunnel to connect to the Cloudflare network using QUIC, a UDP-based protocol. It highlights a specific connectivity issue arising from multiple IP addresses on an interface and the kernel's heuristics for selecting a source address for UDP packets, which differs from TCP. The post explains how to use `recvmsg` and `sendmsg` with control information (specifically `IP_PKTINFO`) to explicitly set the source address, and details a bug in the quic-go library that was fixed to enable this functionality. The successful connection of Cloudflare Tunnel using QUIC is demonstrated.
  14. Road to gRPCThis post details the technical implementation of gRPC support on the Cloudflare edge. It explains the challenges of gRPC's reliance on HTTP/2 trailers and bidirectional streaming, and how Cloudflare overcame these by converting gRPC messages to HTTP/1.1 gRPC-web format internally. It also describes the development of a standalone origin proxy for HTTP/2 origin support and the interoperability testing conducted.
  15. It's crowded in here!This post details the evolution of socket lookup strategies at Cloudflare's edge to handle a large number of public IP addresses and services. It covers the challenges of managing numerous listening sockets, the benefits and drawbacks of using INADDR_ANY, the limitations of the sockets API for port sharing, and the introduction of SO_BINDTOPREFIX for constrained wildcard binding. It also introduces TPROXY as a mechanism to handle services listening on all ports, as implemented in Spectrum.