Proxy Framework Development

5/26/2023

This post introduces Oxy's hook-based extensibility model and dependency injection capabilities, allowing developers to programmatically modify all aspects of a proxy's lifecycle. It details how to implement custom logic for connection handling, authorization, and header manipulation using Rust traits and opaque extensions for state management. The implementation details of hooks and the use of `Any` for type erasure are also discussed.

4/20/2023

This post introduces three new specialized microservices (Splicer, Bumblebee, and Fish) that extend the Oxy proxy framework. Splicer handles generic data piping between sockets, Bumblebee upgrades IP flows to TCP sockets using kernel-level TCP, and Fish forwards IP packets using soft-unicast IPs by rewriting packets and configuring conntrack via netlink. The post details how these services improve reliability by adhering to the Unix philosophy of single responsibility and enabling graceful restarts through file descriptor passing (SCM_RIGHTS) and state transfer. It also discusses the use of JoinSet for managing concurrent tasks and cancellation for pausing requests during upgrades.

3/2/2023

This post introduces Oxy, Cloudflare's new Rust-based proxy framework. It details Oxy's architecture, capabilities, and extensibility model. Key features include support for various on-ramps (HTTP/3, TCP, UDP, Proxy Protocol) and off-ramps (HTTP/1, HTTP/2, UDP, TCP, IP), multi-layer traffic processing with forced decapsulation, advanced tunneling (CONNECT-UDP, CONNECT-IP), TLS integration with BoringSSL (FIPS-compliant and standard versions), and built-in support for bootstrapping, configuration, asynchronous runtime, seccomp hardening, graceful restarts, monitoring (Sentry, Prometheus, Kibana), profiling, and integration testing. Oxy is used in products like Zero Trust Gateway, iCloud Private Relay, and internal egress routing services.

9/14/2022

This post introduces Pingora, a new in-house HTTP proxy built in Rust. It addresses limitations of NGINX, including architectural drawbacks that hurt performance (unbalanced load, poor connection reuse) and difficulties in adding new functionality. Pingora uses Rust for memory safety and performance, a custom HTTP library for flexibility with non-RFC compliant traffic, a multithreaded work-stealing scheduler (Tokio), and an event-based programmable interface similar to NGINX/OpenResty. In production, Pingora has reduced median TTFB by 5ms and 95th percentile TTFB by 80ms due to better connection reuse, saving 434 years of handshake time daily. It also enables faster feature development, such as adding HTTP/2 upstream support and integrating with Cache Reserve.

3/20/2022

This post introduces MASQUE (Multiplexing And Secure Communication over QUIC Express) as a protocol for proxying UDP traffic over QUIC. It details the use of QUIC's DATAGRAM frame and extended HTTP/3 CONNECT requests to establish UDP tunnels. The post explains the encapsulation process, MTU considerations for tunneled UDP datagrams, and the potential for nested tunneling for enhanced privacy. Cloudflare's quiche library's support for the DATAGRAM frame is highlighted.

3/19/2022

This post introduces the fundamental concepts of forward proxying, contrasting it with reverse proxies like the Cloudflare CDN. It details the evolution of proxy protocols, including TCP, SOCKS, and HTTP CONNECT across HTTP/1.1, HTTP/2, and QUIC/HTTP/3. The post explains the challenges of TCP head-of-line blocking and how HTTP/2 and QUIC address multiplexing and stream management for proxying. It also mentions MASQUE as a collaboration effort to design efficient proxy protocols based on IETF standards and provides examples of Cloudflare's use in iCloud Private Relay.

2/17/2022

This post introduces 'tubular', an open-source eBPF-based tooling for extending the Linux socket API. It addresses limitations of the BSD sockets API by enabling services to handle millions of IPs, co-locate multiple services on the same port across different addresses, and support listening on all 2^16 ports for products like Spectrum. Tubular uses eBPF maps for state persistence and a longest prefix match (LPM) trie for efficient binding resolution and precedence encoding. It also details methods for registering sockets with eBPF programs, including using `pidfd_getfd` to access sockets from foreign processes.

10/20/2021

This post details the debugging of a connectivity issue when enabling Cloudflare Tunnel to use QUIC (UDP) for its connection to the Cloudflare network. The problem stemmed from the Linux kernel's default behavior of selecting the wrong source IP address for UDP responses when an interface had multiple IPs and a default route was present. The solution involved using `recvmsg` and `sendmsg` with `IP_PKTINFO` control messages to explicitly set the source IP address, and a bug fix in the quic-go library to correctly pass `SOL_IP` instead of `SOL_TCP` as the control message level during `sendmsg` calls.

10/26/2020

This post details the technical implementation of gRPC support on the Cloudflare edge. It explains the challenges of gRPC's reliance on HTTP/2 trailers and bidirectional streaming, and how Cloudflare overcame these by converting gRPC messages to HTTP/1.1 gRPC-web format internally. It also describes the development of a standalone origin proxy for HTTP/2 origin support and the interoperability testing conducted.

10/12/2019

This post details the evolution of socket lookup strategies at Cloudflare's edge to handle a large number of public IP addresses and services. It covers the challenges of managing numerous listening sockets, the benefits and drawbacks of using INADDR_ANY, the limitations of the sockets API for port sharing, and the introduction of SO_BINDTOPREFIX for constrained wildcard binding. It also introduces TPROXY as a mechanism to handle services listening on all ports, as implemented in Spectrum.