TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

TCP Congestion Control Algorithms

2 posts · Jun 2012 – May 2026

What this trail covers

Cloudflare's TCP congestion control algorithms have evolved to be more responsive to the dynamic and often sporadic nature of network congestion, particularly on mobile networks. By analyzing real-world network data from its global network, Cloudflare tunes its algorithms on a per-network basis to optimize performance, especially for networks experiencing high loss rates. This continuous tuning based on network growth and data analysis is core to Cloudflare's mission of building a better Internet. This post details how Cloudflare leverages its massive Free Plan user base and passive log data, combined with Real User Measurement (RUM) data, to extrapolate and predict user experience, enabling more fine-grained optimization of congestion control algorithms beyond the lifetime of a single connection. Early results show performance increases averaging 10% faster than the prior baseline, with ongoing rollout to all customers planned.

All Cloudflare trails
  1. When "idle" isn't idle: how a Linux kernel optimization became a QUIC bugThis post details a bug discovered in the CUBIC congestion control algorithm when used with Cloudflare's quiche QUIC implementation. The bug, stemming from a Linux kernel optimization for handling idle periods, caused the congestion window (cwnd) to become permanently pinned at its minimum after a congestion collapse event, preventing recovery. The issue was traced to the way quiche's user-space implementation of the idle period adjustment interacted with CUBIC's state machine, leading to rapid oscillations between recovery and congestion avoidance states. A fix was implemented by adjusting the `congestion_recovery_start_time` calculation in `on_packet_sent` to prevent it from being set in the future, effectively resolving the oscillation and allowing CUBIC to recover.
  2. Measuring characteristics of TCP connections at Internet scaleThis post presents aggregate insights about TCP connections established through Cloudflare's global CDN, sharing characteristics of TCP connections that account for about 70% of HTTP requests to Cloudflare. The dataset includes socket-level metadata captured via the Linux kernel's TCP_INFO struct, alongside the SNI and the number of requests made during the connection. The analysis focuses on connections that close gracefully with a FIN packet and have at least one successful HTTP request, presenting characteristics like packet counts and bytes sent. It visualizes these characteristics using cumulative distribution function (CDF) graphs, specifically their empirical equivalents, with log-scaled axes to account for extreme values. The post details the distribution of response packets sent by Cloudflare servers to clients, breaking it down by HTTP protocol version (HTTP/1.X vs. HTTP/2), and analyzes the ratio of packets sent by servers to packets received from clients.
  3. How Cloudflare uses the world’s greatest collection of performance data to make the world’s fastest global network even fasterThis post details how Cloudflare is leveraging its global network visibility and massive Free Plan user base to improve congestion control algorithms. It explains the challenges of Internet congestion, the limitations of existing connection-lifetime-bound algorithms (like loss-based and BBR), and how Cloudflare's unique perspective allows for optimization based on broader network data. The core technical contribution is the method of extrapolating Real User Measurement (RUM) data from passive logs to predict user experience at scale. This allows for more fine-grained optimization of congestion control parameters across the entire network, leading to an average of 10% performance improvement. The post also mentions the migration to a new Rust-powered network technology stack for greater algorithmic flexibility.
  4. When the window is not fully open, your TCP stack is doing more than you thinkThis post details the intricacies of Linux TCP receive buffer and window management, explaining how skmem_rb, sk_rcvbuf, and tcp_adv_win_scale interact. It highlights the problem of exceeding the memory budget due to metadata overhead and the mechanisms like TCP Coalesce and TCP Collapse used by the kernel to manage memory. The post also discusses the impact of RcvQDrop and the importance of reserving space for metadata, even when the advertised window is open.
  5. Optimizing TCP for high WAN throughput while preserving low latencyThis post details the tuning of Linux kernel sysctl parameters (`net.ipv4.tcp_rmem`, `net.ipv4.tcp_wmem`, `net.ipv4.tcp_adv_win_scale`) to optimize TCP performance for high WAN throughput while preserving low latency. It explains the concepts of TCP receive window, Linux autotuning, and the trade-offs involved in setting buffer sizes. The chosen sysctl values aim to achieve a Bandwidth-Delay Product (BDP) of 128 MiB by setting `tcp_rmem` to 8192 262144 536870912 and `tcp_adv_win_scale` to -2, balancing the need for large windows with the desire to avoid latency spikes caused by TCP queue collapsing.
  7. When TCP sockets refuse to dieThis post details the behavior of TCP sockets in various states (SYN-SENT, SYN-RECV, ESTABLISHED) and how TCP keepalives and TCP_USER_TIMEOUT interact to manage connection timeouts. It highlights that idle ESTABLISHED connections can linger indefinitely without keepalives and explains the conditions under which TCP_USER_TIMEOUT influences keepalive behavior. The post also touches upon the SYN-RECV state and the TCP_DEFER_ACCEPT feature.
  8. Traffic Acceleration with Cloudflare Mobile SDKThis post introduces the Cloudflare Mobile SDK's 'Traffic Acceleration' feature, which replaces TCP with a custom UDP-based transport protocol. This allows for performance tuning, bug fixes, and incremental updates to the protocol without kernel coordination, and bypasses middle-box ossification. The SDK aims to reduce latency, drive down network timeouts, and improve app user experiences by implementing novel transport algorithms. Early results show significant reductions in network response time and timeouts, leading to increased conversions for e-commerce apps.
  9. Why we use the Linux kernel's TCP stackThis post explains why Cloudflare primarily uses the Linux kernel's TCP stack for most operations, citing its rich feature set, debugging capabilities, and ecosystem support. It highlights the trade-offs of kernel bypass for latency and performance, and details Cloudflare's use of 'partial kernel bypass' with Solarflare EFVI and Netmap for DDoS mitigation to offload iptables rules to userspace, avoiding IRQ storms. The post also discusses the practical limitations of full userspace TCP stacks for general use, including interference with debugging tools and firewall features.
  10. Optimizing Your Linux Stack for Maximum Mobile Web PerformanceThis post details the challenges of mobile network performance at Layer 4, explaining how default TCP configurations are ill-suited for mobile connections. It introduces TCP Congestion Control, its role in preventing Internet congestion, and how it can misinterpret mobile network issues (like interference-induced packet loss) as congestion. The article uses a water utility analogy to explain congestion control algorithms and the difference between congestive loss and random loss. It highlights the limitations of older Linux kernels (2.6.32.x) and recommends upgrading to newer versions (e.g., 2.6.38) to benefit from improved defaults like a higher initial congestion window (initcwnd) and initial receive window (initrwnd). The post also briefly mentions bandwidth estimation algorithms like Vegas, Veno, and Westwood+ as reactive approaches to lossy networks.
  11. Why mobile performance is difficultThis post details the challenges of mobile network performance due to high latency, variability, and packet loss caused by factors other than congestion. It explains how TCP's congestion avoidance mechanisms, designed for wired networks, react negatively to mobile network conditions, leading to unnecessary slowdowns. The post also highlights packet reordering as another issue in mobile networks. Cloudflare's solution involves customizing TCP stack parameters on web servers and dynamically classifying connections to optimize performance for different network types.
  12. Mobile Web Performance: Optimizing TCP Congestion Control AlgorithmsThis post details Cloudflare's initial efforts to tune its TCP Congestion Control Algorithm based on real-world network data. It explains how mobile network behavior differs from traditional wire-line networks, leading to suboptimal performance with default algorithms. Cloudflare began mapping network behavior and tuning its algorithm to be responsive to network characteristics, specifically noting improvements for mobile networks with high loss rates. The post highlights the benefit of Cloudflare's scale in gathering data to continuously tune its network for optimal performance.