TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

io_uring worker pool management

2 posts · Feb 2019 – Feb 2022

What this trail covers

Cloudflare is exploring and detailing the internal workings of io_uring, a high-performance asynchronous I/O API in the Linux kernel. This includes understanding how io_uring manages its worker thread pools for different types of I/O requests (bounded vs. unbounded), how to monitor and control these pools, and the implications for network I/O performance. The company is investigating the tracepoints and kernel mechanisms involved in worker creation and management to optimize its use of io_uring. This post also explores SOCKMAP, a new eBPF machinery for socket splicing, which aims to achieve zero-copy and zero-wakeup data forwarding purely in the kernel, though initial benchmarks showed it to be slower than other methods.

All Cloudflare trails
  1. Missing Manuals - io_uring worker poolThis post details the internal mechanisms of io_uring's worker pool management, differentiating between bounded and unbounded I/O requests. It explains how to monitor worker creation using kernel tracepoints and `perf`, and demonstrates how to force io_uring to spawn worker threads for socket operations by using the `IOSQE_ASYNC` flag. The post also touches upon the default limits for worker pools based on `RLIMIT_NPROC` and CPU count.
  2. SOCKMAP - TCP splicing of the futureThis post introduces SOCKMAP, a new eBPF machinery for TCP socket splicing. It details how SOCKMAP allows for zero-copy and zero-wakeup data forwarding purely within the kernel by attaching eBPF programs to an eBPF map containing socket descriptors. The post also presents benchmarks comparing SOCKMAP to naive read-write loops, splice(2), and io_submit, highlighting SOCKMAP's theoretical advantages but also its initial performance drawbacks and a discovered bug.