So long, and thanks for all the fish- how to escape the Linux networking stack

This post details Cloudflare's efforts to push beyond the limitations of the Linux networking stack for specific high-performance use cases, particularly in managing soft-unicast IP addresses. It introduces 'fish' (SLATFATF), a custom service for proxying IP packets using soft-unicast address space, and explores the challenges of coexisting packet forwarding with socket binding. The post details the investigation into Netfilter/conntrack interactions and the development of solutions using Netlink and, more effectively, kernel features like TCP_REPAIR and TCP Fast Open to achieve precise control over connection state and avoid collisions, thereby enabling more efficient and reliable network operations for services like WARP.