Data at Cloudflare scale- some insights on measurement for 1,111 interns

What this post added

This post details an internship project that explored detecting network middleboxes and connection tampering using Cloudflare's passive production data. It highlights the challenges of scale, noisy data, and lack of ground truth when analyzing large datasets. The project involved constructing IPTABLES rules to log TCP connections, sampling 1 in 10,000 connections, and analyzing the first 10 packets to develop tampering signatures. It also discusses the iterative process of identifying and mitigating noise sources like DoS traffic, port scans, and Happy Eyeballs, and the reliance on existing research for ground truth validation.