TechPrufer
Sign inGet started

Blogs›Cloudflare›Internal Data & Measurement Initiatives

Internal Data & Measurement Initiatives

Internal Data & Measurement Initiatives

3
posts
2025

Cloudflare has a history of leveraging its scale for unique internal projects and data analysis. This began with ambitious, large-scale data collection and analysis efforts, such as the 'Internet Mince Pie Database' which, while seemingly whimsical, demonstrated the company's ability to mobilize resources for structured data gathering and review. This has evolved to more technically focused initiatives like optimizing data infrastructure for analytics, improving system-level operations, and developing rigorous methodologies for Internet measurement. This post details the scientific practice of Internet measurement, emphasizing its opacity, the importance of methodology (data curation, modeling, validation), ethics, and representation. It introduces active vs. passive and direct vs. indirect measurement techniques, using examples like HTTP traffic spikes in Lviv and packet pair bandwidth estimation. The measurement lifecycle is presented as a core pattern for generating predictive insights.

2025

The tricky science of Internet measurement

10/27/2025

This post introduces the scientific practice of Internet measurement, detailing its challenges due to the Internet's opacity. It outlines the core components of measurement science: methodology (data curation, modeling, validation), ethics, and representation. It differentiates between active and passive measurement techniques, and direct and indirect measurement, providing examples for each. The post also describes the measurement lifecycle as a pattern for generating predictive insights, emphasizing the importance of high-quality data curation.

Data at Cloudflare scale- some insights on measurement for 1,111 interns

10/27/2025

This post details an internship project that explored detecting network middleboxes and connection tampering using Cloudflare's passive production data. It highlights the challenges of scale, noisy data, and lack of ground truth when analyzing large datasets. The project involved constructing IPTABLES rules to log TCP connections, sampling 1 in 10,000 connections, and analyzing the first 10 packets to develop tampering signatures. It also discusses the iterative process of identifying and mitigating noise sources like DoS traffic, port scans, and Happy Eyeballs, and the reliance on existing research for ground truth validation.

Internet measurement, resilience, and transparency: blog takeover from Cloudflare Research and friends

10/27/2025

This post announces a week-long series of blog posts from Cloudflare Research and friends, focusing on Internet measurement, resilience, and transparency. It highlights foundational concepts in internet measurement, the development of frameworks for internet resilience, advancements in post-quantum cryptography, and the analysis of network traffic and Linux networking stack. It also introduces the challenges and opportunities presented by AI agents and proposes new cryptographic protocols for secure and private interactions on the web. The post emphasizes the collaborative and open nature of their research, aiming to push forward a better internet.