Launching Cloudflare’s Gen 13 servers- trading cache for cores for 2x edge compute performance

This post details the transition to Cloudflare's Gen 13 server fleet, featuring AMD EPYC Turin processors. It highlights the challenge of reduced L3 cache per core compared to Gen 12 servers and the resulting latency issues when running the legacy FL1 request handling layer. The post describes how performance counter analysis identified cache miss rates and memory fetch latency as key problems. It also details incremental gains achieved through hardware tuning and PQOS, but emphasizes that the true unlock came from the ongoing FL2 rewrite in Rust. FL2's cleaner architecture and leaner memory access patterns removed the dependency on large L3 caches, allowing Gen 13's higher core counts to deliver up to 2x throughput gains and a 50% boost in performance per watt compared to Gen 12, all while maintaining latency SLAs.