Fixing an old hack - why we are bumping the IPv6 MTU

What this post added

This post details the decision to increase the IPv6 MTU on Cloudflare's edge servers from a hardcoded 1280 bytes to 1400 bytes. This change was driven by the need to support UDP traffic forwarding for Cloudflare Spectrum, which requires a larger MTU to accommodate encapsulated VPN packets. The post explains the original problem with Path MTU discovery and ICMP packet drops in the ECMP routing setup, the temporary fix of reducing the MTU, and the analysis of eyeball MTU distributions to select a new, more appropriate MTU value that minimizes reliance on Path MTU Detection.