The Case of the Recursive Resolvers

This post details the challenges and root cause analysis of a 24-hour outage caused by an attempt to enable DNSSEC on slack.com. It explains the DNS protocol, DNSSEC, and Slack's DNS infrastructure. The post describes three unsuccessful attempts to enable DNSSEC, detailing the specific issues encountered in each, including ISP outages, VPN resolver strictness, and recursive resolver caching behavior leading to NODATA responses. The incident highlights the complexities of DNSSEC rollouts and the importance of understanding resolver caching mechanisms.