TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

Go Compiler Bug Analysis

2 posts · Jun 2015 – Oct 2025

What this trail covers

Cloudflare's Go compiler bug analysis thread continues with the discovery and debugging of a race condition in the arm64 compiler. This post details the investigation into sporadic panics and segmentation faults observed in a kernel configuration service, tracing the root cause to an interaction between the Go runtime's asynchronous preemption mechanism and the `github.com/vishvananda/netlink` library. The analysis highlights the challenges of debugging rare, intermittent bugs at scale and the process of identifying upstream issues.

All Cloudflare trails
  1. How we found a bug in Go's arm64 compilerThis post details the discovery and debugging of a race condition in Go's arm64 compiler that caused sporadic panics and segmentation faults. The investigation involved analyzing coredumps, correlating fatal errors with recovered panics, and identifying the `github.com/vishvananda/netlink` library as a common factor. The root cause was traced to an interaction between the Go runtime's asynchronous preemption mechanism (SIGURG, `asyncPreempt`) and the `NetlinkSocket.Receive` function, leading to stack corruption and invalid memory access when dereferencing the Go scheduler's `m` struct.
  2. DNS parser, meet Go fuzzerThis post details the application of go-fuzz to the `github.com/miekg/dns` Go DNS library. It describes the fuzzing process, including the creation of an initial corpus from pcap data and the use of go-fuzz to discover bugs. The post highlights the discovery of two types of bugs: general robustness issues leading to panics (e.g., 'index out of bounds') and a specific security vulnerability related to handling pointers to empty DNS names, which could lead to a Denial of Service. It also demonstrates how fuzzing can be used to compare the behavior of different parser versions and to uncover bugs related to buffer reuse in the NSEC record packing logic.
  3. Go has a debugger—and it's awesome!Introduces and demonstrates the use of `godebug`, a cross-platform Go debugger that achieves its functionality by rewriting source code to inject debugging probes. It highlights the benefits of this approach for Go's single-binary philosophy and provides examples of its usage in a production environment (RRDNS). It also discusses the broader pattern of source rewriting in Go tooling and suggests the need for a standardized framework.