TechPrufer
Get started
Datadog
Datadog logoDatadog · Feature trail

Postgres JIT Compiler Bug Investigation

1 post · Jan 2025

What this trail covers

Datadog's engineering team investigated and resolved a critical segmentation fault in their Postgres clusters. The issue was traced to a bug in the Arm64 JIT compiler within LLVM. The investigation involved isolating the problem, reproducing it on Arm64 instances, and debugging a Postgres build with JIT debugging support enabled. The root cause was identified as an infinite loop in the Arm64 JIT compilation process for certain expressions, leading to a segfault or a 100% CPU utilization. An upstream fix was developed and applied.

All Datadog trails
  1. Unraveling a Postgres segfault that uncovered an Arm64 JIT compiler bug | DatadogThis post details the investigation and resolution of a critical segmentation fault in Datadog's Postgres clusters, which was traced back to a bug in the Arm64 JIT compiler (specifically within LLVM). The investigation involved isolating the issue, reproducing it on Arm64 instances, and debugging a Postgres build with JIT debugging support enabled. The root cause was identified as an infinite loop in the Arm64 JIT compilation process for certain expressions, leading to a segfault or a 100% CPU utilization.