However improbable: The story of a processor bug

This post details the investigation and resolution of a mysterious processor bug affecting Intel Xeon E5-2650 v4 (Broadwell) processors. It highlights the process of identifying and debugging hardware-level issues that manifest as unexpected software crashes (SIGSEGV, SIGABRT, SIGILL) and the use of core dumps for post-mortem analysis. The discovery of the bug required meticulous examination of system logs and core dumps to rule out software bugs and eventually pinpoint the hardware as the source of the problem.