Temporal Cloud Persistence Layer
Go shadowing + bad choices = our first data loss bug in years

Go shadowing + bad choices = our first data loss bug in years

3/3/2021

What this post added

This post details a data loss bug in Temporal's persistence layer caused by Golang variable shadowing, which occurred when Cassandra returned specific errors during cluster instability. It explains the debugging process, the root cause, and the impact on workflow creation. The post also outlines corrective actions including adding tests for dependency-level failures, implementing a linter for variable shadowing, improving graceful handling of corrupted workflows, and enhancing communication mechanisms for critical bugs.

Read the original post ↗