A postmortem of three recent issues

This postmortem details three distinct infrastructure bugs that intermittently degraded Claude's response quality between August and September 2025. The first bug involved misrouting short-context requests to servers configured for the 1M token context window, exacerbated by a load balancing change. The second bug was a misconfiguration on TPU servers causing token generation errors, leading to unexpected character outputs or syntax errors. The third bug stemmed from a latent issue in the XLA:TPU compiler triggered by a sampling code rewrite, affecting token selection due to mixed precision arithmetic and an approximate top-k operation. The post also discusses challenges in detection and resolution due to overlapping symptoms, platform-specific variations, and privacy controls limiting engineer access to user interactions. It outlines improvements in evaluation sensitivity, continuous production monitoring, and faster debugging tooling, emphasizing the continued importance of user feedback.