TechPrufer
Get started
Atlassian
Atlassian logoAtlassian · Feature trail

Platform Dependency Management and Recoverability

2 posts · Apr 2022 – Nov 2025

What this trail covers

Atlassian's CPR program has significantly improved platform reliability and recoverability by systematically removing dependency tangles in its services architecture. This involved developing new platform services and tools for dependency analysis, rearchitecting the cloud infrastructure into distinct layers (the 'Layer Cake' model), and conducting extensive tabletop exercises to simulate disaster recovery. The program focuses on breaking circular dependencies, particularly hard dependencies, and reducing the blast radius of incidents.

All Atlassian trails
  1. Removing dependency tangles in the Atlassian Platform for increased reliability and recoverabilityThis post details the CPR program's approach to breaking circular dependencies in Atlassian's PaaS. It introduces the 'Layer Cake' model for restructuring infrastructure into ordered layers, emphasizing that components in layer N can only have hard dependencies on lower layers (N-1). It also describes the use of data-driven analysis (surveys, dependency mapping) and tabletop exercises to identify and prioritize risks. Specific risk categories like deployment, source code, artifacts, and build dependencies are discussed, along with pragmatic solutions such as developing low-level provisioning tools and Git mirroring solutions.
  2. Post-Incident Review on the Atlassian April 2022 outageThis post details a major incident where a misconfiguration led to the deletion of 775 customer sites, impacting 775 customers for up to 14 days. The root cause was a communication gap and insufficient system warnings during the execution of a script to delete a legacy Atlassian app. The script incorrectly received site IDs instead of app IDs, leading to site deletion. The response involved a cross-functional incident management team working 24/7 to restore sites. Preventative measures include implementing universal 'soft deletes', accelerating the Disaster Recovery (DR) program for large-scale deletion events, revising incident management processes for large-scale incidents, and creating a large-scale incident communications playbook.