How we knew it was time to leave the cloud

This post details GitLab's decision to move from cloud-hosted CephFS to bare-metal infrastructure due to performance limitations and cost inefficiencies experienced with shared cloud resources. It highlights the challenges of achieving consistent high IOPS on the cloud, the concept of 'noisy neighbors', and the trade-offs between cloud scalability and bare-metal control. The post also emphasizes the importance of observability and detailed monitoring, using Prometheus and custom dashboards to diagnose issues like OSD Journal Latency spikes, which directly impacted system availability.