An SA story about hyperscaling GitLab Runner workloads using Kubernetes

What this post added

This post details a real-world scenario of scaling GitLab Runner workloads using Kubernetes. It highlights the challenges of resource optimization in a shared CI cluster, where initial attempts to minimize pod resources led to increased developer toil and debugging. The post introduces the concept of tiered runner resource profiles ('large-container', 'xlarge') as a solution to accommodate diverse CI job requirements, and discusses the 'human toil factor' and 'opportunity cost tax' associated with overly granular resource management. It also touches upon the benefits of Kubernetes elastic scaling and the potential for using spot compute for cost savings.