TechPrufer
Sign inGet started

Blogs›Shopify›Container Resource Management with Systemd and CGroups

Container Resource Management with Systemd and CGroups

Container Resource Management with Systemd and CGroups

3
posts
2014–2022

This feature thread tracks the development and enhancement of Shopify's approach to containerization, focusing on the use of Docker and CoreOS for building an internal cloud. Initial efforts involved containerizing existing applications, managing secrets, routing, monitoring, and bulletproofing the system. Subsequent development has focused on container resource management, addressing issues with systemd processes within containers not adhering to Kubernetes resource limits due to cgroup driver mismatches. The goal is to ensure efficient and predictable resource utilization for containerized applications.

2022

Spin Infrastructure Adventures: Containers, Systemd, and CGroups - Shopify

7/15/2022

This post details a deep dive into a container resource management issue where systemd processes within Spin instances were not respecting Kubernetes pod memory limits. The investigation revealed that the default cgroupfs driver used by containerd on Google's Container Optimized OS (COS) was the root cause, leading to 'escaped' cgroups and unlimited resource consumption. Experiments with Docker and Podman highlighted that Podman's systemd cgroup driver correctly delegated resource management. The post identifies the upcoming change in COS to default to the systemd cgroup driver as the solution, and discusses the implications of bind-mounting /sys/fs/cgroup read-only.

The Story Behind Shopify’s Isospin Tooling - Shopify

7/5/2022

This post introduces Isospin, Shopify's systemd-based tooling for running applications within the Spin cloud development platform. It details how Isospin leverages systemd's unit management, template unit files, and generators to dynamically configure services, manage dependencies, and isolate processes within a single Linux VM. The post also describes additional tooling developed to address port collisions using a hashing process and to improve service readiness by utilizing systemd's notify socket mechanism.

2014

Building an Internal Cloud with Docker and CoreOS - Shopify

10/15/2014

This post introduces Shopify's adoption of Docker and CoreOS to build an internal cloud for scaling and managing applications. It highlights the benefits of containers (portability, density, consistency, developer/ops workflows) and outlines future deep-dives into containerizing applications, managing secrets, routing, monitoring, bulletproofing, and provisioning. The post emphasizes the shift in responsibility towards developers owning containers and ops focusing on infrastructure.