Cloud Development Environments

1 post · Jun 2022 – Aug 2022

What this trail covers

Shopify has evolved its cloud-based development environments, with a focus on the Spin tool. This post details the challenges and strategies for migrating the Shopify Fulfillment Network (SFN) codebase to Spin, highlighting the complexities of managing dependencies, handling frequent code changes that cause regressions, and the human factors influencing adoption. The team focused on enabling core validation flows to transition from a 'Spiral of Struggle' to a 'Loop of Happy Equilibrium' where developers can efficiently work with complex, multi-repository projects in the cloud.

All Shopify trails

Aug 2022Spin Cycle: Shopify’s SFN Team Overcomes a Cloud-Development Spiral - ShopifyThis post details the challenges of migrating the SFN codebase to the Spin cloud development environment. It outlines the technical problem of managing dependencies (running in-container, mocking, or using shared environments) and the human problem of frequent code changes causing regressions. The solution involved a focused effort to enable basic functionality for common validation flows and encouraging other developers to use Spin and contribute missing functionality, aiming to shift towards a positive feedback loop.
Jul 2022The Story Behind Shopify’s Isospin Tooling - ShopifyThis post details the development and implementation of Isospin, a systemd-based tooling that forms the core of how applications are run within Spin. Isospin leverages systemd's unit management, template units, and generators to dynamically create and manage service dependencies, isolate services, and handle port collisions. It also incorporates systemd's service readiness notifications for more robust application startup. Specific contributions include the use of template units for running multiple copies of dependency services per application, generators for dynamically creating service graphs based on runtime configuration, and a port assignment service to avoid collisions. The `notify-port` script is also introduced to leverage systemd's `NOTIFY_SOCKET` for explicit service readiness communication.
Jun 2022The Journey to Cloud Development: How Shopify Went All-in on Spin - ShopifyThis post details the evolution of Shopify's development environments from a homegrown Ruby program ('dev') and 'railgun' VM to cloud-based solutions. It describes the challenges of outgrowing local development environments due to increasing project complexity and resource constraints. The post outlines early experiments with local Kubernetes and Google Compute Engine (GCE) VMs, highlighting the eventual shift towards cloud-native solutions. It emphasizes the need for elasticity, avoiding VM management, and providing automation for bootstrapping projects, leading to the exploration of Kubernetes pods and the eventual focus on cloud-native development environments like Spin.
Advertisement