TechPrufer
Sign inGet started

Blogs›Shopify›Continuous Integration of Framework Head

Continuous Integration of Framework Head

Continuous Integration of Framework Head

4
posts
2016–2019

This feature thread tracks the strategy and implementation of continuously integrating the latest upstream commits of critical frameworks, such as Rails, into production. Initial efforts focused on upgrading major framework versions, but evolved to a proactive approach of living on the 'edge' of the framework's master branch. This involves automated tools for weekly integration, extensive CI testing against a large test suite, and a phased deployment strategy using canaries to mitigate risks. The introduction of Shipit's merge queue and unsafe commit detection further enhances this by providing automated checks and management of the merge process, ensuring safer and faster deployments by preventing problematic commits from reaching production and automating the handling of reverts.

2019

Living on the Edge of Rails - Shopify

8/21/2019

This post details Shopify's strategy for 'Living on the Edge of Rails,' meaning continuously integrating the Rails master branch into production. It introduces the concept and benefits, such as faster bug fixes and feature adoption, and reduced upgrade effort. The post describes the automation involved, including the 'Solid Track' bot that performs weekly upgrades and opens PRs. It highlights the role of a comprehensive CI suite (130,000+ tests) in catching regressions and the use of canary deployments for safe rollout. The post also mentions contributions to Zeitwerk as a positive community impact of this approach and outlines criteria for other applications considering this strategy (benefit, monitoring, deployment strategy, test suite confidence).

2018

Introducing the Merge Queue - Shopify

6/8/2018

Introduced Shipit's merge queue to automate the process of merging pull requests into the main branch, reducing manual intervention and improving developer workflow. Implemented a browser extension to integrate the merge queue into the GitHub pull request interface, making it the default merging mechanism. Added configurable thresholds for branch divergence to discourage long-lived branches. Enhanced the merge queue to automatically handle pending CI statuses, merging or rejecting pull requests based on results. Developed features for marking commits as unsafe for deploy and automatic revert detection to improve the safety and automation of the deployment pipeline, especially in the presence of bad commits or reverts.

2017

Upgrading Shopify to Rails 5 - Shopify

6/20/2017

This post details the significant effort and technical challenges involved in upgrading Shopify's monolith from Rails 4.2 to Rails 5.0. Key technical contributions include: developing a 'dual' Gemfile and Gemfile.lock strategy to support both Rails versions simultaneously during the upgrade; setting up a parallel CI pipeline to test against both versions, which doubled CI infrastructure demand; implementing conditional logic in the codebase to handle version-specific behaviors; and a phased rollout strategy using Docker containers to mitigate production risks. The post also highlights lessons learned regarding cache serialization issues with Marshal and YAML, and the performance impact of deprecation warnings in production.

2016

Automatic Deployment at Shopify - Shopify

10/19/2016

This post details the automation of the final manual deploy trigger, removing the "Hit deploy button" step and enabling automatic batching and deployment of merges via the GitHub merge button. It introduces the Shipit deploy robot for automated batching and cadence-based deploys, and Spy, a ChatOps robot for visibility and notifications. The post also discusses the performance impact of containerized deployments and the human impact of removing manual steps, leading to the development of automated reminders via Spy.