Code Modularity and Boundary Enforcement

10 posts · Feb 2019 – Feb 2024

What this trail covers

Shopify has evolved its approach to managing large codebases by transitioning from a monolithic architecture to a modular monolith. This involves re-organizing code by real-world concepts (e.g., orders, shipping) instead of software concepts (models, views, controllers), and enforcing strict boundaries between components. Tools like Wedge are used to track and enforce these boundaries by analyzing cross-component dependencies and API access. This initiative aims to improve developer productivity, reduce fragility, and simplify onboarding by decreasing coupling and increasing modularity.

All Shopify trails

Feb 2024A Packwerk Retrospective - ShopifyThis post is a retrospective on Packwerk, detailing its origins as a dependency management tool, its later adoption and subsequent removal of 'privacy checks' which were misused as an API design tool, and its weaknesses and blind spots such as not seeing dynamic constants, relying on Zeitwerk autoload directories, and not seeing dependencies outside the app directory. It also describes an effort to isolate the 'Platform Essentials' package, which revealed a bug in Packwerk and highlighted the need for inversion of control to resolve violations, often making code harder to understand.
Sep 2022Focus On Behavior, Not State, for a More Maintainable Codebase - ShopifyThis post details a shift in code design philosophy from focusing on state (what an object is) to behavior (what an object does) to improve maintainability. It introduces the concept of wrapping domain objects (like 'selling plans') into new classes that expose behavior-oriented methods, rather than directly accessing internal state or database details. This approach aims to reduce coupling, prevent the leakage of internal logic, and make the codebase more resilient to future feature additions like installments or layaway. The post also discusses the need for a translation layer (e.g., an extra table) to bridge the gap between internal domain logic and reporting requirements for data teams.
Apr 2022Building a Business System Integration and Automation Platform at Shopify - ShopifyThis post details the development of a business system integration and automation platform at Shopify, built on Ruby on Rails. It addresses the gap left by modularization of business systems and the limitations of external IPaaS solutions like Mulesoft. The platform's design priorities include being minimal, transparent, flexible, secure, and reusable. Implementation details cover documentation, leveraging Ruby language features, using Rails Engines for modularity, eliminating data storage to promote transparency and security, employing 'Actions' for business logic, and embracing hashes for data flow transparency. The platform aims to enable reliable integrations built quickly by reusing existing infrastructure and code.
Jul 2021Shopify-Made Patterns in Our Rails Apps - ShopifyThis post introduces the 'dev' command-line tool for unifying developer workflows, dependency management, and environment setup across hundreds of projects. It details the 'podded architecture' for horizontal scaling by sharding databases and infrastructure, explaining how each pod runs Shopify independently and how data is split by shop. The post also discusses the application of Domain Driven Design principles to manage complexity within the monolith, emphasizing code organization by real-world concepts and the isolation of dependencies, while acknowledging the necessity of the shop being part of every domain due to the podded architecture. It also touches upon standardization of code writing through tools like the translation platform and maintenance task system, and the use of Monorail for schema enforcement on Kafka events.
Mar 2021Remove Circular Dependencies by Using Dependency Injection and the Repository Pattern in Ruby - ShopifyIntroduced the Repository pattern to remove direct circular dependencies between a pricing gem and its consumers (Shopify Core and Storefront Renderer). Defined a contract (PricingRepositoryInterface) in the gem that consumers must implement to provide data. Leveraged Dependency Injection (Constructor Injection) to pass the repository implementation to the gem. Utilized Sorbet interfaces and type signatures to enforce the contract and ensure type safety.
Advertisement
Nov 2020Adopting Sorbet at Scale - ShopifyThis post details the adoption of Sorbet for static typing in Shopify's Ruby monolith, including the development of open-source tooling like RuboCop Sorbet, Tapioca for RBI generation, and Spoom for programmatic Sorbet interaction. It also shares lessons learned regarding the importance of full coverage, the dangers of `typed: ignore`, and the benefits realized even at `typed: false`, such as preventing `NameErrors` and finding dead code. The post emphasizes an empirical approach to measuring the impact of typing on production errors, using the internal `dev` tool as a case study.
Sep 2020Enforcing Modularity in Rails Apps with Packwerk - ShopifyThis post introduces Packwerk, a new open-source tool for enforcing code boundaries in Rails applications. It details the problem of low cohesion and high coupling in large Rails monoliths, explores existing boundary enforcement solutions (private_constant, gems, tests, Modulation), and explains why a custom solution was necessary. Packwerk enforces dependency and privacy violations by analyzing constant references between packages.
Sep 2020Under Deconstruction: The State of Shopify’s Monolith - ShopifyThis post details the evolution of Shopify's efforts to modularize its Rails monolith, focusing on the development and impact of Packwerk. It highlights the challenges of managing a large codebase, the importance of understanding developer behavior (using Fogg's Behavior Model), fostering grassroots adoption through initiatives like the Architecture Guild, and the need for holistic architectural thinking. The post emphasizes the shift from strict rules to enabling developers by increasing their ability to make positive changes, particularly around restricting Active Record model access to owned components. It also discusses the challenges of dense dependency graphs and circular dependencies.
Mar 2020Refactoring Legacy Code with the Strangler Fig Pattern - ShopifyThis post details the application of the Strangler Fig Pattern to refactor the monolithic Shop model in Shopify's Rails codebase. It outlines a 7-step process: defining an interface for extracted components (using Shopify Capital's locked_settings as an example), changing calls from the old system to the new, creating new data sources (tables) for new data, implementing writers in the new model to write to both new and existing data sources, and finally, removing the old system. The post emphasizes incremental changes, reversibility, and zero downtime.
Feb 2019Deconstructing the Monolith - ShopifyThis post details Shopify's transition from a monolithic architecture to a modular monolith. It explains the advantages and disadvantages of monolithic systems, the challenges encountered at scale, and the rationale behind choosing a modular monolith over microservices. The core of the post describes the 'Componentization' project, which involved re-organizing the codebase by real-world concepts and developing tools like Wedge to enforce modularity and isolate dependencies by tracking and preventing violations of domain boundaries and data coupling across components.