Code Modularity and Boundary Enforcement

2/7/2024

This post is a retrospective on Shopify's Packwerk tool, detailing its origins as a dependency management and privacy enforcement tool. It discusses the removal of privacy checks due to implementation issues and misuse as an API design tool. The post highlights Packwerk's weaknesses, such as its inability to infer runtime constants and its reliance on Zeitwerk, and the challenges developers face in defining package dependencies and resolving violations. It also recounts an experiment to isolate a 'Platform Essentials' package, demonstrating the trade-offs involved in achieving zero violations and dependencies, and uncovering a bug in Packwerk.

9/1/2022

This post introduces a strategy of focusing on behavior over state to improve code maintainability. It details how to wrap domain objects (like 'selling plans') to expose behavior-oriented methods rather than direct state access. This reduces conditional logic and leaks of internal implementation details to other parts of the codebase. The post also discusses how to translate these behavioral abstractions into a format suitable for data teams by creating a translation layer (a new table) to insulate reporting from internal schema changes.

4/22/2022

This post details the development of a business system integration and automation platform at Shopify, built on Ruby on Rails. It addresses the gap left by modularization of business systems and the limitations of external IPaaS solutions like Mulesoft. The platform's design priorities include being minimal, transparent, flexible, secure, and reusable. Implementation details cover documentation, leveraging Ruby language features, using Rails Engines for modularity, eliminating data storage to promote transparency and security, employing 'Actions' for business logic, and embracing hashes for data flow transparency. The platform aims to enable reliable integrations built quickly by reusing existing infrastructure and code.

7/13/2021

This post introduces the 'dev' command-line tool for unifying developer workflows, dependency management, and environment setup across hundreds of projects. It details the 'podded architecture' for horizontal scaling by sharding databases and infrastructure, explaining how each pod runs Shopify independently and how data is split by shop. The post also discusses the application of Domain Driven Design principles to manage complexity within the monolith, emphasizing code organization by real-world concepts and the isolation of dependencies, while acknowledging the necessity of the shop being part of every domain due to the podded architecture. It also touches upon standardization of code writing through tools like the translation platform and maintenance task system, and the use of Monorail for schema enforcement on Kafka events.

3/19/2021

Introduced the Repository pattern and Dependency Injection to resolve circular dependencies between a pricing gem and its consumers (Shopify Core monolith and Storefront Renderer). Defined a contract using a `PricingRepositoryInterface` and common Schema objects within the gem. Implemented the interface in each consumer to access their specific data sources. Leveraged Sorbet for type safety, enforcing the contract and improving code clarity. Outlined testing strategies including mocks for isolated gem testing and integration tests for consumer-gem interaction.

11/19/2020

This post details the adoption of Sorbet for static typing in Shopify's Ruby monolith, including the development of open-source tooling like RuboCop Sorbet, Tapioca for RBI generation, and Spoom for programmatic Sorbet interaction. It also shares lessons learned regarding the importance of full coverage, the dangers of `typed: ignore`, and the benefits realized even at `typed: false`, such as preventing `NameErrors` and finding dead code. The post emphasizes an empirical approach to measuring the impact of typing on production errors, using the internal `dev` tool as a case study.

9/23/2020

Introduced Packwerk, an open-source static analysis tool for Rails applications. Packwerk enforces modularity by defining 'packages' and enforcing dependency and privacy boundaries between them. It addresses the challenges of large monolithic codebases by promoting high cohesion and low coupling, and by providing a mechanism to prevent cross-component associations and unauthorized access to private constants. The tool aims to improve developer productivity and code maintainability.

9/16/2020

This post details the evolution of Shopify's efforts to modularize its Rails monolith over the past three years. It emphasizes the shift from focusing solely on technical solutions like component interfaces to understanding and influencing developer behavior using models like BJ Fogg's. The post highlights the importance of grassroots movements, exemplified by the Architecture Guild, and discusses the challenges of managing dense and circular dependencies within the monolith's dependency graph. It also touches upon the benefits realized, such as easier onboarding, faster test suites, and improved code design.

3/11/2020

This post introduces and details the application of the Strangler Fig Pattern to refactor a large, monolithic 'Shop' model in Shopify's Rails codebase. It outlines a 7-step process: defining an interface for the extracted functionality, changing calls from the old system to the new, creating new data sources if writing is involved, implementing writers in the new model to write to both new and existing data sources, and implicitly, the subsequent steps of strangling the old system and removing legacy code. The post uses the example of extracting 'store settings' and 'locked_settings' related to Shopify Capital to illustrate these steps, emphasizing incremental changes, reversibility, and zero downtime.

2/21/2019

This post details Shopify's transition from a monolithic architecture to a modular monolith. It explains the challenges of the monolith, the drawbacks of microservices, and the benefits of a modular monolith. The implementation involved a 'Componentization' project, re-organizing code by real-world concepts instead of software concepts, and isolating dependencies by defining clean interfaces and exclusive data ownership for each component. A tool called 'Wedge' was developed to track progress towards isolation and highlight violations of domain boundaries and data coupling across boundaries.