BlogsIBMMonorepo Build System Migration

Monorepo Build System Migration

Monorepo Build System Migration

1
posts
2022

This post details the migration of a Python monorepo from Bazel to Pants, focusing on improving CI times, developer experience, and reducing build metadata. The migration was phased, starting with formatters and linters, then tests, then binary rules, and finally Docker images. Key technical achievements include dependency inference, changed file detection, and the reduction of Bazel metadata from 19,000 lines to 2,400 lines of Pants metadata, resulting in CI times dropping from 10-12 minutes to under 4 minutes.

2022

Discover how to successfully move a Python monorepo from Bazel to Pants in stages.

8/23/2022

This post describes a staged migration of a Python monorepo from Bazel to Pants. It details the background of the monorepo's issues (slow CI, complex metadata), the evaluation and selection of Pants based on dependency inference and changed file detection, and the phased migration strategy. Specific technical details include seeding Pants metadata from Bazel, using Pants macros, validating Bazel metadata against Pants' inferred dependencies to uncover missing declarations, leveraging Pants' `--changed` flag for faster test runs, migrating `py_binary` rules, and optimizing Docker images by shedding shared base images. The post quantifies the reduction in build metadata and CI execution time.