Inside GitHub's new Git storage architecture

This post introduces 'Project Rubicon,' GitHub's new Git storage architecture designed for agentic scale. It details the principles of minimizing coordination and decoupling storage from compute. Key innovations include using Azure Blob Storage for immutable Git objects (packs, indexes) and a replicated log service (Ref-CAS) for mutable reference state. Virtual replicas act as stateless caches, rebuilding from Blob Storage and Ref-CAS as needed. Compaction nodes handle background merging and garbage collection in Blob Storage, independent of request serving. The architecture aims to support thousands of writes per second per repository by optimizing reference updates (the only part requiring coordination) and allowing independent scaling of storage and compute components.