How we improved push processing on GitHub

This post details the migration of GitHub's push processing from a monolithic `RepositoryPushJob` in the Ruby on Rails monolith to a decoupled, event-driven architecture using Kafka. The monolithic job was sequential, had a large blast radius, and suffered from conflicting retry requirements. The new architecture introduces a Kafka topic for push events, which are then consumed by independent background jobs, each responsible for a specific set of tasks (e.g., AppsOnPushJob, CodespacesOnPushJob, PullRequestsOnPushJob). This change reduces the blast radius of problems, lowers latency for push-related tasks (e.g., pull request sync time), improves observability, and increases the reliability of push processing from 99.897% to an estimated 99.999%. Investments were made in reliable Kafka publishing, dedicated job workers, improved observability, and a consistent per-event feature flagging system.