How we rebuilt the search architecture for high availability in GitHub Enterprise Server

What this post added

This post details the migration of GitHub Enterprise Server's high availability search architecture from a clustered Elasticsearch setup to utilizing Elasticsearch's Cross Cluster Replication (CCR). It explains the challenges of the previous leader/follower pattern with clustered Elasticsearch, including shard management issues and potential system lockups due to primary shard movement. The new architecture employs independent single-node Elasticsearch clusters on primary and replica servers, with CCR handling index data replication. The post includes a pseudocode example of the bootstrap workflow for enabling CCR, highlighting the need for custom engineering for failover, index deletion, and upgrades, as Elasticsearch's CCR primarily handles document replication.