TechPrufer
Get started
GitHub
GitHub logoGitHub · Feature trail

Transit and Peering Connectivity

3 posts · Sep 2016 – Oct 2017

What this trail covers

This feature thread tracks the evolution of how GitHub manages its network connectivity, focusing on transit and peering relationships to ensure efficient and reliable request routing. It covers the technical details of BGP, cross-connects, private network interconnects (PNI), and internet exchange points (IXPs), as well as the strategic use of GeoDNS and traffic engineering to optimize performance and availability. The thread also addresses the handling of large-scale events like DDoS attacks. This post details the development of GLB (GitHub Load Balancer), a new load balancing solution designed to run on commodity hardware and scale horizontally. It introduces a stateless director tier using Rendezvous hashing to avoid connection disruption during node failures or maintenance, and a L4/L7 split design to separate traffic direction from connection proxying.

All GitHub trails
  1. Transit and Peering: How your requests reach GitHubThis post details the technical aspects of GitHub's network ingress and egress strategies, explaining the concepts of internet transit (billed by bandwidth, announced IP blocks via BGP) and peering (PNI and IXPs). It highlights the shift from relying solely on transit to incorporating peering, which now accounts for 60-70% of traffic, offering performance and availability benefits. The post also touches on GeoDNS for regional routing and the use of BGP route selection for traffic engineering, including incident response adjustments.
  2. Enabling DNS split authority with OctoDNSIntroduced OctoDNS, a tool for managing DNS records across multiple providers. This enables split authority for zones by synchronizing records between providers (e.g., Dynect and Route53) using configuration files stored in Git and a GitHub Flow-like workflow. The tool supports previewing changes ('noop' deploys) and executing them with a `--doit` flag, bringing Infrastructure as Code principles to DNS management.
  3. Introducing the GitHub Load BalancerDetails the design and implementation of GLB (GitHub Load Balancer), a new load balancing solution. Key contributions include: a stateless director tier using Rendezvous hashing for graceful node removal without connection disruption, a L4/L7 split design with 'director' and 'proxy' hosts, and the use of ECMP routing with a focus on avoiding connection termination during rehash events. The post also discusses the challenges of 'stretching an IP' and the benefits of the L4/L7 split for maintainability and DDoS mitigation.