TechPrufer
Get started
GitHub
GitHub logoGitHub · Feature trail

Physical Infrastructure Management

2 posts · Jan 2015 – Oct 2017

What this trail covers

This feature thread tracks the evolution of managing physical infrastructure in a data center environment. Initially, the focus was on the need for on-demand compute resources and the challenges of managing physical hardware. This post details the development of gPanel, a Ruby on Rails application for managing physical components (cabinets, PDUs, chassis, switches, hardware), and automating server provisioning. The system utilizes IPMI, iPXE, Facter, and an Ubuntu PXE image to automate the provisioning process. Subsequent posts would detail further refinements in network connectivity, data center design, and hardware lifecycle management.

All GitHub trails
  1. Evolution of GitHub's data centersThis post details the evolution of GitHub's physical infrastructure, introducing a hub-and-spoke design with four facilities: two Points of Presence (POPs) for networking and two data centers for storage and compute. It describes the homogenous cabinet types, structured cabling for simplified capacity expansion, and the role of gPanel in hardware management, including automated provisioning and validation. The post also highlights the use of cabinet integrators for rack and stack, cabling, and logistics, enabling hardware to be made available for provisioning on the same day cabinets are powered up.
  2. GitHub's Metal CloudThis post details the engineering effort behind gPanel, GitHub's physical infrastructure management application. It describes the pipeline for server provisioning using IPMI, iPXE, Facter, and an Ubuntu PXE image. Key components include hardware data collection and modeling in gPanel, state management (unknown, configuring, firmware_upgrade, breakin, memtesting, failed, ready, installing, installed), automated burn-in processes with custom scripts and MemTest86, and OS installation initiated via Hubot. The post also covers the decommissioning workflow.
  3. How GitHub uses GitHub to document GitHubThis post details the migration of GitHub's Help site from a custom Rails application to a static Jekyll site hosted on GitHub Pages. The previous setup involved a separate Rails app for site management and search, hosted externally, requiring significant operational overhead. The new setup leverages Jekyll's collections, data files for content references (conrefs), and Liquid templating for versioned documentation. Client-side search is implemented using lunr-js. Testing is performed using html-proofer and Capybara. The migration resulted in improved speed due to static file serving and CDN utilization, and reduced engineering effort by using existing GitHub Pages infrastructure.