Visualizing GitHub’s global community
Dec 21, 2020Tal Safran

Summary
This post details the data pipeline for the GitHub homepage globe. It covers querying a data warehouse using Presto and Kafka for real-time data like pull requests, using a repository health score model to surface interesting data, and geocoding user locations with Mapbox's forward geocoding API. It also mentions the use of Airflow for scheduling these data processing tasks.