Automating data removal

This post introduces the subsystem of SCARF responsible for automating the identification and safe removal of unused data types. It details how SCARF scans production data systems to identify unused tables or assets, measuring both static code usage (via Glean) and runtime access patterns. The system orchestrates removal by blocking reads/writes after a notification period and then deleting the data. It also addresses cross-system dependencies and code usage complexities to ensure thorough and safe data cleanup at scale, having removed petabytes of unused data and millions of data types.