Policy Zones: How Meta enforces purpose limitation at scale in batch processing systems

What this post added

Introduced Policy Zones, a system for enforcing purpose limitations on data in large-scale batch processing systems. Policy Zones integrates with Meta's exabyte-scale data warehouse and processing systems, using runtime enforcement and SQL parsing to propagate and enforce privacy annotations across millions of daily data flows, performing trillions of user consent checks per hour. It leverages Information Flow Control (IFC) principles and provides tools for engineers to automatically track and protect data flows by enforcing flow restrictions at runtime. Key components include the Policy Zone Manager (PZM) for reclassification and Governable Data Annotations (GDAs) for precise, governed annotations on datasets.