Zero trust with Kafka

This post details the implementation of zero-trust security for Grab's Kafka clusters. Key technical contributions include: 1. **Mutual TLS (mTLS)** for authentication and encryption, chosen over OAuth/SCRAM for offline peer identity verification. 2. **Hashicorp Vault and its PKI engine** for dynamic generation of short-lived client and server certificates, mitigating compromise risks. 3. **Policy-Based Access Control (PBAC)** with **Open Policy Agent (OPA)** for granular authorization, granting access based on the principle of least privilege. 4. Integration with **Strimzi** (Kafka on Kubernetes operator) for managing mTLS and OPA integration. 5. Detailed descriptions of server and client authentication flows, including CA tree structure. 6. Client setup using a Golang SDK with default mTLS configuration, retry mechanisms, and automatic certificate renewal. 7. Authorization process involving OPA policies written in Rego, managed via GitLab CI and pushed to S3, with Kafka brokers caching authorization outcomes. 8. Self-service authorization for topic owners via Git repository permissions and JSON data sources.