How Zendesk Secures Kafka with Self-Hosted mTLS Authentication System

What this post added

Zendesk built a self-hosted mTLS authentication system for their Kafka clusters using HashiCorp Vault as a private CA. The system includes a PKI auth manager sidecar for Kubernetes clients and a TLS monitor for brokers to manage certificate generation, regeneration, and hot-reloading. The post details the architecture, solution components, and security considerations, including a strategy for root CA rotation due to limitations in certificate revocation.