PII masking for privacy-grade machine learning
Jun 1, 2023Fabrice Harbulot

Summary
This post details the implementation of PII masking for machine learning data streaming pipelines at Grab. It describes a process involving PII tagging in protobuf schemas using a custom `pii.proto` library, a CI pipeline with a Python script to enforce tagging and a whitelist for exceptions, and a Flink application that consumes production data, dynamically masks PII based on schema tags (using HMAC for strings and seeded random generators for numbers/dates), and publishes sanitized data to the staging environment. The goal is to enable data engineers to tune ML models with production data while maintaining privacy by design.
More from the AI-Assisted Analytics trail
- Grab Bench: Evaluating AI on Grab-shaped production work
- How AI is transforming analytics at Grab
- Palana (Part 2): Architecting isolation, identity, and auditability for AI agents
- Palana (Part 1): Why Grab built a secure platform for autonomous AI Agents
- Agent platform (Part 1): How we help Grab build and run AI agents at scale