Grab

PII masking for privacy-grade machine learning

Jun 1, 2023Fabrice Harbulot
PII masking for privacy-grade machine learning
SummaryAI, from the source post

This post details the implementation of PII masking for machine learning data streaming pipelines at Grab. It describes a process involving PII tagging in protobuf schemas using a custom `pii.proto` library, a CI pipeline with a Python script to enforce tagging and a whitelist for exceptions, and a Flink application that consumes production data, dynamically masks PII based on schema tags (using HMAC for strings and seeded random generators for numbers/dates), and publishes sanitized data to the staging environment. The goal is to enable data engineers to tune ML models with production data while maintaining privacy by design.

More from the AI-Assisted Analytics trail