MosaicLeaks: Can your research agent keep a secret?

This post introduces MosaicLeaks, a new benchmark and task for evaluating privacy leakage in deep-research agents that use external tools (like web search) in conjunction with private local documents. It identifies a critical failure mode where individually benign web queries can reveal sensitive information through a 'mosaic effect'. The post proposes a new training methodology, Privacy-Aware Deep Research (PA-DR), which uses situational rewards to train agents to balance task success with minimizing information leakage. PA-DR significantly reduces answer/full-information leakage from 34.0% to 9.9% while improving strict chain success from 48.7% to 58.7%. The work highlights the tension between informative queries for task performance and privacy, and demonstrates that simply prompting agents not to leak is insufficient.