TechPrufer
Get started
LangChain
LangChain logoLangChain · Feature trail

Agentic PII Detection

10 videos · Sep 2026

What this trail covers

Managed Deep Agents provide a framework for building and deploying complex agents, enabling features like PII detection and automated GitHub PR creation from Slack messages. The system integrates with tools like LangSmith for tracing, evaluation, and deployment, and supports various execution environments and authorization mechanisms.

All LangChain trails
  1. Ship a GitHub PR From a Slack Message with Managed Deep AgentsThis post demonstrates the creation of 'Patch', an agent that transforms Slack messages into GitHub Pull Requests using Managed Deep Agents. It details the minimal code required for agent definition (`agent.py`), instructions (`instructions.md`), Slack integration, and sandbox setup, highlighting the ease of shipping production-ready agents with this platform.
  2. The Self-Driving Eval Trick No AI Benchmark BeatsThe post emphasizes a qualitative evaluation methodology for AI agents, focusing on manual, team-based review of real-world examples rather than relying solely on automated benchmarks. This approach aims to catch subtle regressions and improve agent performance by directly observing and discussing specific outputs, drawing a parallel to the evaluation practices in self-driving car vision model development. The core contribution is the advocacy for this 'self-driving eval trick' as a more robust evaluation strategy.
  3. How Jev detects PII vs LLMs, explained in under 20 secondsThis post introduces Jev, a TypeSafe System One model, which offers a novel approach to PII detection by providing calibrated probabilities rather than binary classifications. This allows for more nuanced decision-making within AI agents. The post highlights its integration into LangChain via the langchain-typesafe package, enabling it to function as a Runnable within agentic workflows for guardrail applications.
  4. The 3 Things Clay Measures on Every AI AgentThis post details the evaluation framework Clay uses for AI agents, focusing on three key metrics: quality, throughput, and cost. It explains how factual evaluations, deterministic matching, and LLM-as-a-judge scoring are combined to measure agent accuracy and efficiency. This contributes to the understanding of how to quantitatively assess and improve AI agent performance, building upon the existing capability of agentic PII detection by providing a method for its evaluation.
  5. Middleware for Managed Deep AgentsThe post introduces and demonstrates middleware for LangChain's Managed Deep Agents. It showcases the use of prebuilt PIIMiddleware to redact sensitive customer data (like emails) before it is processed by an LLM, ensuring privacy. Additionally, it provides a practical example of building a custom logging middleware from scratch using the `wrap_tool_call` decorator, illustrating how to extend agent lifecycles with custom logic for policy enforcement, fault tolerance, and rate limiting.
  7. Share your Managed Deep Agent with your team using SlackThis post details the practical steps for deploying a Managed Deep Agent to production using Slack as the user interface. It covers the process of initializing a Slack channel, authorizing the agent's connection, deploying the agent, and monitoring its activity through LangSmith traces. Additionally, it demonstrates how to customize the agent's presentation (name, icon, color) and configure message triggers for different interaction patterns. This builds upon the concept of agent middleware by showing a concrete deployment scenario.
  8. The One-Line Integration That Got Clay's Agents Into LangSmithThis post details the integration of Clay's agents, built on the Vercel AI SDK, into LangSmith for tracing. The primary technical contribution is the demonstration of a one-line code change to achieve this integration. It highlights the subsequent focus on cost observability, a fast UI for trace reading, quick onboarding for engineers, and exporting traces to Snowflake for deeper analysis, all facilitated by the LangSmith integration.
  9. Catch Agent Regressions Before You Ship: Evals for Managed Deep AgentsThis post details the evaluation framework for Managed Deep Agents, focusing on catching regressions and improving agent performance. It introduces the concept of evals as composed of an environment, job, and check, and demonstrates scaffolding with `mda evals init`. The post highlights the use of coding agents to generate eval tasks and LangSmith for monitoring and analysis, including datasets, traces, and rewards. It also presents simpler pytest-based evals and discusses production deployment via nightly CI.
  10. Score Every Production Trace with an LLM Judge, from Your Terminal (LangSmith CLI)This post extends the capabilities of LangSmith by introducing the ability to score production traces using an LLM judge directly from the terminal via the LangSmith CLI. It details the process of installing LangSmith skills, using a coding agent to create a 'frustration judge' evaluator, and then reviewing flagged traces in the LangSmith UI. The post also covers how to update the sampling rate to control costs associated with this continuous evaluation. This enables real-time, scalable evaluation of agent performance in production environments.
  11. How Clay Monitors Millions of Agent RunsThis post details how Clay monitors millions of agent runs by leveraging LangSmith's insights product and online evaluators. It addresses the challenge of manual review becoming infeasible at scale and explains how these tools provide visibility into agent behavior. The core contribution is the practical application of LangSmith for large-scale agent monitoring and evaluation, enabling engineers to understand and improve agent performance without manual inspection of every run.