How GitLab uses prompt guardrails to help protect customers

This post details GitLab's implementation of prompt guardrails for its AI features, specifically GitLab Duo. It explains the rationale behind prompt guardrails, focusing on mitigating risks like sensitive data exposure and malicious prompts. The technical implementation involves structured prompts with context boundaries (e.g., `<selected_code>`, `<log>`), filtering tools like Gitleaks for secrets scanning, and instruction tuning to guide AI behavior. It also discusses different approaches to guardrails (system-level filters, instruction tuning, sidecar/gateway solutions) and GitLab's hybrid strategy. Key lessons learned include balancing restrictiveness, the need for continuous updates, and the ongoing nature of security.