TechPrufer
Get started
Dropbox
Dropbox logoDropbox · Feature trail

LLM Security Research and Mitigation

2 posts · Jul 2023 – Sep 2024

What this trail covers

Dropbox engineers conduct ongoing security research on large language models, identifying vulnerabilities such as repeated token attacks. They have developed methods to exploit and demonstrate these vulnerabilities, including extracting training data. This research informs the development of new attack vectors and the discovery of gaps in existing mitigation strategies, leading to the identification of multi-token repeat vulnerabilities. The findings are shared with model providers like OpenAI. Dropbox has also implemented Lakera Guard as an in-house service to secure LLM-powered applications, focusing on prompt injection and content moderation. This involves deploying Lakera Guard as a Docker container, integrating it into LangChain-based security chains, and measuring its impact on latency and security coverage. The company is planning to expand this integration across all LLM-using products.

All Dropbox trails
  1. How we use Lakera Guard to secure our LLMsThis post details Dropbox's integration of Lakera Guard to secure their LLMs, focusing on prompt injection and content moderation. It outlines the evaluation criteria (in-house deployment, low latency, long context support, confidence scores, continuous improvement), testing methodology using Garak, and the integration architecture using LangChain. Key learnings include achieving a 7x latency improvement for long prompts and collaborating with Lakera on detection efficacy. The post also mentions ongoing research into LLM vulnerabilities like repeated token attacks and false positives, and plans to expand Lakera Guard integration across all LLM products.
  2. Bye Bye Bye...: Evolution of repeated token attacks on ChatGPT modelsThis post details the discovery of multi-token repeated sequence attacks that bypass OpenAI's single-token filtering, enabling training data extraction from GPT-3.5 and GPT-4. It builds upon prior research into prompt injection and single-token repetition attacks, demonstrating how repeating multi-token sequences (e.g., 'jq_THREADS') can induce model divergence and reveal memorized training data. The research was shared with OpenAI, leading to patches.
  3. Don't you (forget NLP): Prompt injection with control characters in ChatGPTThis post details a novel prompt injection technique discovered by Dropbox security engineers, involving the use of control characters (like carriage-return and backspace) within LLM prompts to circumvent system instructions and cause unexpected model behavior. The research demonstrates how specific control characters, when encoded and sent in large quantities, can lead to models forgetting previous instructions or answering unrelated questions. This work contributes to the understanding of LLM vulnerabilities and informs the development of preventative measures.