Bye Bye Bye...: Evolution of repeated token attacks on ChatGPT models

This post details the discovery of multi-token repeated sequence attacks that bypass OpenAI's single-token filtering, enabling training data extraction from GPT-3.5 and GPT-4. It builds upon prior research into prompt injection and single-token repetition attacks, demonstrating how repeating multi-token sequences (e.g., 'jq_THREADS') can induce model divergence and reveal memorized training data. The research was shared with OpenAI, leading to patches.