Many-shot jailbreaking

This post introduces and details a new jailbreaking technique called 'many-shot jailbreaking' that leverages the expanded context window of LLMs. It explains the mechanism of using a large number of faux dialogues within a prompt to elicit harmful responses, and discusses the observed correlation between the number of 'shots' and attack success. The post also highlights the increased susceptibility of larger models due to their in-context learning abilities and describes prompt-based mitigation strategies that have shown significant success in reducing attack rates.