Strengthening our defenses against distillation attacks

This post details Anthropic's development of a two-stage classifier system to combat distillation attacks, drawing parallels to their cybersecurity and biology safeguards. It introduces 'preserved thinking' to limit reasoning extraction by preventing new API accounts from editing prior context and restricting reasoning access to the requesting organization. The post also outlines measures to make repeated distillation attempts difficult, including stricter usage limits, account bans, and charging for blocked requests. These defenses have reduced reasoning token extraction by 10-100x.