Mitigating a token-length side-channel attack in our AI products

What this post added

This post details the discovery and mitigation of a token-length side-channel attack affecting AI products that use streaming responses. Cloudflare collaborated with researchers to patch a vulnerability in Workers AI and AI Gateway by adding a 'p' property with a variable random length string to JSON objects in streaming responses, obscuring token lengths and preventing attackers from inferring plaintext information. This mitigation is automatically applied to all users of Workers AI and AI Gateway.