Streaming and longer context lengths for LLMs on Workers AI

This post introduces streaming response capabilities for all LLMs on Workers AI, enabling token-by-token delivery of responses using Server-Sent Events (SSE). It also announces increased context and sequence lengths for the Llama-2 model (from 768/256 to 2048/1800 for int8, and 3072/2500 for fp16) and the addition of a 16-bit full-precision Llama-2 variant. Examples are provided for using SSE with the REST API and a Worker script, along with client-side JavaScript for consuming the event stream.