Sidekick’s Improved Streaming Experience - Shopify

This post details the technical implementation of an improved streaming experience for Sidekick. It introduces a buffering Markdown parser that uses a finite state machine (FSM) implemented in a Node.js Transform stream to handle ambiguous Markdown sequences, preventing rendering jank. It also describes an asynchronous content resolution and multiplexing strategy using Server-Sent Events, where tool content (referred to as 'cards') is resolved asynchronously and multiplexed into the main LLM response stream. This allows for immediate rendering of the initial LLM response while placeholders are populated with resolved card content, improving perceived performance and user experience.