Sidekick’s Improved Streaming Experience - Shopify

What this post added

This post details improvements to the streaming experience for Sidekick, an AI assistant. It addresses Markdown rendering jank by implementing a buffering Markdown parser using a Node.js Transform stream and a finite state machine to handle ambiguous Markdown syntax. It also prevents response delays by multiplexing asynchronous content resolution (tool calls) into a single stream using Server-Sent Events. This approach allows for immediate streaming of LLM responses while asynchronously populating resolved tool content, improving user experience and handling multi-intent requests. Special 'card:' protocol links are used to signal asynchronous content resolution, which are then parsed by the Markdown parser.