TechPrufer
Get started
Shopify
Shopify logoShopify · Feature trail

Server-Sent Events for Real-time Streaming

1 post · Nov 2022

What this trail covers

Shopify has implemented Server-Sent Events (SSE) to simplify its real-time data streaming architecture, particularly for the BFCM Live Map. This move from a polling model to SSE has significantly reduced data latency, improved performance, and streamlined the overall system by removing intermediate presentation layers and message bus bottlenecks. The new architecture leverages a Flink-based data pipeline and a horizontally scalable Golang SSE server, load-balanced by Nginx, to deliver data within milliseconds of its availability.

All Shopify trails
  1. Using Server Sent Events to Simplify Real-time Streaming at Scale - ShopifyThis post details the implementation of a Golang-based Server-Sent Events (SSE) server to replace a polling communication model for the BFCM Live Map. It explains the benefits of SSE for unidirectional server-to-client communication, contrasting it with WebSockets and polling. The implementation involves subscribing to Kafka topics, handling client connections, and ensuring scalability through Nginx load balancing and horizontally scalable pods. A Java client using okhttp-eventsource was developed for load testing to determine pod capacity.