TechPrufer
Get started
Slack
Slack logoSlack · Feature trail

Unified Grid Architecture

3 posts · Jul 2017 – Jul 2024

What this trail covers

This thread tracks the evolution of Slack's Enterprise Grid architecture towards a unified, org-wide view for users. Initial posts detail the challenges of a workspace-centric model with increasing multi-workspace usage. This post details the foundational re-architecture to an org-wide data model, including changes to API frameworks, data routing, and client-side storage to provide a single, unified view of user data across all workspaces within an Enterprise Grid. It covers strategies for migrating APIs and permission checks, including handling APIs that rely on workspace context, and the use of a prototype to validate the approach. The project became a foundational component of the IA4 redesign, with a two-phase approach to API migration and the development of tools to support the effort.

All Slack trails
  1. Unified Grid: How We Re-Architected Slack for Our Largest CustomersThis post details the technical challenges and architectural changes required to transition Slack's backend and clients from a workspace-centric to an org-wide data model for Enterprise Grid users. It describes the 'Unified Grid' project, including the creation of a new boot API for org-wide data, updates to the API framework to support the new model, and strategies for fixing APIs: allowing workspace-agnostic APIs to function, prompting users to select a workspace for workspace-specific APIs, and iterating over relevant workspaces as a fallback. It also discusses the decision to cap relevant workspaces at 50 for performance reasons and the two-pass approach for API migration (internal usage then full correctness).
  2. How Slack Built Shared ChannelsThis post details the technical challenges and architectural decisions involved in introducing Shared Channels, which fundamentally altered Slack's workspace-centric data partitioning model. It describes the shift from a model where all data belonged to a single workspace's shard to one where shared channel data resides on a single originating workspace's shard, with a new `shared_channels` table acting as a bridge. Key technical challenges addressed include decoupling channel privacy from table/ID encoding by introducing an `is_private` flag and consolidating logic around a single channel object model, and extending the edge cache service (Flannel) to support external user visibility across workspace boundaries.
  3. Maximum Warp: Building Migrations for Slack Enterprise GridThis post details the re-architecture of Slack's data model for Enterprise Grid to accommodate cross-team channels and shared data by introducing an organization-level database shard. It describes the process of migrating legacy data, including translating team IDs and user IDs, and moving DM/Group DM data to the organization shard. A generic data migration framework with idempotent handlers for validation, transformation, and insertion is introduced. The post also covers the implementation of rate limiting for migration speeds, the use of an asynchronous job queue system for offloading migration tasks, and the development of monitoring tools (job queue inspector, real-time progress bot, Grafana dashboards) to ensure safe and transparent migration.