Personalized channel recommendations in Slack

This post details the development of a channel recommendation system. It describes the problem of curating channels as companies grow and introduces a solution using recommender systems. The approach involves building a Spark pipeline to ingest read/write log data from Hive, creating user-channel activity matrices, and using collaborative filtering (item-based) with cosine similarity and k-Nearest Neighbors regression to predict user activity in new channels. It also introduces activity transforms derived from linear least squares regression to refine predictions by accounting for scale differences between channels. The post also covers infrastructure for serving recommendations, interaction models, triggering logic, and the use of internal experiment systems for rollout and measurement.