Personalized Search Relevance

2 posts · Sep 2016 – Feb 2017

What this trail covers

This thread tracks the evolution of Slack's search capabilities, focusing on improving the relevance and user experience of search results. Initial posts detailed the challenges of search within Slack's unique environment and introduced the concepts of 'Recent' and 'Relevant' search. This post details the development of a personalized relevance sort and a 'Top Results' module, incorporating user interaction history ('work graph') and machine learning to significantly increase click-through rates. Subsequent posts detail the development of a channel recommendation system using collaborative filtering and activity transforms to help users curate their channel list.

All Slack trails

Feb 2017Search at SlackThis post details the development of a new personalized relevance sort for Slack search and the 'Top Results' module. It describes the unique challenges of search in Slack compared to web search, the advantages Slack has (user interaction history, no spam, controlled document structure), and the limitations of the existing 'Relevant' search. The post explains the two-stage approach to learning to rank: using Solr for initial retrieval and application layer re-ranking. It details the pairwise transform labeling strategy, overcoming click-position bias by oversampling lower-ranked results, and the key features used in the trained SVM model (message age, Lucene score, user affinity, channel priority, author affinity, message characteristics). The release to 50% of users showed a 9% increase in clicked searches and a 27% increase in clicks at position 1. The 'Top Results' module, showing the top 3 relevant results above recent results, further improved search sessions per user and reduced searches per session.
Sep 2016Personalized channel recommendations in SlackThis post details the development of a channel recommendation system. It describes the problem of curating channels as companies grow and introduces a solution using recommender systems. The approach involves building a Spark pipeline to ingest read/write log data from Hive, creating user-channel activity matrices, and using collaborative filtering (item-based) with cosine similarity and k-Nearest Neighbors regression to predict user activity in new channels. It also introduces activity transforms derived from linear least squares regression to refine predictions by accounting for scale differences between channels. The post also covers infrastructure for serving recommendations, interaction models, triggering logic, and the use of internal experiment systems for rollout and measurement.
Advertisement