How We Scaled Our Cache and Got a Good Night's Sleep

This post details the technical decision-making process for scaling Grab's cache system. It outlines the problem of a growing load on the single-threaded Redis cache, evaluates three potential solutions (master-slave, Twemproxy, custom sharding), and explains the chosen custom sharding approach. The implementation involved creating a `ShardedCache` interface in Go, compositing it with the existing `Cache` interface, and utilizing consistent hashing. The deployment strategy involved a two-day phased rollout with double-writing to the new system and cache population via a cron job to ensure reliability and minimize disruption. The outcome demonstrated improved P99 latency and better resource utilization.