Saving another 100TB of RAM with math (and Rust)

This post details the optimization of consistent hashing in the Pingora Backend Router (PBR) service, specifically addressing excessive memory usage in the pingora-ketama library. The core problem identified was the memory footprint associated with representing servers with multiple hashes. The solution involved refining the consistent hashing algorithm by: 1. Increasing the number of hashes per server (from 1 to 160) to improve workload distribution and reduce the coefficient of variation. 2. Implementing server weights proportional to disk space (using ketama algorithm principles) to scale workload allocation based on resource availability. This optimization resulted in reclaiming over 100TB of RAM globally.