ELS: latency based load balancer, part 1

This post introduces the concept of load balancing and its importance in backend systems. It discusses the limitations of Round-Robin and Join the Shortest Queue (JSQ) strategies, particularly in scenarios with slow or failing machines. The post then introduces the Expected Latency Selector (ELS) as a probabilistic load balancer that uses metrics like success latency, failure latency, and outstanding requests to make intelligent routing decisions. Initial benchmarks comparing ELS, JSQ, and Round-Robin are presented, highlighting ELS's performance advantages under various load conditions and failure scenarios.