Designing Resilient Systems Beyond Retries (Part 2): Bulkheading, Load Balancing, and Fallbacks

This post introduces and details three key resiliency patterns: bulkheading, load balancing, and fallbacks. Bulkheading is explained as a method of isolating failures at various levels (infrastructure, OS, process, connection/thread pools) to prevent cascading failures. The Hystrix library for Go is mentioned as supporting bulkheading via `MaxConcurrentRequests`. Load balancing strategies (proxy, client-side, lookaside, service mesh) are discussed, with Grab's internal client-side solution CSDP using etcd highlighted. Fallback strategies (fail silently, local computation, cached values, backup service) are presented, with Grab-Kit's support for cached fallbacks and a backup service for core functionality noted.