Request Routing Without Single Points

Routing components like DNS, load balancers, and reverse proxies must avoid becoming bottlenecks or failure points. Start with multiple independent instances of each routing tier, paired with health checks to remove unhealthy nodes automatically. Use round-robin, least-connections, or weighted algorithms to distribute traffic, ensuring traffic is never sent to broken endpoints.

Avoid centralized state by storing routing metadata in distributed stores like etcd or ZooKeeper, or using gossip protocols for peer discovery. Clients can cache routing info briefly but should refresh it periodically or on failures. Ensure clients retry with fallback endpoints if their preferred node is unreachable.

Design for failure at every layer: if a load balancer fails, another should take over quickly using floating IPs or DNS-based failover. Use consistent hashing for partitions where possible, so reassigning work affects only a fraction of keys. Test routing resilience under partial outages with chaos engineering exercises.