Load balancer algorithms: Round Robin, Least Connections, and IP Hash

When your application scales horizontally to twenty backend servers, you cannot hand clients twenty separate IP addresses. A load balancer acts as the single reverse proxy gateway to your cluster, intercepting incoming traffic and distributing requests among healthy backend instances.

Picking the wrong load balancing algorithm leads to hot spots, uneven CPU saturation, and cascading outages. Here is how the four standard algorithms work and when to use each.

                  ┌─────────┐   Sequential (A → B → C)   ┌──────────────┐
                  │ Client  │ ─────────────────────────> │ Backend A    │
                  └────┬────┘                            ├──────────────┤
                       │                                 │ Backend B    │
                       ▼                                 ├──────────────┤
              ┌─────────────────┐                        │ Backend C    │
              │  Load Balancer  │ ─────────────────────> └──────────────┘
              └─────────────────┘   Least Connections / IP Hash

1. Round Robin (Sequential Distribution)

Requests are forwarded to backend nodes in a strict circular rotation: Server 1 → Server 2 → Server 3 → Server 1.

How it works: The load balancer increments an integer pointer modulo the number of healthy nodes.
Overhead: Extremely low (O(1) memory and compute, stateless).
When to use: Stateless microservices deployed on identical server hardware handling uniform request durations (e.g., standard static API responses).
When it breaks: If requests have wildly differing execution times (a 2ms cache read vs a 4,000ms video transcode), servers handling heavier requests accumulate backlogs while idle nodes sit empty.

2. Weighted Round Robin (Capacity-Aware)

Assigns a static weight to each backend node corresponding to its compute capacity.

How it works: An 8-core, 32GB RAM server with weight 3 receives three requests for every single request sent to a 2-core, 8GB instance with weight 1.
When to use: Heterogeneous server fleets during rolling hardware migrations or cost-optimized cloud setups mixing instance families (e.g., c6g.xlarge vs c6g.4xlarge).

3. Least Connections (Dynamic State Tracking)

Rather than following a static sequence, the load balancer tracks the live count of active open connections on every backend node and routes new requests to the least busy server.

Server 1: [== 12 active sockets ==]
Server 2: [==== 28 active sockets ====]
Server 3: [= 3 active sockets =]  <── Route incoming request here

How it works: Inspects active TCP sockets, HTTP connection pools, or WebSocket sessions.
When to use: Variable-length transactions, long-polling requests, persistent WebSockets, or database connection poolers where connection lifespan varies from milliseconds to hours.

4. IP Hash (Deterministic Persistence)

Computes a cryptographic or modular hash of the client’s source IP address:

server_index = hash(client_ip) % N

How it works: Guarantees that any request originating from the same client IP always reaches the exact same backend server instance (sticky routing).
When to use: In-memory caching scenarios, stateful shopping cart sessions without shared Redis backends, or local file uploads chunked across multiple HTTP requests.
Limitation: If many clients share a corporate NAT gateway or proxy (e.g., thousands of users on an office subnet sharing one public IP), that single IP hashes to one server, creating a severe hot spot.

Algorithm Decision Matrix

Algorithm	State Overhead	Handles Unequal Nodes	Handles Unequal Work	Best Use Case
Round Robin	Zero	❌ No	❌ No	Stateless microservices, identical hardware
Weighted Round Robin	Low	✅ Yes	❌ No	Mixed hardware instances, canary testing
Least Connections	Medium	Partial	✅ Yes	WebSockets, file uploads, complex SQL queries
IP Hash	Low	❌ No	❌ No	Local memory sessions, stateful caching

In modern cloud environments, Layer 7 load balancers (AWS ALB, Envoy, Nginx, HAProxy) commonly combine Least Outstanding Requests (LOR) with active health checks to prevent routing to degraded instances.