Client Side Load Balancer in Microservices | Spring Cloud LoadBalancer

The Restaurant Host Analogy

Imagine a busy restaurant with three cash registers. In a traditional setup, customers form one massive line at the entrance. A head host stands at the front, inspects each arriving customer, and directs them to register one, register two, or register three. That host is a centralized server side load balancer. If the host gets distracted, calls in sick, or becomes overwhelmed by a sudden crowd, the entire restaurant grinds to a halt.

Now imagine a modern self service system. Before customers enter the dining room, a digital board lists the status of all three registers. Each customer looks at the board on their own phone, sees that register two has the shortest queue, and walks directly to register two. The decision making is distributed to the client. There is no bottleneck at the door, and if one register closes, the customers simply select another register from their list.

This is the exact shift from traditional server side load balancing to client side load balancing. In a microservices architecture, instead of routing every inter service call through a hardware load balancer, the calling microservice queries the service registry, chooses an instance using a load balancing algorithm, and calls that instance directly.

This lecture covers server side versus client side load balancing, the retirement of Netflix Ribbon, Spring Cloud LoadBalancer implementation, custom routing algorithms, and @LoadBalanced mechanics.

---

Server Side vs Client Side Load Balancing

Understanding the architectural trade offs is a classic system design interview question.

Feature	Server Side Load Balancing (Nginx, F5, AWS ALB)	Client Side Load Balancing (Spring Cloud LoadBalancer)
Routing Location	Dedicated proxy server between client and service	Inside the calling microservice client process
Network Hops	Two network hops: Client -> Load Balancer -> Service Instance	Single network hop: Client -> Service Instance directly
Single Point of Failure	The load balancer itself can become a bottleneck	Distributed across all clients; failure of one client does not affect others
Service Discovery	Load balancer must be updated with backend IPs	Client queries Eureka directly for fresh instance lists
Latency Overhead	Adds proxy latency to every call	Zero proxy overhead; direct socket connection
Typical Domain	Public internet edge to internal services (API Gateway)	East West inter service communication inside private network

In production systems, both patterns coexist: an external server side load balancer directs public traffic to your API Gateway, while client side load balancing handles communication between internal microservices.

---

The Transition from Ribbon to Spring Cloud LoadBalancer

In early Spring Cloud releases, Netflix Ribbon was the default client side load balancer. However, Netflix placed Ribbon into maintenance mode in 2018. The Spring team developed Spring Cloud LoadBalancer as a modern, reactive, non blocking alternative that integrates natively with Project Reactor and WebClient while maintaining full backwards compatibility with RestTemplate.

To use Spring Cloud LoadBalancer in Spring Boot 3:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

---

Using @LoadBalanced with RestTemplate

The standard way to activate client side load balancing is adding @LoadBalanced to your RestTemplate bean definition:

package com.example.orderservice.config;

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

@Configuration
public class ClientConfig {

    @Bean
    @LoadBalanced
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

Now, instead of hardcoding hostnames and ports like http://localhost:8082/product/1, you use the registered application name from Eureka:

package com.example.orderservice.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

@Service
public class OrderService {

    @Autowired
    private RestTemplate restTemplate;

    public ProductDTO fetchProductDetails(String productId) {
        // Notice the URL uses the virtual service name PRODUCT-SERVICE
        String url = "http://PRODUCT-SERVICE/product/" + productId;
        return restTemplate.getForObject(url, ProductDTO.class);
    }
}

What Happens Under the Hood?

When you annotate RestTemplate with @LoadBalanced:

1. Spring Cloud registers a LoadBalancerInterceptor in the RestTemplate interceptor chain.
2. When restTemplate.getForObject("http://PRODUCT-SERVICE/product/101") is called, the interceptor intercepts the HTTP request.
3. The interceptor extracts the service name (PRODUCT-SERVICE) from the host portion of the URI.
4. It asks ReactiveLoadBalancer to choose an available ServiceInstance from Eureka's local cache.
5. The load balancer returns an instance, for example 192.168.1.45:8082.
6. The interceptor reconstructs the final URI: http://192.168.1.45:8082/product/101.
7. The actual HTTP call proceeds directly to that instance.

---

Load Balancing Algorithms

By default, Spring Cloud LoadBalancer uses the Round Robin algorithm. If PRODUCT-SERVICE has three instances (A, B, C), requests cycle sequentially: A -> B -> C -> A -> B -> C.

You can customize the algorithm per service using @LoadBalancerClient:

package com.example.orderservice.config;

import org.springframework.cloud.loadbalancer.annotation.LoadBalancerClient;
import org.springframework.context.annotation.Configuration;

@Configuration
@LoadBalancerClient(name = "PRODUCT-SERVICE", configuration = CustomLoadBalancerConfiguration.class)
public class ProductServiceLoadBalancerConfig {
}

Implementing Random Load Balancing

package com.example.orderservice.config;

import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.loadbalancer.core.RandomLoadBalancer;
import org.springframework.cloud.loadbalancer.core.ReactorLoadBalancer;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.cloud.loadbalancer.support.LoadBalancerClientFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.core.env.Environment;

public class CustomLoadBalancerConfiguration {

    @Bean
    public ReactorLoadBalancer<ServiceInstance> randomLoadBalancer(
            Environment environment,
            LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RandomLoadBalancer(
            loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier.class),
            name
        );
    }
}

---

Client Side Caching in Spring Cloud LoadBalancer

Querying Eureka over the network for every single HTTP request would introduce massive latency. To prevent this, Spring Cloud LoadBalancer uses client side caching:

1. The client maintains an in memory cache of available instances for each service name.
2. The cache refreshes periodically (default is every thirty five seconds).
3. Requests select an instance from this in memory list with zero network overhead.

You can tune cache properties in application.properties:

# Enable or disable client caching
spring.cloud.loadbalancer.cache.enabled=true

# Cache expiration time to live
spring.cloud.loadbalancer.cache.ttl=30s

# Cache capacity
spring.cloud.loadbalancer.cache.capacity=256

---

Interview Questions & Pitfalls

Q1: What is the primary difference between server side and client side load balancing?

Server side load balancing routes traffic through an intermediate proxy server (such as Nginx or AWS ALB), introducing an extra network hop. Client side load balancing runs inside the calling microservice process: the client queries service discovery directly and dispatches requests to the target instance with zero intermediate proxies.

Q2: How does @LoadBalanced work internally on a RestTemplate bean?

Spring Cloud registers a LoadBalancerInterceptor into the RestTemplate. When an HTTP request is made, the interceptor extracts the service name from the host part of the URL, delegates to ReactiveLoadBalancer to select an instance from Eureka, rewrites the URL with the real IP and port, and dispatches the request.

Q3: What happens if a chosen microservice instance goes down before the client cache refreshes?

The client may attempt to call the dead instance and encounter a connection timeout or connection refused error. This is why client side load balancing must be paired with fault tolerance mechanisms such as Resilience4j Retry and Circuit Breaker, allowing the client to automatically retry another instance when one fails.

Q4: Can client side load balancing be used without Eureka or another discovery service?

Yes. You can define a static list of service instances in your application.properties or application.yml file using Spring Cloud LoadBalancer static configuration, bypassing service discovery entirely.

Q5: Why did Spring Cloud replace Netflix Ribbon with Spring Cloud LoadBalancer?

Netflix placed Ribbon into maintenance mode. Spring Cloud LoadBalancer was created as a native, non blocking, reactive implementation that integrates cleanly with Spring WebFlux, Project Reactor, and modern Spring 3 architecture without legacy Netflix dependencies.