Service Discovery with Eureka

Every client so far has called http://localhost:8082. That is fine for one instance on one laptop. In production, instances are created and removed dynamically: traffic grows and you start three more product-service instances; it falls and you stop them; a machine dies and its instance goes with it. Their addresses cannot be written into order-service.

The problem with a hardcoded URL

Picture three product-service instances, each with its own IP and port, and order-service pointing at one of them:

• Single point of failure: if instance #1 goes down, order-service cannot reach #2 or #3.
• No load balancing: #1 takes all the traffic while #2 and #3 sit idle.
• Tight coupling: moving product-service to another address means changing and redeploying order-service first.
• Hard to test: dev, QA and production use different addresses, so the configuration keeps changing, and mistakes creep in.

The fix is service discovery: services look each other up by name at runtime. This chapter uses Eureka, Netflix's discovery server, integrated into Spring Cloud. Consul is a popular alternative that works on the same ideas.

Server and clients

• Eureka server acts as a phone book. It holds the details of every instance of every registered service: service name, instance ID, IP, port, health status.
• Eureka clients are your services. A client does two things:
  1. registers itself with the server, and
  2. discovers the instances of other services through the server.

The caller chooses the instance. That choice is client-side load balancing, the subject of chapter 5.

Step 1: the Eureka server

Create a new project on start.spring.io and add the server dependency (version from the Spring Cloud BOM, as in chapter 3):

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

Enable the server:

@SpringBootApplication
@EnableEurekaServer        // creates the server's beans: registry, REST endpoints, dashboard
public class EurekaServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(EurekaServerApplication.class, args);
    }
}

Configure it:

spring.application.name=eureka-server
server.port=8761

# A server is not a client of itself: do not register, do not fetch the registry
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false

Both client properties default to true, because the Eureka server library contains the client too. For a single server, turn them off. Start it and open http://localhost:8761: the dashboard shows No instances available.

Step 2: register the services

In product-service, add the client dependency:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

And point it at the server:

server.port=8082
spring.application.name=product-service

# where the Eureka server is (/eureka is its API path)
eureka.client.service-url.defaultZone=http://localhost:8761/eureka

# both are true by default; shown here for clarity
eureka.client.register-with-eureka=true
eureka.client.fetch-registry=true

• spring.application.name is the service ID: the name other services will use to find this one. Eureka shows it in capitals, PRODUCT-SERVICE.
• defaultZone is the server's URL. Without it the client cannot find the server.
• A service that only calls others could set register-with-eureka=false; one that is only called could set fetch-registry=false.

Start product-service. The dashboard now lists PRODUCT-SERVICE with its instance ID (host:product-service:8082) and status UP. Do the same for order-service (spring.application.name=order-service, port 8081) and it appears too.

Step 3: call a service by name

With DiscoveryClient and RestTemplate

Spring Cloud's DiscoveryClient returns the registered instances of a service:

@Autowired
DiscoveryClient discoveryClient;

public void callProductAPI(String id) {
    List<ServiceInstance> instances = discoveryClient.getInstances("product-service");

    URI uri = instances.get(0).getUri();       // choosing an instance is your job
    String response = new RestTemplate()
            .getForObject(uri + "/products/" + id, String.class);

    System.out.println("Response from Product api call is: " + response);
}

No address is hardcoded: host and port come from the registry, only the relative path is in the code. But choosing the instance is left to you. get(0) sends everything to one instance; real code needs a load-balancing rule such as round robin.

With Feign

Feign does the lookup and the load balancing for you. Remove the url and keep the service ID as the name:

// before: fixed address
@FeignClient(name = "product-service", url = "${product-service.url}")

// after: looked up in Eureka by name
@FeignClient(name = "product-service")
public interface ProductClient {

    @GetMapping("/products/{id}")
    String getProductById(@PathVariable("id") String id);
}

Feign now asks the Spring Cloud LoadBalancer for an instance of product-service, which gets the list from Eureka and picks one. The load balancer comes from spring-cloud-starter-loadbalancer. The Eureka client starter already depends on it, so with Eureka you do not need to add it; you would only add it explicitly with another discovery mechanism.

With two product-service instances on ports 8082 and 8083, four calls alternate between them:

Product 1 from port 8083
Product 2 from port 8082
Product 3 from port 8083
Product 4 from port 8082

That is the whole setup. The rest of the chapter answers the questions that come next:

1. How does the server know whether a client is up or down?
2. Where is the registry stored?
3. Is the Eureka server a single point of failure?
4. Does every call now need two hops, one to Eureka and one to the service?
5. Can a client call an instance that is already dead?

1. How the server knows a client is up or down

Deregistration

When a client shuts down gracefully, it tells the server. Stopping product-service on 8083 logs, on the client:

Unregistering application PRODUCT-SERVICE with eureka with status DOWN
DiscoveryClient_PRODUCT-SERVICE/host:product-service:8083 - deregister  status: 200

and on the server:

Cancelled instance PRODUCT-SERVICE/host:product-service:8083 (replication=false)

Heartbeats

If a client dies without deregistering (a crash, a network failure), the server learns it from heartbeats. Each client renews its lease periodically. If the server hears nothing for the lease duration, it evicts the instance.

Client side:

# send a heartbeat every N seconds (default 30)
eureka.instance.lease-renewal-interval-in-seconds=30

# tell the server: if you hear nothing for N seconds, remove me (default 90)
eureka.instance.lease-expiration-duration-in-seconds=90

Server side:

# self-preservation (default true) stops evictions when many heartbeats go missing at once
eureka.server.enable-self-preservation=false

# how often the eviction task looks for expired leases (default 60000 ms)
eureka.server.eviction-interval-timer-in-ms=6000

Self-preservation exists because a sudden mass of missing heartbeats usually means a network problem between clients and server, not that all clients died. Rather than empty the registry, the server keeps the entries. In development it gets in the way of seeing evictions, so it is switched off here.

To watch an eviction, break the rule that the expiration must be longer than the renewal interval: renew every 60 s, expire after 5 s. The client is running and healthy, but the server stops hearing from it in time:

Evicting 1 items (expired=1, evictionLimit=1)
DS: Registry: expired lease for PRODUCT-SERVICE/host:product-service:8084
Cancelled instance PRODUCT-SERVICE/host:product-service:8084 (replication=false)

The instance vanished from the registry about 6 seconds after it started, while still running. In real settings, keep the expiration well above the renewal interval (the defaults, 90 and 30, allow three missed heartbeats).

2. Where the registry is stored

In memory only: there is no database. The registry is a map from application name to that application's instances, each held in a Lease:

ConcurrentHashMap<String, Map<String, Lease<InstanceInfo>>> registry;
//                appName      instanceId

Each InstanceInfo holds the instance ID, app name, IP address, host name, port, status (UP, DOWN...), last renewal time, lease duration and more. Logs print an entry as APPNAME/instanceId, for example PRODUCT-SERVICE/host:product-service:8082.

3. Is the Eureka server a single point of failure?

With one server, yes. If it goes down, its in-memory registry goes with it and no client can discover anything. In production, run a cluster, typically three servers on different machines:

Each server is also a client of the others: it registers with them, fetches their registry and replicates changes. So each one turns its client settings back on and lists its peers.

# eureka-1
spring.application.name=eureka-server
server.port=8761
eureka.instance.hostname=eureka-1
eureka.client.register-with-eureka=true
eureka.client.fetch-registry=true
eureka.client.service-url.defaultZone=http://eureka-2:8762/eureka/,http://eureka-3:8763/eureka/

eureka-2 and eureka-3 follow the same pattern, each listing the other two. Clients list all three:

eureka.client.service-url.defaultZone=http://eureka-1:8761/eureka/,http://eureka-2:8762/eureka/,http://eureka-3:8763/eureka/

If one server goes down, clients use another. Replication between servers is eventually consistent: an instance registered on eureka-1 reaches the others after a short delay, so for a moment two servers can give different answers.

4. Two hops per call?

No. A client does not ask the server before every call. With fetch-registry=true it downloads the registry at startup, keeps a local copy, and uses that copy to find instances. It refreshes the copy periodically:

# refresh the local copy every N seconds (default 30)
eureka.client.registry-fetch-interval-seconds=30

The cost is that the copy can be behind. Start order-service at the same moment as product-service: its first fetch finds no product-service instance, and calls fail with

FeignException$ServiceUnavailable: [503] ... Load balancer does not contain an instance for the service product-service

until the next refresh, about 30 seconds later.

5. Can a client call a dead instance?

Yes. Stop product-service on 8083 gracefully. The server removes it at once, but order-service still has it in its local copy, so calls keep alternating between both instances and every other call fails until the copy refreshes:

Product 5 from port 8082
500 Internal Server Error        <- sent to 8083, which is gone
Product 7 from port 8082
500 Internal Server Error

Stale data is a trade-off, not a bug:

• A short refresh interval keeps the copy fresh but floods the server with fetches.
• A long interval spares the server but leaves dead instances in the copy longer.

Several layers cache along the way. The server keeps a read-only response cache (refreshed every 30 s by default), the client refreshes its copy every 30 s, and the load balancer caches its instance list too. So after a crash, a minute or more can pass before every caller stops trying the dead instance. Retries (chapter 8) and circuit breakers (chapter 9) cover that window.

Next: Feign spread the calls across both instances without being told how. Chapter 5 opens up that client-side load balancer: how it gets the instances, how it chooses one, and how to change the algorithm.

Check yourself

1. Give four problems with hardcoding another service's URL.
2. What two things does a Eureka client do?
3. Why does a single Eureka server set register-with-eureka and fetch-registry to false?
4. What is spring.application.name used for in discovery?
5. With DiscoveryClient and RestTemplate, what remains your job?
6. How does Feign call a service by name, and which library chooses the instance?
7. Name the two ways the server learns that an instance is gone.
8. What do lease-renewal-interval-in-seconds and lease-expiration-duration-in-seconds control, and what are their defaults?
9. What is self-preservation and why does it exist?
10. How is the registry stored?
11. How do you avoid a single point of failure, and what consistency do you get?
12. Why is Eureka not called on every request, and what is the cost?
13. A product-service instance was stopped. Why can order-service still send requests to it?

Answers

1. Single point of failure; no load balancing; tight coupling (moving the service means changing the caller); configuration changes per environment, which is error-prone.
2. It registers itself with the server and discovers other services' instances through it.
3. The server library includes the client; a lone server has nothing to register with or fetch from.
4. It is the service ID: the name under which instances register and by which others look them up.
5. Choosing an instance from the list, i.e. load balancing.
6. @FeignClient(name = "product-service") without a url; Spring Cloud LoadBalancer gets the instances from Eureka and picks one.
7. A deregistration request on graceful shutdown, and missing heartbeats (lease expiry) otherwise.
8. How often the client sends heartbeats (default 30 s) and how long the server waits without one before evicting (default 90 s).
9. When many heartbeats go missing at once, the server assumes a network problem and stops evicting, so the registry is not wiped by a network glitch.
10. In memory only: a map from app name to a map of instance ID to Lease<InstanceInfo>. No database.
11. Run a cluster of servers (typically three), each registering with and replicating to the others, and list them all in the clients. Replication is eventually consistent.
12. Clients fetch the registry at startup, cache it locally and refresh it periodically (default 30 s). The cost is that the copy can be stale.
13. Its local copy of the registry (and the load balancer's cache) still lists the instance until the next refresh.