Java Engineering Notes

Appearance

API Gateway in Microservices — Part 1 | Spring Cloud API Gateway | Routing and Load Balancing ​

The International Airport Analogy ​

Imagine an international airport where incoming passengers arrive from across the globe. The airport does not allow passengers to wander directly onto luggage ramps, security rooms, or refueling docks. Instead, every arriving traveler passes through a central international terminal.

At this central gate, security officers check passports, customs officers inspect baggage, and digital departure boards direct travelers to their connecting gates. The central terminal insulates the airport's internal operations from the outside world, centralizes security, and controls traffic flow.

In microservices architecture, the API Gateway is that central international terminal. Without an API Gateway, client applications (web browsers, mobile apps, third party partners) would need to know the individual IP addresses and port numbers of twenty different microservices, manage authentication tokens repeatedly, and deal with cross origin resource sharing on every server. An API Gateway acts as the single reverse proxy entry point for all external traffic, routing requests dynamically to internal microservices while managing cross cutting concerns.

This lecture covers the fundamental need for an API Gateway, Spring Cloud Gateway architecture based on Netty, route definitions, predicates, built in filters, and dynamic routing with Eureka.

Why Every Microservice Architecture Needs an API Gateway ​

Exposing internal microservices directly to the outside world creates severe architectural problems:

[Without API Gateway]
  Web Client    -----> Order Service (:8081)
  Mobile App    -----> Product Service (:8082)
  Partner API   -----> Payment Service (:8083)
  * Client must manage 20 different URLs
  * Every service must implement auth, CORS, and SSL
  * Internal ports and topology are exposed publicly

With an API Gateway, the architecture simplifies into a clean boundary:

[With API Gateway]
  Web Client    ---+
  Mobile App    ----+---> [ API Gateway (:8080) ] ----> Order Service (:8081)
  Partner API   ---+              |               ----> Product Service (:8082)
                                  |               ----> Payment Service (:8083)
                         Cross-Cutting Concerns:
                         - Single public endpoint
                         - Centralized Authentication
                         - Rate Limiting & SSL Termination
                         - Path Routing & Load Balancing

Key Responsibilities of an API Gateway ​

  1. Single Entry Point: Clients connect to one domain (e.g. api.company.com).
  2. Reverse Proxy & Routing: Forwards requests to the appropriate internal microservice based on URL paths or headers.
  3. Security: Centralized SSL termination, authentication, and authorization.
  4. Resilience: Rate limiting and circuit breaking at the front door.
  5. Protocol Translation: Translates external HTTP/REST or WebSocket traffic to internal gRPC or messaging protocols.

Spring Cloud Gateway Architecture ​

Unlike older gateways such as Netflix Zuul 1 (which used a blocking one thread per connection model on standard Tomcat), Spring Cloud Gateway is built on top of Spring WebFlux, Project Reactor, and Netty.

Because it uses non blocking asynchronous I/O, a single gateway instance can handle tens of thousands of concurrent connections with very low memory overhead.

Core Building Blocks: Route, Predicate, Filter ​

Spring Cloud Gateway architecture consists of three fundamental concepts:

  1. Route: The basic building block of the gateway. It is defined by an ID, a destination URI, a collection of predicates, and a collection of filters.
  2. Predicate: A condition evaluated against the incoming HTTP request. If the predicate evaluates to true, the request matches the route. Examples: Path predicate, Header predicate, Method predicate.
  3. Filter: Intercepts requests and responses before and after routing to the downstream service. Filters can inspect, add, or modify headers, query parameters, or request payloads.
Incoming Request -> Gateway Handler Mapping -> Route Matched by Predicates
                          |
                   Pre-Filters Execute
                          |
                   Forwarded to Proxied Service
                          |
                   Post-Filters Execute
                          |
Outgoing Response <- Client Receives Response

Setting Up Spring Cloud Gateway ​

Create a Spring Boot project with the following dependencies:

xml
<dependencies>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-gateway</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
    </dependency>
</dependencies>

Important: Do not include spring-boot-starter-web in a Spring Cloud Gateway project. Gateway uses non blocking Netty through spring-boot-starter-webflux. Including the standard web starter introduces blocking Tomcat and causes application startup failure.

Configuring Routes via application.properties ​

You can declare routes declaratively in your configuration file:

properties
server.port=8080
spring.application.name=api-gateway

# Service Discovery registration
eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/

# Route 1: Product Service
spring.cloud.gateway.routes[0].id=product-service-route
spring.cloud.gateway.routes[0].uri=lb://PRODUCT-SERVICE
spring.cloud.gateway.routes[0].predicates[0]=Path=/product/**

# Route 2: Order Service
spring.cloud.gateway.routes[1].id=order-service-route
spring.cloud.gateway.routes[1].uri=lb://ORDER-SERVICE
spring.cloud.gateway.routes[1].predicates[0]=Path=/order/**

Notice the URI prefix: lb://PRODUCT-SERVICE. The lb:// scheme instructs Spring Cloud Gateway to use the client side load balancer and Eureka service registry to discover live instances of PRODUCT-SERVICE dynamically.

Common Built In Predicates ​

Spring Cloud Gateway includes numerous built in route predicates:

1. Path Predicate ​

Matches requests matching an ant style path pattern:

properties
spring.cloud.gateway.routes[0].predicates[0]=Path=/api/v1/orders/**

2. Method Predicate ​

Matches requests based on HTTP method:

properties
spring.cloud.gateway.routes[0].predicates[0]=Method=GET,POST

3. Header Predicate ​

Matches if a specific header is present and optionally matches a regular expression:

properties
# Matches if X-Request-Source header is present and contains "mobile"
spring.cloud.gateway.routes[0].predicates[0]=Header=X-Request-Source, mobile.*

4. Query Predicate ​

Matches if a query parameter exists in the URL:

properties
# Matches only if URL contains ?category=electronics
spring.cloud.gateway.routes[0].predicates[0]=Query=category, electronics

Common Built In Gateway Filters ​

Filters allow modifying the HTTP exchange before or after it reaches the downstream service:

1. StripPrefix ​

Removes path segments before forwarding. If a client calls /serviceA/users/1 and you specify StripPrefix=1, the downstream service receives /users/1:

properties
spring.cloud.gateway.routes[0].filters[0]=StripPrefix=1

2. AddRequestHeader and AddResponseHeader ​

Injects custom headers into the request or response:

properties
spring.cloud.gateway.routes[0].filters[0]=AddRequestHeader=X-Gateway-Timestamp, 1788616200
spring.cloud.gateway.routes[0].filters[1]=AddResponseHeader=X-Served-By, Spring-Cloud-Gateway

3. PrefixPath ​

Appends a prefix to the request path before forwarding:

properties
spring.cloud.gateway.routes[0].filters[0]=PrefixPath=/internal/api

Defining Routes Programmatically via Java DSL ​

In addition to properties, you can configure routes using the RouteLocatorBuilder fluent Java API:

java
package com.example.gateway.config;

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class GatewayRoutesConfig {

    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
            .route("order-service-route", r -> r
                .path("/order/**")
                .and()
                .method("GET", "POST")
                .filters(f -> f
                    .addRequestHeader("X-Caller", "Gateway-Proxy")
                    .stripPrefix(1)
                )
                .uri("lb://ORDER-SERVICE")
            )
            .route("product-service-route", r -> r
                .path("/product/**")
                .filters(f -> f.addResponseHeader("X-Cache-Status", "MISS"))
                .uri("lb://PRODUCT-SERVICE")
            )
            .build();
    }
}

Interview Questions & Pitfalls ​

Q1: What is the underlying server engine in Spring Cloud Gateway, and why does it not use Tomcat?

Spring Cloud Gateway uses Netty and Spring WebFlux. It avoids Tomcat because traditional servlet containers use a blocking one thread per request model. Netty provides a non blocking event loop architecture, allowing a single gateway instance to manage tens of thousands of concurrent connections with minimal threads and memory.

Q2: What happens if you include spring-boot-starter-web in a Spring Cloud Gateway project?

The application will fail to start. spring-boot-starter-web introduces standard Spring MVC and embedded Tomcat, which conflicts with spring-cloud-starter-gateway's reactive WebFlux and Netty foundation. Gateway requires reactive dependencies only.

Q3: What does the lb:// URI scheme mean in a route definition?

The lb:// prefix instructs Spring Cloud Gateway to use client side load balancing through Spring Cloud LoadBalancer. Instead of treating the hostname as a physical IP or DNS name, it treats it as a logical service name registered in Eureka, resolving live instances dynamically.

Q4: What is the difference between a Predicate and a Filter in Spring Cloud Gateway?

A Predicate determines whether a route matches an incoming request by evaluating request attributes (path, method, headers, query params). A Filter determines what to do with the request and response before and after forwarding it (modifying headers, logging, authentication, rate limiting).

Q5: How does the StripPrefix filter prevent routing path mismatches?

Clients frequently call the gateway using path namespaces like /order-service/orders/123. The downstream service only listens on /orders/123. Using StripPrefix=1 strips the first segment (/order-service) from the URL before forwarding, matching the downstream controller mapping.