Design an API gateway

01 · CONTEXT

One front door for many services

In a microservices architecture, a client may interact with more than one front-end service. Without an intermediary, the client has to keep track of several endpoints and handle failures resiliently, and it has to understand how the services are decomposed, which complicates maintenance and refactoring. Public endpoints also increase the attack surface and require hardening.

Um API gateway is a centralized entry point for interactions between clients and the application's services: it acts as a reverse proxy and routes client requests to the appropriate services. Without it, each public service has to handle concerns such as authentication, SSL, and per-client rate limiting on its own.

02 · ROUTING AND AGGREGATION

One endpoint, several services

The gateway uses application-layer routing and offers clients a single endpoint. Some requests are simply forwarded to the appropriate service; others the gateway handles by fanning the call out across several services.

One example from the pattern's documentation is a product details page: the product data is spread across several services, such as product information, price, and reviews. A mobile network is usually much slower and has much higher latency than a non-mobile network, and separate calls over a high-latency cellular network are inefficient and can cause dropped connections or incomplete responses.

1. The mobile app sends a single request to the gateway, instead of one for each service.

2. The gateway dispatches the request to the services needed for the operation.

3. The results come back to the gateway, which aggregates them before returning them to the client.

4. If one of the calls takes too long, it may be acceptable to time out and return a partial set of data.

A variation of the pattern gives each type of client its own gateway, the so-called backend for frontend. As for aggregation, the Azure documentation suggests considering an aggregation service behind the gateway, instead of building it into the gateway.

03 · EDGE TASKS

Identity and other cross-cutting concerns

Besides routing, the gateway can take on cross-cutting concerns on behalf of the backend services: management of client-facing certificates, authentication, TLS termination, monitoring, protocol translation, and rate limiting.

For authentication and authorization, the gateway can check whether the client is authorized to make the request; it can also authenticate the user and pass the services an access token with information about them.

1. The client knows only the gateway, which routes the request to the appropriate service.

2. Identity and authorization checks happen at the edge, backed by this design's identity provider.

3. Rate limiting is one of the concerns the gateway takes on behalf of the services.

4. The gateway can pass the services an access token with information about the user.

5. The gateway also brings some consistency to request logging and monitoring.

04 · RATE LIMITING

Token bucket and leaky bucket

Amazon API Gateway throttles requests with the token bucket algorithm, in which one token corresponds to one request. The configured rate is the pace, in requests per second, at which tokens enter the bucket, and the burst is the bucket's capacity.

When requests exceed the steady-state rate and the burst, the gateway starts throttling them, and the client may receive the 429 Too Many Requests error response. With a token bucket, a burst can allow the predefined limit to be exceeded.

The choice of key also matters. In Amazon API Gateway, the per-client limit uses the API key associated with the usage plan as the identifier. In NGINX there can be several limit_req directives at the same time, for example one per IP address and another per virtual server, and the nodelay parameter avoids delaying the excess requests.

05 · GATEWAY LIMITS

The front door can fail too

The gateway can introduce a single point of failure and also a bottleneck. The guidance is to avoid the single point of failure with multiple gateway instances and to size it for traffic peaks, not just for the average load.

1. In this design there is more than one gateway instance, which avoids the single point of failure.

2. A load balancer in front spreads the requests across the instances.

3. Each instance must be sized for traffic peaks, not just for the average.

4. The gateway adds a network hop to every request before it reaches the services.

This extra hop increases response time, although for most applications the cost of one more round trip is negligible. Compromising the gateway can expose all downstream services, and business logic should never be offloaded to it.

In practice, nginx and HAProxy are open-source reverse proxies, and the Azure documentation recommends preferring the gateway and ingress solutions the platform itself offers, when they meet the security and control requirements.