Table of contents
An API gateway's job is to be the single, consistent entry point for external and internal API traffic into a system made of multiple services — handling routing, authentication, rate limiting and other cross-cutting concerns in one place instead of every service reimplementing them independently. Done well, it removes real duplicated effort and inconsistency across services. Done poorly — as a dumping ground for business logic that belongs in the services themselves — it becomes a bottleneck and a single point of failure that's harder to reason about than the problem it was meant to solve.
This guide covers the patterns that make an API gateway actually earn its place in an enterprise integration architecture — routing and authentication design, rate limiting, the single-gateway vs. backend-for-frontend decision, and where a gateway's responsibilities should stop.
- Core value
- One place for cross-cutting concerns, not per-service duplication
- Should not contain
- Business logic — that belongs in the services
- Single gateway vs. BFF
- Depends on how different your client types' needs actually are
- Biggest risk
- The gateway becoming a single point of failure or a bottleneck
What an API gateway should actually do
The value of an API gateway comes from centralizing concerns that are genuinely cross-cutting — the same for every service, not specific to any one of them. That means: authentication and authorization (verifying who's calling and what they're allowed to do, once, rather than in every service), routing (directing a request to the right backend service based on the path, headers or other criteria), rate limiting and throttling (protecting backend services from being overwhelmed, whether by legitimate traffic spikes or abuse), and request/response transformation at the edge (adapting a legacy service's response format for a modern client, for example, without changing the service itself).
What it should not do is contain business logic — deciding whether an order is valid, calculating a price, or any decision specific to what the underlying service actually does. A gateway with business logic creeping into it becomes a second place every business rule change has to be made, defeating the purpose of having clearly-owned services in the first place, and it makes the gateway itself a much riskier thing to change.
Single gateway vs. backend-for-frontend (BFF)
This is the first real architectural fork. A single API gateway serves all client types (web, mobile, third-party integrations) through the same entry point with the same general-purpose API shape. A backend-for-frontend (BFF) pattern instead gives each client type its own tailored gateway layer, shaped specifically around what that client needs — a mobile BFF might aggregate several backend calls into one response to minimize round trips over a mobile network, while a web BFF might not need that optimization at all.
| Approach | Best fit | Tradeoff |
|---|---|---|
| Single API gateway | Client types have similar needs; API consistency matters more than per-client optimization | Can force compromises that don't fit any one client perfectly |
| Backend-for-frontend (BFF) | Client types have genuinely different needs (e.g. mobile bandwidth constraints vs. web) | More gateway layers to build and maintain — real added complexity |
| Hybrid (a shared gateway with thin BFF layers behind it) | Most real enterprise systems with more than one or two client types | Requires deliberate design to avoid duplicating logic across BFFs |
The gateway is a single point of failure by design — treat it that way
Every request through the gateway depends on the gateway being up, which means it needs a level of redundancy, monitoring and careful change management disproportionate to its apparent simplicity. A gateway outage takes down access to every service behind it at once, even if those services themselves are healthy — invest in its reliability accordingly, and avoid the trap of treating it as a lightweight component just because its individual responsibilities are conceptually simple.
API gateway vs. service mesh
These solve related but distinct problems, and conflating them leads to redundant infrastructure. An API gateway manages north-south traffic — requests coming into the system from outside (external clients, other systems). A service mesh manages east-west traffic — communication between services inside the system. Both handle some overlapping concerns (routing, security policies), but at different boundaries: a gateway is the front door, a mesh governs traffic once requests are already inside. Most enterprise systems need a gateway well before they need a service mesh — a mesh earns its complexity at a scale of internal service-to-service communication that most systems, including many with a dozen or more microservices (see our Spring Boot microservices guide for when that scale is genuinely reached), haven't hit yet.
Rate limiting and authentication design
Rate limiting at the gateway should be tiered deliberately, not applied as one blanket rule — different API consumers (an internal service, a trusted partner, a public-facing client) typically warrant different limits, and a single limit calibrated for the most permissive case leaves the system exposed to the most demanding one. Authentication should be verified once at the gateway and propagated to backend services as a trusted, signed context (a validated token or claims, not raw credentials re-verified at every service) — re-implementing authentication logic in every service both duplicates effort and creates more places for a security bug to hide.
When you might not need a dedicated gateway yet
For a small number of services behind a single client type, a dedicated API gateway can be more infrastructure than the problem warrants — a well-structured monolith or a couple of services with straightforward routing may not need one yet. Introduce a gateway when you have enough services, enough client diversity, or enough cross-cutting concerns (auth, rate limiting, transformation) duplicated across services that centralizing them clearly pays for itself, consistent with the same "match infrastructure to actual need" principle covered in our Kubernetes for startups guide for container orchestration specifically.
- Confirmed the gateway's responsibilities are limited to genuinely cross-cutting concerns, not business logic
- Decided single-gateway vs. BFF based on how different client types' real needs are, not by default
- Distinguished the gateway's north-south role from any internal service mesh's east-west role
- Designed rate limiting as tiered by consumer type, not a single blanket rule
- Planned authentication to be verified once at the gateway and propagated as trusted context
- Invested in the gateway's own reliability and monitoring proportionate to its single-point-of-failure role
Designing an enterprise integration architecture?
Talk to our engineering team about API gateway design, service boundaries and integration patterns for your system.
Frequently asked questions
What should an API gateway actually be responsible for?+
Cross-cutting concerns that are the same across services — authentication and authorization, routing, rate limiting, and request/response transformation at the edge. It should not contain business logic specific to what any one service does.
Should I use a single API gateway or a backend-for-frontend (BFF) pattern?+
Default to a single gateway unless your client types (web, mobile, partner integrations) have genuinely different needs that justify tailored BFF layers. A BFF is easier to add later than to consolidate once several have grown independently.
What's the difference between an API gateway and a service mesh?+
A gateway manages north-south traffic — requests coming into the system from outside. A service mesh manages east-west traffic — communication between services inside the system. Most systems need a gateway well before they need a mesh.
How should rate limiting work at an API gateway?+
Tiered by consumer type rather than one blanket rule — internal services, trusted partners and public clients typically warrant different limits, and a single limit calibrated for the most permissive case leaves the system exposed.
Does every microservices architecture need an API gateway?+
Not immediately. A small number of services behind a single client type may not need a dedicated gateway yet — introduce one once you have enough services, client diversity, or duplicated cross-cutting concerns that centralizing them clearly pays off.
Why is an API gateway considered a single point of failure?+
Every request into the system passes through it, so a gateway outage blocks access to every service behind it, even healthy ones. This means it needs redundancy, monitoring and careful change management disproportionate to how conceptually simple its responsibilities are.
Written by
CodeSurge AI Engineering Team
The CodeSurge AI team designs and builds AI systems, SaaS products and enterprise integrations for clients in India, the UAE and beyond — this section shares the architecture patterns, cost drivers and implementation tradeoffs we work through on real projects.