Proxy vs Load Balancer vs API Gateway (System Design)

Forward vs reverse proxy, L4 vs L7 load balancers and their algorithms, what an API gateway adds (auth, rate limits, routing), and how the three stack in a…

Proxies, load balancers and API gateways: common questions

What is the difference between a load balancer and an API gateway?

A load balancer spreads requests across identical copies of one service and skips unhealthy ones. An API gateway is the single front door for many different services: it checks who is calling, applies rate limits and quotas, and routes each path to the right service, usually through that service’s load balancer.

Is a load balancer a reverse proxy?

Yes. A layer 7 load balancer is a reverse proxy whose main job is choosing a server. A layer 4 load balancer forwards TCP or UDP connections without reading the HTTP inside, so it is a lower-level kind of the same idea.

What is the difference between a forward proxy and a reverse proxy?

A forward proxy works for clients: they are configured to send traffic through it, and websites see the proxy instead of them. A reverse proxy works for servers: clients reach it without knowing, and it passes requests to servers they never see.

Related chapters

Keep learning

Overview

Normal request GET /api/orders/42 · valid token No token GET /api/orders/42 · no Authorization header 101st call this minute GET /api/orders/42 · same API key, limit is 100/min A server just died GET /api/orders/42 · orders-2 crashed 10 s ago Another service GET /api/users/7 · valid token API gateway no token → 401 Unauthorized Rejected at the edge. No service did any work. The token check lives in one place, written once, for every service behind the gateway. key k_81: 101/100 this minute → 429 Too Many Requests, Retry-After: 30 Throttled at the edge. The servers never saw it. The gateway counts per API key or per user, which a load balancer does not know about. Works for The clients The servers The API and its owners Who knows it is there The client (it is configured to use it) Only the server side Clients see one API address Main job Filter, log or hide outgoing traffic Stand in for servers: TLS, caching, compression Spread requests over healthy copies Apply API rules: who, how often, which service Looks at Destination host HTTP request L4: IP and port. L7: path, headers, cookies Path, method, token, API key, body Behind it The whole internet One app or a few Many copies of the same service Many different services Knows about users Employees, maybe Yes: identity, plan, quota Typical tools Squid, Zscaler, corporate proxies Nginx, HAProxy, Caddy, Envoy AWS ALB / NLB, HAProxy, Nginx, Envoy Kong, AWS API Gateway, Apigee, Envoy Gateway system design · core concepts · networking Proxy vs load balancer vs API gateway: three boxes in the middle All three sit between a caller and a server and pass requests along. They differ in whose side they are on and what they decide. A load balancer and an API gateway are both kinds of reverse proxy, which is why the names get mixed up.

1. One idea, four jobs

A proxy is anything that receives a request, does something with it, and sends it on as if it were the original caller. What it does with it gives it a name. Forward proxy Sits with the clients and goes out to the internet for them. The website sees the proxy, not you. client side Reverse proxy Sits with the servers and takes requests in for them. The client sees the proxy, not the servers. server side Load balancer A reverse proxy with one main decision: which of several identical servers gets this request. which copy API gateway A reverse proxy that knows your API: who is calling, how often, and which service owns this path. which rules A short way to keep them apart: a load balancer picks which copy, a gateway decides whether, and which service.

2. Forward proxy vs reverse proxy

Forward proxy: speaks for the client Your office laptop is set to send all web traffic through proxy.corp:3128. The proxy blocks some sites, logs the rest, and fetches pages for you. The websites see the company’s IP address. The client must be configured to use it. Reverse proxy: speaks for the server Your users type shop.com and reach Nginx. Nginx passes the request to app servers on a private network. Users never learn those servers exist. Nothing on the client changes. In system design interviews, “proxy” almost always means a reverse proxy. What it does for the servers behind it: Ends TLS Holds the certificate and decrypts HTTPS once, so app servers speak plain HTTP inside the private network. Hides the servers Only the proxy has a public IP. Servers can move, restart or change ports without clients knowing. Caches and compresses Serves repeated static files and responses from memory, and gzips or brotli-compresses the rest. Buffers slow clients Reads a slow phone upload fully before handing it over, so an app worker is not held for 20 seconds. Adds headers X-Forwarded-For carries the real client IP, because the server now only sees the proxy’s address. Absorbs some abuse Connection limits, request size limits and timeouts stop simple floods before the app.

3. Load balancer: which copy gets this request

Stateless app servers are easy to copy. Ten copies need one address in front and something that spreads requests among them and skips the broken ones. request arrives at one public address healthy list servers passing /health pick one by the algorithm and pass the reply back Layer 4 (transport) Sees only IP addresses and ports. Forwards whole TCP or UDP connections without reading them. Very fast, works for any protocol, can pass TLS through untouched. Example: AWS NLB. Layer 7 (application) Ends TLS and reads the HTTP request. Can route /images to one pool and /api to another, retry, set cookies, and balance per request, not per connection. Example: AWS ALB, Nginx, HAProxy. Round robin Next server in turn. Fine when requests cost about the same. A bigger machine gets 2 of every 3 requests. Also used to send 5% to a new version. Least connections Send to whoever is busiest least. Better when some requests take 10 ms and others 3 s. good default Hash of a key Power of two choices Pick two servers at random, send to the less loaded one. Nearly as good as least connections, cheap at scale. Sticky sessions A cookie pins a user to one server. Works, but that server’s death logs them out. Better: keep session state in Redis. What a load balancer does not do: it does not know who the user is, and it does nothing for the database behind the servers. See traffic jumps 10× for why “add a load balancer” is only half an answer.

4. API gateway: one front door for many services

With microservices, a phone app might need orders, users, payments and search, each a separate service with its own load balancer. Without a gateway, every app knows every address, and every service writes its own login check and rate limiter. A gateway gives clients one address and runs the shared rules once, in order: API gateway Orders service TLS ends · request ID added x-request-id: 7f3a… for tracing verify the JWT signature and expiry → 401 if bad rate limit user 81: 37 of 100 this minute → 429 if over business logic may this user see order 42? (ownership stays here) log + metrics latency, status, which client Check the token once at the edge, then pass the user ID inward in a header. Rate limits and quotas Per user, per API key or per plan: free gets 100/min, paid gets 5,000/min. Routing and versions REST or JSON outside, gRPC inside. Rename fields without touching services. One call from the phone, three calls behind it, one combined reply. This per-client version is called a backend for frontend (BFF). Not business rules Whether user 81 owns order 42 belongs in the orders service. A gateway full of business logic becomes a second monolith.

5. Lab: trace one request

Pick a request, then switch the gateway and the load balancer on and off. Watch where each request is stopped, routed or lost.

6. Side by side

Forward proxy Reverse proxy Load balancer API gateway

7. In a real system they stack, and one tool can be all three

phone, browser, partner caching reverse proxy near users L4 load balancer spreads across gateway copies API gateway auth, limits, routing L7 load balancer per service Service copies orders-1, -2, -3 The gateway is itself several copies behind a load balancer, so it is not a single point of failure. Nginx or Envoy One config can be a reverse proxy, a load balancer and a basic gateway. The label is the job, not the product. NLB is a layer 4 load balancer, ALB a layer 7 one, API Gateway a managed gateway. Many teams use ALB alone until they need per-user limits or API keys. An Ingress or Gateway API controller (often Nginx or Envoy) is the reverse proxy and L7 load balancer at the cluster edge. Service mesh Istio or Linkerd put a small proxy next to every service. It load-balances and retries calls between services, inside the system.

8. In the interview

You need Put in One app, several identical servers, survive one dying A load balancer (L7 for HTTP) HTTPS, compression, static files in front of one app A reverse proxy Many services, one public API, shared login and rate limits An API gateway, with a load balancer per service behind it Partners with API keys, quotas per plan, usage billing An API gateway Long-lived connections: WebSockets, game servers, raw TCP An L4 load balancer, or an L7 one with sticky connections Control what employees reach on the internet A forward proxy “A load balancer will handle the load” It spreads load across app servers. The database, cache and queues behind them still take every request. “The gateway is a single point of failure” Only if you run one. Run several copies behind an L4 load balancer or DNS, like any other stateless tier. Auth only at the gateway The gateway checks identity. Each service still checks permission and ownership, and inside traffic should not be trusted blindly. A gateway for one service Usually overkill. A load balancer plus auth middleware in the app is simpler until there are several services or outside API consumers. ★ Remember Load balancer: a reverse proxy that picks a healthy copy. L4 sees ports, L7 sees HTTP. API gateway: a reverse proxy that applies API rules (auth, limits, routing) across many services. They stack, and one tool can play all three. Name the job, then the product.

Go deeper

← Traffic jumps 10×: what breaks first · API security and rate limits · gRPC between services · JWT · consistent hashing

More courses

Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
Docker — What an image is, why layers cache, and how containers find each other on a network.
Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.