Traffic Jumps 10×: What Breaks First? (System Design)

Why the database, usually its connection limit, breaks before stateless app servers. PgBouncer, caching, read replicas and write queues, with a lab.

Scaling for a traffic spike: common questions

Traffic jumps 10×. What breaks first?

Usually the database, and often its connection limit. App servers are stateless and easy to add; the database is shared and much harder to scale. Postgres runs one process per connection, so 10× more app servers can exhaust max_connections quickly.

How do you add app servers without overloading the database?

A load balancer spreads traffic across app servers but does nothing for the database. Protect it with a connection pooler such as PgBouncer, a cache for hot reads, read replicas for read-heavy traffic, fixed slow queries, and queues for writes that do not need to happen immediately.

Related chapters

Keep learning

Overview

Connection pooler (PgBouncer) Cache hot reads reads go to copies, writes stay on the primary Fix the slow query add the missing index: 4 ms → 1.5 ms Queue deferrable writes half the writes are drained later at a steady rate primary CPU: queries queue faster than they finish replica CPU Lab: turn traffic up, then add fixes Postgres connections Primary CPU Each replica CPU , while the database CPU still has room : r.cpuFull ? ✗ The database CPU is saturated. Queries queue, every app server waits, and timeouts spread back to users. With these fixes, nothing in this model breaks up to 40×. system design · core concepts Traffic jumps 10×: what breaks first, and how to protect the database App servers are easy to add. The database they all share is not. Two common interview questions, one answer.

1. The two questions

“Traffic jumps 10×. What breaks first, and why?” Asked by @MansiCodez. Weak answer: “the servers”. Strong answer: the database, usually its connection limit, then its CPU. “How do you add app servers without overloading the database?” Asked by @dark_coderz. Weak answer: “a load balancer”. Strong answer: pooling, caching, read replicas, and queues for writes that can wait.

2. Why the database breaks first

10× requests Load balancer spreads them out App servers stateless: add 10× more One database shared by all of them Every layer before the database scales by adding copies. The database is the one thing they all share. App servers are stateless Any server can serve any request, so you add more behind the load balancer in minutes. The database is shared state One primary takes every write. Adding a second one means replication or sharding, which takes weeks, not minutes. Connections run out before CPU Each new app server opens its own pool. 10× servers means 10× connections, even if most of them sit idle. Postgres makes this sharp. It runs one operating-system process per connection, a few megabytes each, and the default max_connections is 100. Forty app servers with a pool of 20 want 800 connections. Postgres refuses the rest with “too many clients already”, and raising the limit trades that error for memory pressure and slower context switching. MySQL uses a thread per connection and hits a similar wall later.

3. Try it

Start at 10× with no fixes. Then add the fixes one at a time and watch which limit moves. The numbers are a small model to reason with, not a benchmark: 500 requests/s at 1×, two 4 ms queries per request, a 16-core primary, max_connections 200. Real limits come from measuring your own system (see numbers to know).

4. What protects the database

Each fix, what it relieves, and what it costs What it relieves What it costs Connection pooler (PgBouncer, RDS Proxy) Thousands of app connections share a few dozen real ones. Fixes the connection limit Transaction mode breaks session features (session variables, some prepared statements, advisory locks) Cache hot reads (Redis, CDN) Most repeated reads never reach the database Read replicas Spreads read-heavy traffic across copies Replication lag: a user may not see their own write. Does nothing for writes Fix slow queries A missing index can make every query hold a connection several times longer Usually cheap. Often the best first move Queue writes that can wait Emails, analytics, counters go to a queue and are written at a steady rate, so spikes do not hit the primary Those writes are delayed, and you now run a message queue Shard (last) Spreads writes across several primaries Cross-shard joins and transactions get hard. Only when writes alone outgrow one machine database load removed by a load balancer primary takes every write until you shard Postgres default max_connections

5. Traps

“Add a load balancer.” It spreads traffic across app servers and does nothing for the database behind them. More app servers can make the database worse, because each one brings its own connection pool. If a quiz expects “load balancer”, it is only part of the answer. “Read replicas” for a write spike. Replicas only help reads. Writes need fewer writes (queues, batching) or, eventually, sharding.

Recap: say this in the interview

★ Answer Next: proxy vs load balancer vs API gateway, and what each box in front of the servers actually does.

More courses

Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
Docker — What an image is, why layers cache, and how containers find each other on a network.
Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.