Network-Efficient APIs: HTTP Caching, ETags, Field Selection

Make APIs fast on mobile with compression, HTTP/2, Cache-Control, ETags and 304s, If-Match for lost updates, field selection, expand and batching.

Network-efficient API design: common questions

What is the difference between no-cache and no-store?

no-cache lets a cache store the response but it must check with the server before every use; no-store forbids storing it anywhere, which suits payment and other sensitive data.

How does If-Match prevent lost updates?

The client sends the ETag of the version it edited. The server applies the write only if the resource still has that ETag, otherwise it returns 412 Precondition Failed so the client reloads instead of overwriting someone else’s change.

Related chapters

Keep learning

Overview

Home Wi-Fi Good 4G Weak signal on a train Lab: how long does the event page take to load? Compression on (JSON shrinks to about a fifth) before the screen is ready moveMs ? Round trips dominate. Fewer, better-shaped requests help more than smaller ones. Bytes dominate. Compress, cache, and send fewer fields. No request at all: the copy is still fresh (max-age=60). 0 KB. Lab: John opens the same event page again and again No validators ETag + If-None-Match ETag + max-age=60 John opens event 123 Asha edits the event description on the server. Asha edits the event A minute passes: the cached copy is now stale and must be checked. A minute passes requests sent saved by the cache Every visit pays the full 48 KB, even when nothing changed. Every visit still costs one round trip, but an unchanged page costs only a few hundred bytes, and edits show up at once. Inside the minute there is no request at all, the fastest option. Edit the event inside that minute and John sees old text until it expires: that is the price. GET /venues/12/events/123/prices → 200, ETag: "v7" (both Asha and Ravi load the form) Ravi’s app reloads (ETag "v8"), shows Asha’s change, Ravi re-applies Silver with If-Match: "v8" → 200, ETag: "v9" Lab: two venue managers edit the same price list Plain PUT PUT with If-Match Both open the form Asha raises Gold to ₹3,000 Ravi raises Silver to ₹1,800 Ravi reloads and retries Start again Gold on the server Silver on the server current ETag Both changes kept. The 412 told Ravi his copy was out of date before anything was overwritten. Asha’s ₹3,000 is gone. Ravi’s PUT sent the whole list from his stale form, silently writing Gold back to ₹2,500. Nobody got an error.

Network-efficient APIs: caching, ETags and response shaping

A clean API can still feel slow on a phone in a train. The fix is rarely a faster server: it is fewer round trips, fewer bytes, and not downloading what the app already has. This chapter covers the HTTP tools for that (compression, caching, ETags, conditional writes) and the design tools (picking fields, expanding links, bundling requests).

Where the time goes on a phone

Every request pays a round trip: the time for a message to reach the server and the reply to come back. On mobile that is tens to hundreds of milliseconds before a single byte of data arrives. Then the body has to travel, which depends on its size and the bandwidth. Play with both: On a weak signal, four requests made one after another cost over a second and a half before any data moves. So the two levers of this chapter are: make fewer sequential requests, and send fewer bytes in each. The server’s own speed is often the smallest part.

Free wins at the protocol level

fewer bytes The client sends Accept-Encoding: gzip, br. The server compresses and says Content-Encoding: br. Repetitive JSON keys compress very well. Persistent connections fewer handshakes A new HTTPS connection costs extra round trips for TCP and TLS. Keeping it open (the default since HTTP/1.1) lets later requests skip that. HTTP/2 multiplexing parallel requests Many requests share one connection at the same time, so three independent calls cost about one round trip, not three.

HTTP caching: not asking at all

John’s app private cache (only John) shared cache (everyone in Mumbai) API gateway may cache too Event service A response can be stored at each hop. The Cache-Control header says who may store it and for how long. The fastest request is the one never sent. HTTP caching (RFC 9111) lets the server label each response with rules, and every cache on the path obeys them. The rules live in Cache-Control: Fresh for 300 seconds. Inside that time a cache answers without asking the server. Same idea, but only for shared caches like a CDN, and it overrides max-age there. public / private private: only the user’s own browser or app may store it. public: shared caches may too. May be stored, but must be checked with the server before every use. Not “don’t cache”. Never write it anywhere. For payment details and anything sensitive. After expiry, serve the old copy for up to 30 more seconds while fetching a new one in the background (RFC 5861). How long to cache is a product question, and the answer is set by the most volatile field in the response. That is a reason to split resources: if seat counts live inside the event page, the whole page can only be cached for seconds. Moved to their own endpoint, the event page can be cached for minutes. Browser / app GET /events/123 (name, venue, description) GET /events/123/availability (seats left) GET /bookings/9 (Priya’s booking) GET /payments/77 (card details) What the ticket site caches. Seat counts can be a couple of seconds old because booking re-checks them anyway. Shared caches don’t store responses to requests carrying an Authorization header unless the response explicitly allows it, which is a good default. Never mark per-user data public: a CDN would happily serve Priya’s booking to John.

Validators: ETag and 304 Not Modified

Strong ETag Changes whenever any byte changes. Required for conditional writes (If-Match). Weak ETag Means “same meaning”, maybe different bytes (say, a re-ordered JSON key). Fine for 304s, never matches If-Match. The older date-based validator. Only one-second precision, so two edits in the same second look identical.

Conditional writes: If-Match stops lost updates

The same ETag protects writes. Asha and Ravi both manage the venue and open the price form for event 123 at the same time. Each edits one tier and saves. Run it without and with a precondition: To stop old clients from skipping the check, the server can demand it: a write without If-Match gets 428 Precondition Required (RFC 6585). Compare with

Response shaping: let the client say what it needs

One fat response for everyone The event list returns every field: full description, all 40 ticket tiers, venue address, seating map URL. The phone list shows a name, a date and a lowest price. The client picks Summary rows by default, details on the detail endpoint, and parameters to ask for exactly the fields or linked data a screen uses. Field selection (a “wish list”). The client names the fields it wants. JSON:API calls these sparse fieldsets. Indie Night Expansion on demand. By default a booking links to its event ("event_id": "evt_123" plus a link). With expand the server embeds it, saving the follow-up call. A wish template goes one step further: the client sends a mock of the nested shape it wants, which is the idea GraphQL turned into a language. Costs: each field combination is a different URL, so caches fill with near-duplicates (fix the field order and offer a few named views such as ?view=summary). Every field still needs its own permission check, and unbounded expand chains need a depth limit. For big lists, also offer “only what changed”, such as ?updated_since=…, next to the you already have.

Request bundles, and when GraphQL is cheaper

A request bundle (batch endpoint) packs several independent calls into one HTTP request. The app’s home screen wants three events and Priya’s next booking: one round trip instead of four. Partial failure The outer request succeeds while item b fails. Every item needs its own status and error, and the client must handle each one. Caching is lost Bundles travel as POST, so the CDN and the ETags above no longer help. Cached GETs can beat a batch. Slowest item wins One reply means the whole bundle waits for its slowest part. Rate limits and auth must count and check each item. HTTP/2 already helps Over HTTP/2, three parallel GETs cost about one round trip and stay cacheable. Batch mainly for HTTP/1.1 clients or writes. Few round trips Small payload HTTP/CDN caching Simple server Plain REST + Cache-Control + ETag REST + fields / expand Batch endpoint GraphQL wins when many different screens need different nested shapes. Cacheable REST wins when many users read the same thing. GraphQL chapter the next chapter

Trade-offs: freshness, cache keys and Vary

Cache longer Fewer requests, cheaper servers, faster screens. But users may see a changed time or price up to max-age late. Cache shorter or revalidate Always correct. But every view costs a round trip, and a traffic spike lands on the origin. The cache key Caches look up a stored response by method and URL. ?city=mumbai&sort=date and ?sort=date&city=mumbai are two entries unless you normalise. Adds request headers to the key. Vary: Accept-Encoding keeps gzip and br copies apart. Vary: Authorization makes the cache per-user, which a CDN barely helps with. When Asha changes the event, a CDN copy stays until it expires unless you purge it. Short s-maxage plus a purge on write is a common pair. One operational trap: a cache can hide a broken release. If a gateway keeps serving yesterday’s good response, the new version can fail for minutes before anyone notices. Smoke-test releases with Cache-Control: no-cache on the request, and watch origin errors, not just edge errors (proxies, load balancers and gateways (opens in a new tab)).

In the interview

Next: work that doesn’t fit in one request and reply, such as long-running jobs, webhooks and bulk operations. Spotify returns Cache-Control and ETags, and playlists carry a snapshot_id so clients skip unchanged data. Caching and snapshot_id Markets and track relinking change the payload per country. Track relinking

More courses

Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
Docker — What an image is, why layers cache, and how containers find each other on a network.
Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.