Async APIs: 202 Accepted, Webhooks and Bulk Operations

Design long-running jobs with 202 Accepted, HMAC-signed webhooks with retries and dedupe, event streams, AsyncAPI, and bulk endpoints with per-item results.

Asynchronous API design: common questions

How should a REST API handle a request that takes minutes?

Return 202 Accepted with a Location header pointing to a job resource and a Retry-After hint. The client polls the job, or gets a webhook, until it reaches succeeded, failed or cancelled, and then follows the link to the result.

Why must webhook receivers dedupe by event id?

Senders retry until they see a 2xx, so if a reply is lost the same event arrives again. That is at-least-once delivery: the receiver stores each event id and skips any it has already seen.

Related chapters

Keep learning

Overview

Still waiting for a worker. The app sleeps for Retry-After seconds instead of asking in a tight loop. A worker picked it up. Progress lets the app draw a real bar, and a longer Retry-After spreads the polls out. Done. The job links to the result instead of carrying 5,000 seats in its body, and the 4 bad rows wait at the errors link. Still 200 OK: reading the job worked. The failure is the job’s state, so it lives in the body with the same error shape as everywhere else. Lab: Asha imports 5,000 seats for a new venue It works The file is bad Asha cancels One job, one state at a time. The three end states are final: a job never leaves them. Asha presses Cancel Next poll Start again connection refused 503 Service Unavailable (partner is deploying) +1 min no reply within 10 s, counted as a failure +5 min partner handled it and answered 200, but the reply was lost on the way back +30 min · receiver: new evt_901, sends Priya a confirmation SMS · receiver: evt_901 already seen, answer 200 and skip · receiver: sends Priya a second SMS Lab: delivering booking.confirmed (evt_901) to a partner Partner briefly down Reply lost Partner down for days Receiver trusts every delivery Receiver dedupes by event id Nothing sent yet. Each attempt waits longer than the one before. Send the event Next attempt The duplicate arrived, but the receiver stored evt_901 the first time and skipped it. Backoff gave the partner six minutes to come back without flooding it. Lab: POST /venues/v_12/seats:batch with 5 seats Partial, 207 Multi-Status Partial, 200 + per-item array Request items Fix rows 2 and 3

Long jobs, events and bulk: beyond request and reply

Everything so far was one request, one quick reply. Some work takes minutes, some news happens when nobody is asking, and some clients need to send 500 things at once. Each case needs its own shape: a job you can check on, an event pushed to you, or one call that reports on many items.

When one request and one reply is not enough

seats in Asha’s import file for a new venue ~4 min to validate rows and create the seat map where many proxies, gateways and phones give up waiting If the API holds the connection while it works, the gateway (opens in a new tab) cuts it after a minute. Asha sees an error, but the server keeps going, so she does not know whether her seats exist. She retries and imports them twice. The same happens to a refund run: a concert is cancelled and 12,000 bookings must be refunded through the payment provider, which only accepts so many calls per second. That takes hours, not seconds. Hold the line One request waits for the whole job. Times out, ties up a server thread, and a retry repeats the work. Accept, then track

Async request-reply: 202 Accepted and a job resource

The import request returns 202 Accepted: “I have taken this, it is not done yet”. The Location header points to a new job resource and Retry-After hints how many seconds to wait before asking. Step through it: States are a closed list The job is not the result When it succeeds, the job links to the seats. Some APIs answer the final poll with 303 See Other pointing there instead. Cancel is a state change DELETE /jobs/job_77 or POST /jobs/job_77/cancel. Patterns for API Design calls this a state transition operation: only legal from some states. Make the start retry-safe The first POST still carries an , so a retried import returns the same job instead of starting a second one. Keep finished jobs for a while (a few days) so a client that was offline can still read the outcome, then expire them. Polling a job is cheap if the job resource sends an ETag, which the efficiency chapter

Webhooks: the API calls the partner

Partner subscribes POST /webhook-subscriptions Something happens Priya’s booking confirmed Ticket site POSTs the event to the partner’s URL Partner answers 2xx within a few seconds The partner hosts the endpoint, but you design its contract: one envelope for every event. Thin event Just the type, the id and a link. The partner fetches GET /bookings/bk_5Rt9 with its own credentials, so it always gets the latest state and nothing private leaks if the URL is wrong. Fat event The whole booking in the body. Saves a call, but it can be stale by the time it lands, and every field becomes part of the contract you must keep.

Signing: proving the event came from you

The partner’s URL is on the public internet. Anyone who finds it can post a fake booking.confirmed, or capture a real one and send it again tomorrow. So each delivery carries a signature made with a secret only you and the partner know: Read t and v1 t older than 5 min? reject: replay Recompute HMAC over the raw bytes Compare in constant time mismatch: reject Changing the body breaks the signature (tampering). Putting the timestamp inside the signed text stops an old, valid event being replayed. API security

Delivery: retries, duplicates and dead letters

Networks fail, so the sender retries until it sees a 2xx. That gives at-least-once delivery: never lost, sometimes twice. Exactly-once over the internet is not on offer, so the receiver dedupes by the event id. Run each case with both receivers: Backoff with jitter Wait longer each time (1 min, 5 min, 30 min, …) plus a random bit, so 10,000 failed deliveries do not all retry in the same second. Then stop Receiver: 2xx fast Check the signature, store the event, answer 200, and do the slow work from a queue. Slow receivers time out and cause the very retries they then have to dedupe.

Order is not guaranteed

Ticket site sends Partner receives booking.confirmed version 3 first try fails with 503, queued for retry booking.cancelled version 4 Priya cancels a minute later arrives first: partner marks it cancelled retry arrives late: a naive receiver flips it back to confirmed Retries, parallel senders and network paths reorder events. Nothing in HTTP keeps them in line. Carry a version Each event names the resource version (or a per-resource sequence). The receiver ignores anything not newer than what it stored. Or re-fetch Treat the event as “bk_5Rt9 changed” and GET the booking. Whatever order the nudges arrive in, the GET returns the current truth. Thin events make this natural.

Streams: when events never stop

Server-Sent Events browser, one way The seat map for a hot show streams seat.held and seat.released over one long HTTP response. On reconnect the browser sends Last-Event-ID to resume. both ways For a queue-for-tickets screen that also sends “still here” pings. More power, but you own heartbeats, reconnects and scaling of open sockets. A log like Kafka Payments, email and analytics each read the bookings topic at their own pace, keep their own offset and can replay last week. Ordered within a partition only. Stream events are usually fat: the client watching the seat map wants the data now, not a link to fetch. Wire-level basics are on the realtime page (opens in a new tab), and queues versus pub/sub versus logs on the messaging page (opens in a new tab). Partners outside your company usually get webhooks, not direct access to your log. OpenAPI describes request and reply. For event-driven APIs there is AsyncAPI: one document lists the channels (such as bookings), the messages on each with their schemas, who publishes and who subscribes, and the protocol (Kafka, AMQP, MQTT, WebSocket). It gives events the same contract, docs and code generation that OpenAPI gives REST

Bulk operations: many items, one call

One transaction. Right when the items only make sense together, like the seats of one section or a booking with its payment. Partial success Cap the size Say 100 items per call, enforced with a clear 400. Anything bigger is a file import, which is the async job from section 2. Retries need care: a retried partial batch must not create the 3 good seats again, so the batch takes an Idempotency-Key too, or each item carries a client id the server can recognise.

Choosing, and the interview line

who starts client needs main failure 202 + poll the job up to one poll interval nothing but HTTP wasted polls, slow to notice a public, signed endpoint duplicates, reordering, endpoint down SSE / WebSocket server, while connected under a second an open connection misses events while disconnected Log (Kafka) consumer reads inside your network They combine: the import returns 202 and a job, and also fires import.completed to anyone subscribed, so a client may poll, listen, or both. Next: all these contracts will need to change. Evolving an API shows how to change them without breaking the apps and partners that already depend on them. GitHub answers repository statistics with 202 while a background job runs. 202 in the wild Real webhooks

More courses

Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
Build a real API with NestJS — One ticket-booking API, built in order: a plain Node server first, then NestJS modules, validation, a real database, and the features real clients lean on.
Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
Kafka — A beginner-to-senior path through event logs, reliable delivery, production operations, and disaster recovery.
AI engineering for software engineers — Eight free reads, in order, for software engineers moving into AI: when to build an agent, how to build one, how to test it, and how the model works underneath.
Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
Docker — What an image is, why layers cache, and how containers find each other on a network.
Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.
Kubernetes — Kubernetes from zero to operators: why it exists, Pods, Deployments and Services, then configuration, networking, storage, scheduling, scaling and security.