Transactional Outbox Pattern with Kafka Explained

In this chapter

Save and publish without losing either

What breaks

The order is saved to the database, then the Kafka send times out. The order exists, but no event was sent. A database write and a Kafka send are two systems, so they can never be one atomic step (the dual-write problem).

Three ways to respond

• Retry the send forever: The user waits on Kafka.
• Use a transactional outbox: Save order + event row together. A relay publishes later. (the fix)
• Publish first, then save: Now you can get an event with no order.

The fix: Use a transactional outbox

Two separate writes can’t be atomic. The outbox puts both in one database transaction: if it rolls back, neither exists; if it commits, both do. A relay (polling) or CDC (log tailing) publishes the row and retries until Kafka acks. That makes delivery at-least-once, never at-most-once.

Remember: One transaction, then relay. Expect resends.

More on Reliability problems

• Kafka acks=all and min.insync.replicas Explained
• Idempotent Kafka Consumers: Handle Duplicate Events
• Kafka Outbox Duplicates: Dedupe by Event ID, Not Offset

Keep learning

• Previous: Kafka Fan-Out: One Event, Many Consumer Groups
• Next: Kafka acks=all and min.insync.replicas Explained
• All Kafka tutorial chapters

More courses

• Jobs, linked to lessons — Browse jobs where every requirement links to the chapter that teaches it. See how ready you are, how long the gap takes to close, and which listings look stale.
• Match a job post to courses — Paste a job post. Each skill is matched to a track here and put in learning order, foundations first.
• React from open source with patterns — Patterns extracted from Material UI, Radix, React Aria, React Hook Form, Redux Toolkit, TanStack, and Next.js — not slides about hooks.
• Database in depth — How a query becomes work. Indexes, isolation, locks, and the plans that explain why a page is slow — not an ORM tour.
• System Design — Core concepts first (SQL vs NoSQL, CAP, indexing, replication, sharding), then designs grouped by what carries them: the database, Redis, Kafka, or many services.
• Microservices — Microservice patterns one diagram at a time: how to split a monolith, how services talk, and how data stays consistent without distributed transactions.
• Frontend system design — How a UI is sliced when it is a product, not a page: ownership of state, cache, routes, and what happens when the network lies.
• TypeScript in depth — The type system as a design tool. Inference, narrowing, generics, and the errors that mean the model is wrong — not a catalog of utility types.
• CSS in depth — See the box, then the page. Visual chapters from the box model through Flexbox surprises, Grid, selectors, and units, then backgrounds, shapes, and visual effects, each with a playground.
• Accessibility — Foundations first (WCAG in plain words, semantic HTML, lists, alt text), then hands-on HTML labs, a full pass of WAI-ARIA 1.3, and a final fix-the-page challenge.
• Docker — What an image is, why layers cache, and how containers find each other on a network.
• Redis — Fifteen Redis patterns, one picture each: cache, session, lock, counter, rate limiter, queue, and when Redis is the wrong tool.