Microservices

Chapter 10 of 27 · Part 5: Data management

Event sourcing

Save an aggregate as the sequence of events that changed it, and rebuild state by replaying them.

key idea / coordinatorservice / good pathin progress / watch outproblem / failureneutral

The problem with traditional saving

A normal table keeps only the latest state, so history, auditing and event publishing all have to be added on.

Two ways to save an OrderTraditional persistenceCurrent state in one ORDER rowUpdates overwrite historyEvents and audit log are bolted onObject-relational mismatchEvent sourcingMany rows in an EVENTS tableAppend only: history keptEvents are the dataSimple, serializable events

What event sourcing does

Instead of the current state, store every state change as an event. The current state is rebuilt by replaying them.

EVENTS table for one orderAn order is stored as a list of events, never updated, only appended.EVENTS table (append only)event_idevent_typeentity102OrderCreatedOrder 101103OrderApprovedOrder 101104OrderShippedOrder 101105OrderDeliveredOrder 101Current state = replay these events in order.
Rebuilding an aggregateLoad eventsfor one aggregatenew Order()empty objectapply(event)once per eventCurrent stateready to useThis is a fold (reduce) over the list of events.

Commands become events

An update is split into two parts: one decides what happened, the other applies it.

Process and applyCommande.g. ReviseOrderprocess()checks rulesreturns eventsapply(event)changes statecannot failSave eventsappend to storeprocess() decides. apply() just records what happened.

Publishing and speed

Saved events can be published reliably. Snapshots shorten long histories.

Reliably publishing eventsSave eventsone atomic insertEVENTS tablepermanent recordEvent relaytails the DB logMessage brokersubscribers readSaving an event is atomic, so event sourcing doubles as reliable event publishing.
Snapshots speed up loadingEvents 1 to NskippedSnapshot Nsaved stateEvent N+1applyEvent N+2applyLoad the latest snapshot, then only the events after it.

Details you must handle

Events live forever, so their shape will change. Duplicates will arrive.

Evolving events with upcastingOld event v1stored foreverUpcasterconverts on loadEvent v2current shapeAggregateonly knows v2
Handling duplicate messagesRelational event storeInsert id into PROCESSED_MESSAGESIn the same transaction as the eventsNoSQL event storeStore message id inside the new eventsIf none: save a pseudo event
Parts of an event storeEvent storedatabase + brokerEVENTSevery eventENTITIESaggregate + versionSNAPSHOTSsaved statesOptimistic locking uses the version to stop two writers overwriting each other.

Is it worth it?

Event sourcing gives history and reliable events, and costs a different way of coding.

Benefits and drawbacks of event sourcingBenefitsReliable event publishingComplete, accurate historyAvoids object-relational mismatchTime machine for new needsDrawbacksNew programming modelEvent schemas must evolveDeleting data is tricky (GDPR)Querying needs CQRS

Event-driven versus event-sourced

Many systems are event-driven without being event-sourced. The difference is where the truth lives.

Two ways to use eventsEvent-drivenState saved in a tableEvents announce changesTable and events can driftEvent-sourcedEvents are the entityState = replay of eventsOne source of truthIn an event-sourced entity, the latest state is what you get by applying all its events.

Brokers matter too. A traditional broker such as RabbitMQ is built for destructive consuming: once a consumer acknowledges a message, it is removed. A durable log such as Kafka keeps messages after they are consumed, so other consumers can read them and existing ones can reprocess them. Replay needs a durable log.

Ephemeral queue versus durable logEphemeral queueConsumer acksMessage is removedBest for fast, short-lived workDurable logConsumer readsMessage staysReplay and new consumers

From Practical Event-Driven Microservices Architecture (ch. 3 and 4).

The broker as the event store

With a durable broker you can skip the events table and keep the events only on the log. This removes the need to write two places and keep them equal. A read model then builds a query-friendly projection from the same stream.

Events stored on the brokerProduct servicevalidates commandsEvent logstores the eventsRead modeldenormalized viewRebuild the read model, or start a new one, by reading the log from the start.

The price is that a query for one entity is awkward. A log is not a database, so loading one aggregate's events may mean scanning a topic. Kafka Streams state stores can help. Business rules that need a query (for example, "only 25% of products may cost over 400") must read an eventually consistent view and can see stale data.

Good fitPoor fit
Complex query needs (with CQRS read models)Simple, isolated services
Core domain; new projections wanted laterSmall data, strong consistency, rare change
Strong audit requirementsTeams not ready for snapshots and retention work

A cousin is command sourcing: store the commands as received, not the events. It lets you replay user requests after fixing a bug or compare algorithm versions. It also adds async feedback problems, since a rejected command is discovered later. Most of the time, event sourcing is enough.

From Practical Event-Driven Microservices Architecture (ch. 4 and ch. 7).

Replay, compaction and forgetting

A compacted Kafka topic keeps only the latest message for each key. Publish one whole document per entity, keyed by id, and the topic works like a table that anyone can read from offset zero.

Log compaction by keyBefore (key = product id)off 1 key Aoff 2 key Boff 3 key Coff 5 key Coff 8 key CAfter compaction (older C removed)off 1 key Aoff 2 key Boff 8 key CExample offsets. The book keeps offset 8.

New consumers and rebuilds then need no special tool: the model is rebuilt by the same code that handles live events, and more instances can speed it up.

A permanent log clashes with the right to be forgotten. One answer is crypto shredding: encrypt each user's events with that user's key, and delete the key to make the data unreadable.

Crypto shreddingEncryptone key per userLog keepsencrypted eventsUser leavesdestroy the keyUnreadableevents remainOnly encrypt in services that hold sensitive data, or do it in a shared sidecar.

From Practical Event-Driven Microservices Architecture (ch. 3 and 4).

Joining streams into one event

Sometimes a consumer needs data from several streams, which have different keys and speeds. That makes ordering and concurrency hard. A process manager can join them into one denormalized event for that boundary.

A process manager builds a denormalized orderOrder eventsorder dataInventory eventsstock dataProduct eventsproduct dataProcess managermerges, no domain logicShipping orderone fat eventFat events are the price. In return, ordering and concurrency are solved once, by design.
Keep the manager thin. Its job is to adapt and merge, so it acts like an anti-corruption layer. Business rules stay in the owning service. For event-sourced events, prefer intent-revealing names such as OrderAddressChanged; make a document or fat event for consumers who need many fields.

From Practical Event-Driven Microservices Architecture (ch. 8, Denormalized Event Schema).

Recap

Quick check

How does an event-sourced app know the current state of an order?
Show answer

It loads all of the order's events (or a snapshot plus later events), creates an empty Order and calls apply() for each event in order.