Save the order, then publish the event. 23 orders vanished.

transcript

Picture month-end at a fintech startup. 847 payments charged to customers. 23 of them never created an order. The culprit? Two innocent lines of code.

Line one: save the order. The insert succeeds. Then the process dies. A pod killed in a rolling deploy. Out of memory. Take your pick. Line two never runs. No event is published. Payment, inventory, email: never told. The order exists. Nothing downstream knows.

Save, then publish? A crash between them loses the event. So publish first? Now the insert fails a constraint. Consumers act on an order that doesn't exist. Two systems, no shared transaction. That's the dual-write problem.

The fix: write the order and the event into the same database. One transaction. The event goes into an outbox table. Commit, and both rows exist. Fail, and neither does. A separate publisher reads the outbox and sends it to Kafka. It marks it published only after Kafka acks.

But here's the catch. The publisher sends, then crashes before marking it. On restart, it sends again. The outbox is at least once. So consumers deduplicate by event ID.

Never write to your database and your broker as two steps. Write once, to the database. Let the outbox carry the event.

Full breakdown in the lecture.