What is the dual write problem?

A service writes to two systems without a shared transaction, for example committing a payment to its database and publishing an event to a message broker. If the broker is down after the commit, the event is lost with no record to retry, so the payment is recorded but the rest of the system never hears about it.

What is the transactional outbox pattern?

The service writes the event into an outbox table in the same database and the same transaction as the business change, so both commit together or neither does and the event cannot be lost. A separate relay polls the outbox and publishes each event, retrying until the broker is reachable.

Why can a payment event be lost with a naive dual write?

The event only exists in flight to the broker; nothing durable records that it should be sent. If the broker is offline or the process crashes after the database commit, there is no row to retry, so the event is gone for good while the payment itself remains recorded.

Failure Modes Dual Write → Outbox

Same payments in.
Different money out.

Every payment is saved, then announced to other systems, and both steps must succeed. Knock the broker offline: the naive design loses money for good, while the outbox holds every cent.

charged $0 · 0 sent

Naive, two writes

in ledger: $0
lost: $0

payment app

charges

postgres

stores

payments

broker

delivers

lost forever

nothing to retry

ledger

records

Transactional outbox, one write

in ledger: $0
held in outbox: $0

payment app

charges

postgres

stores

payments

outbox

relay

polls from outbox

broker

delivers

ledger

records

The broker is up: every event reaches the ledger, so both lanes stay in sync. The broker is down: the naive lane loses its events for good; the outbox holds and replays them.

How the outbox actually works read more

The naive app writes twice: it commits the payment to its database, then publishes an event to the broker. Nothing ties the two together, so if the broker is down after the commit, the event is gone with no record it was ever meant to send.

The outbox makes it one write: the event is inserted into an outbox table in the same database, in the same transaction as the payment, so both rows commit together or neither does.

BEGIN;
  INSERT INTO payments (..., status='captured');
  INSERT INTO outbox   (event='payment.captured', status='pending');
COMMIT;  -- one transaction

A separate relay reads the pending rows and publishes them, marking each done on success. Broker down? The rows just wait, and go out when it is back. A durable queue in front of the broker would not help: that is still a second system, so it is still a dual write. The outbox works because it lives in your own database.

The cost: the relay republishes at least once, so consumers must be idempotent, and delivery is eventual, not instant.