Why do message brokers deliver the same event twice?

At-least-once delivery is the practical guarantee: a broker redelivers when an acknowledgment times out, a consumer crashes after processing but before acking, or a producer retries a publish. Exactly-once delivery across a network cannot be guaranteed at the transport, so duplicates are a matter of when, not if.

What is an idempotent consumer?

A consumer that produces the same result no matter how many times it receives the same event. The standard implementation records each event id in a processed_events table with a unique constraint, inside the same database transaction as the business write, so a duplicate delivery aborts harmlessly instead of double-counting.

What is effectively-once processing?

The combination of at-least-once delivery with idempotent processing: events may arrive multiple times, but their effect is applied exactly once. It is the practical alternative to the myth of exactly-once delivery, and it pairs the transactional outbox on the producer side with an idempotent consumer on the consumer side.

Why a separate processed_events table instead of a unique column on a business table?

A consumer's effect is often not one inserted row: it may update a balance, touch several tables, or handle many event types, so there is no single business row to carry a unique event id. A dedicated processed_events table gives every handler one transactional dedup mechanism and keeps delivery bookkeeping out of the domain schema, with its own retention. When a handler does insert exactly one row per event, a unique event_id column on that table is a legitimate equivalent.

Failure Modes At-Least-Once Delivery → Idempotent Consumer

Delivered twice.
Counted once.

Brokers deliver at-least-once, so the same event can arrive twice: after a timeout, after a crash, after a retry. Redeliver one: the naive consumer credits the ledger twice; the idempotent one counts it once.

charged $0 · 0 delivered · 0 duplicated

Naive, counts every delivery

in ledger: $0
overcount: $0

broker

delivers

consumer

applies

postgres

stores

ledger

Idempotent, records once

in ledger: $0
deduped: $0

broker

delivers

consumer

checks id

postgres

stores

processed_events

ledger

discarded

no duplicates yet

Deliver a few events, then redeliver one: brokers repeat deliveries after timeouts, retries, and crashes. That delivery was a repeat: the naive ledger counted it again; the idempotent one knew the id and dropped it.

How the idempotent consumer actually works read more

"Exactly-once" delivery is a myth. Brokers redeliver after a timeout or a crash, so the same event can show up more than once. A naive consumer just applies whatever arrives, so a repeat credits the ledger twice and money gets created out of nothing.

The fix: record each event's id in a processed_events table with a unique constraint, in the same transaction as the ledger update:

BEGIN;
  INSERT INTO processed_events (event_id) VALUES ($1);
  -- unique(event_id): a duplicate aborts the whole transaction
  UPDATE ledger SET balance = balance + $2;
COMMIT;

A duplicate hits that constraint and the whole transaction rolls back, harmlessly. Put together, at-least-once delivery plus an idempotent consumer gets you effectively-once.

Why a table, not a column? This handler updates a balance, it doesn't insert a fresh row, so there's nothing to attach a unique event id to. A handler that does insert one row per event can skip the table and put a unique event_id column on that row instead.

The trade-off: this table grows with every event, that's one more write to index, and the dedup key has to be the event id, never the payload. Two separate $20 payments still both need to count.