Why do message brokers deliver the same event twice?

At-least-once delivery is the practical guarantee: a broker redelivers when an acknowledgment times out, a consumer crashes after processing but before acking, or a producer retries a publish. Exactly-once delivery across a network cannot be guaranteed at the transport, so duplicates are a matter of when, not if.

What is effectively-once processing?

The combination of at-least-once delivery with idempotent processing: events may arrive multiple times, but their effect is applied exactly once. It is the practical alternative to the myth of exactly-once delivery, and it pairs the transactional outbox on the producer side with an idempotent consumer on the consumer side.

Why a separate processed_events table instead of a unique column on a business table?

A consumer's effect is often not one inserted row: it may update a balance, touch several tables, or handle many event types, so there is no single business row to carry a unique event id. A dedicated processed_events table gives every handler one transactional dedup mechanism and keeps delivery bookkeeping out of the domain schema, with its own retention. When a handler does insert exactly one row per event, a unique event_id column on that table is a legitimate equivalent.

Failure Modes At-Least-Once Delivery → Idempotent Consumer

Delivered twice.
Counted once.

Q: What is an idempotent consumer?

A consumer that produces the same result no matter how many times it receives the same event. The standard implementation records each event id in a processed_events table with a unique constraint, inside the same database transaction as the business write, so a duplicate delivery aborts harmlessly instead of double-counting.

Every payment event arrives at least once, and sometimes more: retries, timeouts, crashes. Redeliver one: the naive design counts the same money twice, while the idempotent one counts it exactly once.

charged $0 · 0 delivered · 0 duplicated

Naive, counts every delivery

in ledger: $0
overcount: $0

broker

delivers

consumer

applies

postgres

stores

ledger

Idempotent, records once

in ledger: $0
deduped: $0

broker

delivers

consumer

checks id

postgres

stores

processed_events

ledger

discarded

no duplicates yet

Deliver a few events, then redeliver one: brokers repeat deliveries after timeouts, retries, and crashes. That delivery was a repeat: the naive ledger counted it again; the idempotent one knew the id and dropped it.

How the idempotent consumer actually works read more

Exactly-once delivery is a myth: brokers redeliver after timeouts and crashes. The naive consumer applies whatever arrives, so a repeat credits the ledger twice and money is created from nothing.

The idempotent consumer records each event id in a processed_events table with a unique constraint, in the same transaction as the ledger write:

BEGIN;
  INSERT INTO processed_events (event_id) VALUES ($1);
  -- unique(event_id): a duplicate aborts the whole transaction
  UPDATE ledger SET balance = balance + $2;
COMMIT;

A duplicate rolls back, harmlessly. At-least-once delivery plus idempotent processing adds up to effectively-once.

Why a separate table? The consumer's effect is often not one inserted row: it may update a balance or touch several tables, so there is no business row to carry a unique event id. When a handler does insert exactly one row per event, a unique event_id column on that table works just as well.

The cost: a table that grows with every event, one more indexed write, and the dedup key must be the event id, never the payload: two legitimate $20 payments must both count.