05 - Transactional Outbox

Transactional Outbox

Problem

We need a database change and an event publication to behave consistently, but PostgreSQL and Kafka are separate systems.

The Dangerous Approach

orderRepository.save(order);       // DB commit
kafkaTemplate.send("order-events", event); // Kafka

Failure:

DB commit       ✓
Kafka publish   ✗

Now the order exists, but downstream services never receive OrderCreated.

The reverse ordering has the opposite problem.

Outbox Pattern

Write the business record and the event into the same database transaction.

flowchart LR
    Service["Order Service"]
    Service --> TX["One DB Transaction"]
    TX --> Orders[("orders")]
    TX --> Outbox[("outbox")]
    Outbox --> Publisher["Outbox Publisher"]
    Publisher --> Kafka["Kafka"]

Java

@Transactional
public void createOrder(Order order) {
    orderRepository.save(order);
 
    outboxRepository.save(
        OutboxMessage.of("OrderCreated", order.id())
    );
}

.NET

await using var tx =
    await db.Database.BeginTransactionAsync();
 
db.Orders.Add(order);
db.OutboxMessages.Add(
    OutboxMessage.For("OrderCreated", order.Id));
 
await db.SaveChangesAsync();
await tx.CommitAsync();

Now:

orders INSERT ✓
outbox INSERT ✓

or:

orders INSERT ✗
outbox INSERT ✗

Publishing Later

A separate publisher reads the outbox:

outbox
   │
   ▼
Publisher
   │
   ▼
Kafka

A simplified query:

SELECT *
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 100;

After successful publication, mark the record as published.

Important

Outbox does not magically create exactly-once processing.

A crash can happen here:

Publish Kafka event ✓
Mark outbox row published ✗
Application crashes

The publisher may publish the same event again.

Therefore consumers should be idempotent.

Idempotency

Outbox Table

A practical table might contain:

id
aggregate_id
event_type
payload
created_at
published_at

id is particularly useful as a unique event/message ID.

Why This Is Better Than 2PC

2PC attempts to coordinate multiple resources in a distributed transaction.

Outbox instead says:

DB transaction
     ↓
durably record intent
     ↓
asynchronously publish

This is usually simpler and more resilient for event-driven microservices.

Interview Question

Does Outbox guarantee Kafka publication?

It guarantees that the intent to publish is durably stored with the business transaction. The publisher must still successfully deliver the event, retry failures, and handle duplicates.

Remember

Outbox solves DB → Event consistency. Idempotency solves duplicate processing.