Engineering Notes

03 - Event Driven Architecture

Properties1
tagsevent-driven, architecture, kafka, deep-dive

3 min read

Event-Driven Architecture

Mental model In an event-driven system, a service

publishes facts about things that happened. Other services can react without the producer knowing exactly who the consumers are.

Synchronous Approach

sequenceDiagram
    participant O as Order
    participant P as Payment
    participant I as Inventory
    participant N as Notification

    O->>P: Process Payment
    P-->>O: Success
    O->>I: Reserve Inventory
    I-->>O: Success
    O->>N: Send Notification
    N-->>O: Sent

The Order service is coupled to the downstream services.

Event-Driven Approach

flowchart LR
    Order["Order Service"] -->|"OrderCreated"| Kafka["Kafka"]

    Kafka --> Payment["Payment"]
    Kafka --> Inventory["Inventory"]
    Kafka --> Notification["Notification"]

Order only publishes:

OrderCreated

Consumers independently react.

Event Is a Fact

Good event names usually represent something that already happened:

OrderCreated
PaymentCompleted
InventoryReserved
ShipmentCreated

Compare:

ProcessPayment
ReserveInventory

Those sound more like commands.

Remember Command = please do this. Event = this already happened.

Commands vs Events

Command Event

Request to perform an action Fact about something that happened Usually one intended handler Can have many consumers ReserveInventory InventoryReserved ProcessPayment PaymentCompleted

Benefits

Decoupling

The Order service doesn’t need to know every consumer.

Independent scaling

Payment consumers:     10
Notification consumers: 3
Inventory consumers:    8

Each can scale independently.

Fan-out

One event can trigger multiple workflows.

             OrderCreated
                  │
        ┌─────────┼─────────┐
        ▼         ▼         ▼
     Payment   Inventory  Notification

But Events Add Complexity

Don't use Kafka just because you can. Event-driven

systems introduce asynchronous behavior, eventual consistency, duplicate messages, ordering concerns, retries, dead-letter handling, monitoring challenges, and more difficult debugging.

Failure Scenario

Suppose:

Order DB → COMMIT
Kafka publish → FAIL

Now the order exists but consumers don’t know about it.

This is one of the central problems we will solve with:

[[Transactional Outbox]]

Another Failure Scenario

Suppose Kafka publishes successfully but the consumer crashes:

Kafka
  │
  ▼
Payment Service
  │
  ├── payment succeeds
  │
  └── process crashes before offset commit

The message can be delivered again.

Therefore we need:

[[Idempotency]]

When Should We Prefer Events?

Good candidates:

  • order lifecycle notifications
  • inventory updates
  • payment completion
  • shipment updates
  • analytics
  • audit events
  • email/notification workflows

Not everything should become an event.

If the caller needs an immediate answer:

GET /products/123

a synchronous API is usually simpler.

Interview Question

Why not make every microservice communicate through

Kafka?

Because asynchronous communication adds complexity. Use it when decoupling, asynchronous processing, fan-out, buffering, or independent scaling provides a meaningful benefit.

Project Decision

We’ll use both:

Synchronous REST
    ↓
Queries / immediate responses
 
Kafka
    ↓
Domain events / asynchronous workflows

This is a more realistic architecture than trying to make everything event-driven.

Next

[[Transactional Outbox]]

The next problem is critical:

How do we guarantee that a database transaction and an event publication don’t get out of sync?

Graph View