Engineering Notes

Caching

3 min read

Improves performance of systems

Where do we cache data

Data is cached everywhere, from the front end to the back end!

DB Caching Strategy

Ref

Cache Aside / Lazy Loading

  • Load from cache
  • Read and write from / to cache first

Read Through

Write Through

  • Write to both cache and db at the same time

Write Back

  • Write to cache first
    • Then update DB async

Write Around

Cache Invalidation

Cache invalidation refers to the process of ensuring that stored (cached) data is updated or removed when the original source changes, so users don’t see outdated information. It is about correctness

  • Prioritize accuracy: Always make sure your system updates or clears cached data promptly when users make important changes, so they see the latest information.
  • Set thoughtful rules: Choose cache expiry settings and invalidation strategies based on how often your data changes, instead of relying on defaults.
  • Track cache versions: Attach version numbers or timestamps to cached data to ensure old cache entries aren’t accidentally used after updates or system changes.

Ref Ref

Cache Invalidation Strategies

  • Explicit deletion after a write
  • TTL expiration
  • Versioning
  • Cache tags
  • Event-driven invalidation

Cache Eviction

Cache eviction is the process of removing data from a cache when it becomes full to make space for new or more relevant data. It ensures that limited cache memory is used efficiently while keeping frequently accessed data available. It is about cache capacity/resources

  • Removes less useful or outdated data when cache reaches capacity, maintaining efficient memory usage.
  • Uses policies like LRU, LFU, and FIFO to decide which data to evict and improve performance.

Cache Eviction Policies

Ref3 Ref4 Ref5

Policies

TTL Least Recently Used (LRU) Least Frequently Used (LFU) First-In-First-Out (FIFO) Most Recently Used (MRU) Random Replacement (RR)

Use Cases of Eviction Policy

FIFO

  • Task Scheduling in Operating System
  • Message Queues
  • Cache for Streaming Applications

RR

  • Non-Critical Caching Environments
  • Simulation and Testing
  • Resource-Constrained Systems

Edge Cases & Failure Modes

How would you mitigate these three classic distributed caching problems in your architecture?

• Cache Stampede (Thundering Herd 1): A popular cache key expires, and thousands of concurrent requests hit the database simultaneously to recompute it.

• Cache Penetration: Attackers send thousands of requests for non-existent keys (e.g., IDs that don’t exist in the DB), bypassing the cache every time and flooding the DB.

• Cache Avalanche: A large percentage of cached items expire at the exact same second, causing a sudden spike on the backend

References

Graph View

Backlinks