System Design Interview Question: Explain Caching Strategies at Scale

Caching appears in almost every design. Strong answers name the cache layer, the pattern, how data stays fresh and what happens when the cache fails.

Layers of caching

1. Browser - Cache-Control headers for static assets and some API responses.
2. CDN - images, scripts and cacheable pages served close to users worldwide.
3. Application cache - Redis or Memcached for computed results and hot database rows.
4. In-process cache - tiny, very hot data inside each server (config, feature flags).
5. Database cache - the database's own buffer pool.

Patterns

• Cache-aside: app reads cache, falls back to DB and fills the cache. Most common; only requested data is cached.
• Read-through: the cache itself loads missing data from the DB.
• Write-through: writes go to cache and DB together; cache always fresh, writes slower.
• Write-behind: write to cache, flush to DB asynchronously; fast but risks data loss.

Keeping data fresh

• TTLs as a safety net on everything.
• Explicit invalidation (delete the key) when data changes.
• Event-driven invalidation from database change streams for multi-service systems.
• Versioned keys (product:42:v7) to sidestep invalidation races.

Eviction

When memory is full: LRU (least recently used) is the common default; LFU (least frequently used) suits stable hot sets.

Failure modes to mention

• Cache stampede: a hot key expires and thousands of requests hit the DB; fix with request coalescing/locks or early refresh.
• Hot keys: one celebrity's profile overloads one cache node; replicate hot keys or add an in-process layer.
• Cold start: after a cache flush the DB takes full load; warm critical keys first.
• Inconsistency: decide what staleness is acceptable per data type (prices vs view counts).