Caching layers: where to cache and what it costs you
The fastest request is the one that never reaches your database. Most "make it faster" and many "make it cheaper" design questions have the same answer - cache - and the real question is where: at the edge, at the API, inside the compute, or in front of the database. Each layer has a different hit ratio, a different way of going stale, and a different failure mode when it empties. AWS V runs CloudFront, ElastiCache and DynamoDB hands-on; this lesson is the design view, with the one cache you already own: the memory of a warm Lambda environment.
Need to know: a cache returns a copy that may be stale for up to its TTL. The hit ratio (hits / all requests) decides whether it helps. Cache-aside (lazy loading): read the cache, on a miss read the database and fill the cache - only what is asked for is cached, a miss costs one extra round trip. Write-through: every write updates the cache too - always fresh, but you cache data nobody reads. When a popular key expires, every client misses at once and hits the database together - a cache stampede; stagger TTLs or let one request refill.
A cache inside the function
try-prices keeps prices in a Map at module level, which lives as long as the execution environment. A database read costs 300 ms:
$ cd ~/oncall-lab/labs/aws/try6
$ head -12 prices/index.mjs
const cache = new Map();
const TTL_MS = Number(process.env.TTL_MS || 60000);
async function fromDatabase(sku) {
await new Promise((r) => setTimeout(r, 300));
return { sku, price: 19.99 };
}
export const handler = async (event) => {
const hit = cache.get(event.sku);
if (hit && Date.now() - hit.at < TTL_MS) { console.log('cache hit', event.sku); return { ...hit.value, cached: true }; }
console.log('cache miss', event.sku);
const value = await fromDatabase(event.sku);
cache.set(event.sku, { value, at: Date.now() });
$ for s in A A A B A; do aws lambda invoke --function-name try-prices --payload "{\"sku\":\"$s\"}" out.json >/dev/null; cat out.json; echo; done
{"sku":"A","price":19.99,"cached":false}
{"sku":"A","price":19.99,"cached":true}
{"sku":"A","price":19.99,"cached":true}
{"sku":"B","price":19.99,"cached":false}
{"sku":"A","price":19.99,"cached":true}
$ aws logs tail /aws/lambda/try-prices --format short | grep -oP 'cache \w+ \w|\tDuration: [\d.]+ ms' | paste - -
cache miss A Duration: 300.30 ms
cache hit A Duration: 0.90 ms
cache hit A Duration: 0.90 ms
cache miss B Duration: 300.30 ms
cache hit A Duration: 0.90 ms
A miss costs the database round trip (and here the first one also the cold start); a hit costs under a millisecond. But this cache is per environment: ten concurrent environments = ten separate caches, each with its own misses, and all of it is gone when the environment is recycled - after a deployment, a configuration change, an idle period or an error:
$ aws lambda update-function-configuration --function-name try-prices --environment 'Variables={TTL_MS=30000}' --query LastUpdateStatus --output text
InProgress
$ aws lambda wait function-updated-v2 --function-name try-prices
$ aws lambda invoke --function-name try-prices --payload '{"sku":"A"}' out.json >/dev/null; cat out.json; echo
{"sku":"A","price":19.99,"cached":false}
Good for configuration, tokens and small reference data; not a shared cache. A shared cache lives outside the compute: ElastiCache, DAX, or the edge.
The layers
| Layer | Service | Caches | Invalidate by | Note |
|---|---|---|---|---|
| browser / client | Cache-Control headers | static files, API responses | TTL, versioned file names | free, and the hardest to purge |
| edge | CloudFront | static assets, media, cacheable API GETs, close to users | TTL, create-invalidation, versioned paths | cache key = path + chosen headers / query / cookies; origin shield; also absorbs DDoS with Shield |
| API | API Gateway REST stage cache | method responses per stage, 0.5-237 GB, TTL 300 s default | TTL, flush stage cache | REST APIs only (not simulated here) |
| compute | warm Lambda / container memory | config, tokens, reference data | environment recycling | per instance, not shared |
| data | ElastiCache (Valkey, Redis OSS, Memcached) | query results, sessions, leaderboards, rate limits | TTL, delete key | Redis/Valkey: replication, Multi-AZ, persistence; Memcached: simple, multi-threaded, no replication |
| data | DynamoDB Accelerator (DAX) | DynamoDB item and query reads, microseconds | write-through for items; TTL | API-compatible client; only for DynamoDB |
| data | RDS / Aurora read replicas | not a cache: read scaling, asynchronous | - | replica lag; queries still cost database CPU |
Rules of thumb:
- Static and shared content goes to the edge. CloudFront in front of S3 (with origin access control) or an ALB serves most bytes without touching the origin, lowers latency for distant users and costs less per GB than direct S3 egress. Global Accelerator is not a cache: it routes TCP/UDP over the AWS backbone to the nearest healthy Region endpoint (static anycast IPs, fast failover) - for non-HTTP traffic or when you need fixed IPs.
- Read-heavy database, same queries: ElastiCache cache-aside, with a TTL that matches how stale the data may be. Sessions belong in ElastiCache (or DynamoDB), never in one server's memory - that is what makes a web tier stateless and scalable.
- DynamoDB hot reads in microseconds: DAX. Not for write-heavy or strongly consistent reads.
- Before adding a cache, fix the query: an index, a projection, fewer round trips.
Operating a cache
- Watch the hit ratio (CloudFront
CacheHitRate, ElastiCacheCacheHits/CacheMisses): a falling ratio means a changed cache key, a TTL that is too short, or new traffic patterns. - Size for the cold cache. After a flush, a failover or a deploy the database takes the full load - can it? If not, warm the cache first or put a queue in front of the writes.
- Evictions (ElastiCache
Evictions) = the cache is too small for the working set; memory pressure on Redis/Valkey also shows asDatabaseMemoryUsagePercentage. - Never trust a cache for correctness: money, stock and permissions are read from the source when it matters, or invalidated on every write.
- Negative caching: cache "not found" briefly too, or a missing key sends every request to the database.
On the exam (SAA-C03 3.2 "high-performing and elastic", 3.3, 3.4; SOA-C03 2.1): CloudFront for static and dynamic content at the edge (and to cut S3 egress costs), ElastiCache for session state and query caching (Redis/Valkey for replication and persistence, Memcached for simple multi-threaded caching), DAX for DynamoDB in microseconds, read replicas for read scaling, API Gateway caching to reduce calls to the backend, Global Accelerator for static IPs and fast Regional failover.
In an interview: "Your RDS database is at 90% CPU from reads. What do you do?" - "First the queries - the slow query log or Performance Insights for the top statements and missing indexes. Then caching: cache-aside in ElastiCache for the hot, repeated reads with a TTL the business accepts, and read replicas for reads that must be fresh-ish. I'd watch the hit ratio and make sure the database survives a cold cache."
You can now: explain TTL, hit ratio, cache-aside, write-through and stampedes, use and judge an in-function cache, place CloudFront, API Gateway caching, ElastiCache, DAX and read replicas for a requirement, tell CloudFront from Global Accelerator, and operate a cache by its hit ratio, evictions and cold-start load.