OnCallReadyStart learning

AWS Lesson 39.26 · AWS VI: Architecting on AWS & the exams · 12 min read

Caching layers: where to cache and what it costs you

In plain words

A library keeps popular books on a shelf by the door so readers do not have to wait for one from the basement. The shelf is fast, but if a new edition arrives, the copy by the door is out of date until someone swaps it. Caching is that shelf: at the edge near users (CloudFront), in front of the database (ElastiCache), or in your own pocket for a moment (a warm Lambda's memory). The art is deciding what goes on the shelf and for how long.

Caching layers: where to cache and what it costs you

The fastest request is the one that never reaches your database. Most "make it faster" and many "make it cheaper" design questions have the same answer - cache - and the real question is where: at the edge, at the API, inside the compute, or in front of the database. Each layer has a different hit ratio, a different way of going stale, and a different failure mode when it empties. AWS V runs CloudFront, ElastiCache and DynamoDB hands-on; this lesson is the design view, with the one cache you already own: the memory of a warm Lambda environment.

Need to know: a cache returns a copy that may be stale for up to its TTL. The hit ratio (hits / all requests) decides whether it helps. Cache-aside (lazy loading): read the cache, on a miss read the database and fill the cache - only what is asked for is cached, a miss costs one extra round trip. Write-through: every write updates the cache too - always fresh, but you cache data nobody reads. When a popular key expires, every client misses at once and hits the database together - a cache stampede; stagger TTLs or let one request refill.

A cache inside the function

try-prices keeps prices in a Map at module level, which lives as long as the execution environment. A database read costs 300 ms:

$ cd ~/oncall-lab/labs/aws/try6
$ head -12 prices/index.mjs
const cache = new Map();
const TTL_MS = Number(process.env.TTL_MS || 60000);
async function fromDatabase(sku) {
  await new Promise((r) => setTimeout(r, 300));
  return { sku, price: 19.99 };
}
export const handler = async (event) => {
  const hit = cache.get(event.sku);
  if (hit && Date.now() - hit.at < TTL_MS) { console.log('cache hit', event.sku); return { ...hit.value, cached: true }; }
  console.log('cache miss', event.sku);
  const value = await fromDatabase(event.sku);
  cache.set(event.sku, { value, at: Date.now() });
$ for s in A A A B A; do aws lambda invoke --function-name try-prices --payload "{\"sku\":\"$s\"}" out.json >/dev/null; cat out.json; echo; done
{"sku":"A","price":19.99,"cached":false}
{"sku":"A","price":19.99,"cached":true}
{"sku":"A","price":19.99,"cached":true}
{"sku":"B","price":19.99,"cached":false}
{"sku":"A","price":19.99,"cached":true}
$ aws logs tail /aws/lambda/try-prices --format short | grep -oP 'cache \w+ \w|\tDuration: [\d.]+ ms' | paste - -
cache miss A		Duration: 300.30 ms
cache hit A		Duration: 0.90 ms
cache hit A		Duration: 0.90 ms
cache miss B		Duration: 300.30 ms
cache hit A		Duration: 0.90 ms

A miss costs the database round trip (and here the first one also the cold start); a hit costs under a millisecond. But this cache is per environment: ten concurrent environments = ten separate caches, each with its own misses, and all of it is gone when the environment is recycled - after a deployment, a configuration change, an idle period or an error:

$ aws lambda update-function-configuration --function-name try-prices --environment 'Variables={TTL_MS=30000}' --query LastUpdateStatus --output text
InProgress
$ aws lambda wait function-updated-v2 --function-name try-prices
$ aws lambda invoke --function-name try-prices --payload '{"sku":"A"}' out.json >/dev/null; cat out.json; echo
{"sku":"A","price":19.99,"cached":false}

Good for configuration, tokens and small reference data; not a shared cache. A shared cache lives outside the compute: ElastiCache, DAX, or the edge.

The layers

LayerServiceCachesInvalidate byNote
browser / clientCache-Control headersstatic files, API responsesTTL, versioned file namesfree, and the hardest to purge
edgeCloudFrontstatic assets, media, cacheable API GETs, close to usersTTL, create-invalidation, versioned pathscache key = path + chosen headers / query / cookies; origin shield; also absorbs DDoS with Shield
APIAPI Gateway REST stage cachemethod responses per stage, 0.5-237 GB, TTL 300 s defaultTTL, flush stage cacheREST APIs only (not simulated here)
computewarm Lambda / container memoryconfig, tokens, reference dataenvironment recyclingper instance, not shared
dataElastiCache (Valkey, Redis OSS, Memcached)query results, sessions, leaderboards, rate limitsTTL, delete keyRedis/Valkey: replication, Multi-AZ, persistence; Memcached: simple, multi-threaded, no replication
dataDynamoDB Accelerator (DAX)DynamoDB item and query reads, microsecondswrite-through for items; TTLAPI-compatible client; only for DynamoDB
dataRDS / Aurora read replicasnot a cache: read scaling, asynchronous-replica lag; queries still cost database CPU

Rules of thumb:

  • Static and shared content goes to the edge. CloudFront in front of S3 (with origin access control) or an ALB serves most bytes without touching the origin, lowers latency for distant users and costs less per GB than direct S3 egress. Global Accelerator is not a cache: it routes TCP/UDP over the AWS backbone to the nearest healthy Region endpoint (static anycast IPs, fast failover) - for non-HTTP traffic or when you need fixed IPs.
  • Read-heavy database, same queries: ElastiCache cache-aside, with a TTL that matches how stale the data may be. Sessions belong in ElastiCache (or DynamoDB), never in one server's memory - that is what makes a web tier stateless and scalable.
  • DynamoDB hot reads in microseconds: DAX. Not for write-heavy or strongly consistent reads.
  • Before adding a cache, fix the query: an index, a projection, fewer round trips.

Operating a cache

  • Watch the hit ratio (CloudFront CacheHitRate, ElastiCache CacheHits / CacheMisses): a falling ratio means a changed cache key, a TTL that is too short, or new traffic patterns.
  • Size for the cold cache. After a flush, a failover or a deploy the database takes the full load - can it? If not, warm the cache first or put a queue in front of the writes.
  • Evictions (ElastiCache Evictions) = the cache is too small for the working set; memory pressure on Redis/Valkey also shows as DatabaseMemoryUsagePercentage.
  • Never trust a cache for correctness: money, stock and permissions are read from the source when it matters, or invalidated on every write.
  • Negative caching: cache "not found" briefly too, or a missing key sends every request to the database.

On the exam (SAA-C03 3.2 "high-performing and elastic", 3.3, 3.4; SOA-C03 2.1): CloudFront for static and dynamic content at the edge (and to cut S3 egress costs), ElastiCache for session state and query caching (Redis/Valkey for replication and persistence, Memcached for simple multi-threaded caching), DAX for DynamoDB in microseconds, read replicas for read scaling, API Gateway caching to reduce calls to the backend, Global Accelerator for static IPs and fast Regional failover.

In an interview: "Your RDS database is at 90% CPU from reads. What do you do?" - "First the queries - the slow query log or Performance Insights for the top statements and missing indexes. Then caching: cache-aside in ElastiCache for the hot, repeated reads with a TTL the business accepts, and read replicas for reads that must be fresh-ish. I'd watch the hit ratio and make sure the database survives a cold cache."

You can now: explain TTL, hit ratio, cache-aside, write-through and stampedes, use and judge an in-function cache, place CloudFront, API Gateway caching, ElastiCache, DAX and read replicas for a requirement, tell CloudFront from Global Accelerator, and operate a cache by its hit ratio, evictions and cold-start load.

Why it helps

Many "make it faster" and "make it cheaper" requirements have the same answer: serve copies instead of recomputing. But every cache can be stale, can empty at the worst moment, and can stampede the database when a popular key expires. Knowing the layers - CloudFront, API Gateway caching, in-process memory, ElastiCache, DAX - and how to operate them by hit ratio and cold-start load is core design judgement and a frequent exam topic.

Commands in this lesson

cd head aws

FAQ

What is cache-aside?

The application reads the cache first; on a miss it reads the database and writes the result to the cache with a TTL. Only data that is actually requested is cached, and a cache failure only makes reads slower. The cost is one extra round trip on every miss and possibly stale data until the TTL expires.

Is a cache inside a Lambda function useful?

For small, slowly changing data such as configuration or tokens, yes: module-level variables survive between invocations in a warm execution environment. But each environment has its own copy, and it is lost when environments are recycled after deployments, configuration changes, idle time or errors - it is not a shared cache.

ElastiCache for Redis OSS / Valkey or Memcached?

Valkey or Redis OSS when you need replication, Multi-AZ failover, persistence, and data structures such as sorted sets for leaderboards or counters. Memcached for a simple, multi-threaded cache of strings with no replication, where losing the cache only costs a refill.

What is a cache stampede?

When a popular key expires, many requests miss at the same moment and all query the database together, which can overload it. Prevent it with staggered or randomized TTLs, letting one request refill the key while others wait or get the stale value, and by sizing the database to survive a cold cache.

Is Global Accelerator a cache?

No. Global Accelerator gives static anycast IP addresses and routes TCP and UDP traffic over the AWS network to the nearest healthy Regional endpoint, with fast failover. It improves latency and availability for non-HTTP or IP-pinned traffic but does not store responses the way CloudFront does.

In an interview Mid

Your RDS database is at 90% CPU from reads. What do you do?

First the queries: find the top statements and missing indexes. Then caching: cache-aside in ElastiCache for the hot, repeated reads with a TTL the business accepts, and read replicas for reads that need fresher data. CloudFront in front of cacheable API responses removes load before it reaches the application. I would watch the hit ratio and make sure the database can survive a cold cache after a flush or failover.

Also asked: What is the difference between CloudFront and Global Accelerator? · When would you use DAX? · How do you keep cached data from going stale?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.