OnCallReadyStart learning

AWS Lesson 39.34 · AWS VI: Architecting on AWS & the exams · 12 min read

A Well-Architected review in practice

In plain words

A Well-Architected review is like a car inspection with a friendly mechanic. Instead of waiting for something to break, you go through a checklist together - brakes, lights, tyres, fuel use - look at the actual car rather than the owner's memory of it, and leave with a short list of what to fix first. Nobody fails; the point is that the car is safer when you drive away.

A Well-Architected review in practice

AWS III introduced the Well-Architected Framework through its reliability pillar. A review is how teams actually use it: a structured conversation, workload by workload, through the framework's questions - with evidence from the account instead of opinions - that ends in a short, prioritised list of risks and an owner for each fix. It is not an audit and nobody fails it; a review with no findings was not looked at closely enough.

Need to know: six pillars, each with design principles and questions:

PillarThe question it asksTypical findings
Operational excellencecan you run, observe and change it safely?no alarms, manual deploys, no runbooks, no post-incident reviews
Securitywho can do what, and how would you know?wide roles, long-lived keys, public data, no encryption or logging
Reliabilitydoes it recover from failures and meet its RPO / RTO?one AZ, no DLQ, untested backups, no quotas headroom
Performance efficiencydoes it use the right resources, and keep up as demand changes?wrong compute type, no caching, oversized or undersized
Cost optimizationdoes it deliver value at the lowest price?idle resources, no commitments on the baseline, no budget, no tags
Sustainabilitydoes it minimise the energy and resources it needs?always-on dev, x86 where Graviton works, data kept forever

Each question's answers are best practices; a missing one is a high risk issue (HRI) or a medium risk issue (MRI). The AWS Well-Architected Tool in the console records the workload, the answers, the risks and an improvement plan, saves milestones to compare later, and adds lenses with extra questions for a kind of workload: Serverless, Containers, SaaS, Financial Services, Generative AI (the wellarchitected API is not simulated here).

Collect the evidence

review.sh gathers read-only evidence for the try- workload - a queue, two functions, an assets bucket - grouped by pillar. Read it first: every line is a CLI call you already know.

$ cd ~/oncall-lab/labs/aws/try6
$ sed -n '1,25p' review.sh
#!/usr/bin/env bash
# review.sh - read-only evidence for a Well-Architected review of one workload
# usage: bash review.sh [PREFIX]    (PREFIX = the workload's name prefix, default try-)
set -uo pipefail
P=${1:-try-}
echo "== Reliability"
for q in $(aws sqs list-queues --queue-name-prefix "$P" --query 'QueueUrls[]' --output text); do
  name=${q##*/}
  case $name in *-dlq) continue ;; esac
  if aws sqs get-queue-attributes --queue-url "$q" --attribute-names RedrivePolicy --output text | grep -q deadLetterTargetArn; then
    echo "  ok    queue $name has a DLQ"
  else
    echo "  HIGH  queue $name has no DLQ: a poison message is retried until it expires"
  fi
done
for f in $(aws lambda list-functions --query "Functions[?starts_with(FunctionName,'$P')].FunctionName" --output text); do
  d=$(aws lambda get-function-event-invoke-config --function-name "$f" --query DestinationConfig.OnFailure.Destination --output text 2>/dev/null)
  if [ -n "$d" ] && [ "$d" != None ]; then echo "  ok    function $f sends failed async events to $d"; else echo "  MED   function $f drops failed async events (no on-failure destination)"; fi
done
for b in $(aws s3api list-buckets --query "Buckets[?starts_with(Name,'oncall-$P')].Name" --output text); do
  v=$(aws s3api get-bucket-versioning --bucket "$b" --query Status --output text)
  if [ "$v" = Enabled ]; then echo "  ok    bucket $b is versioned"; else echo "  HIGH  bucket $b is not versioned: one delete or overwrite loses data"; fi
done
echo "== Operational excellence"
n=$(aws cloudwatch describe-alarms --alarm-name-prefix "$P" --query 'length(MetricAlarms)' --output text)
$ bash review.sh
== Reliability
  HIGH  queue try-orders-in has no DLQ: a poison message is retried until it expires
  MED   function try-intake drops failed async events (no on-failure destination)
  MED   function try-notify drops failed async events (no on-failure destination)
  HIGH  bucket oncall-try-assets-111122223333 is not versioned: one delete or overwrite loses data
== Operational excellence
  HIGH  no alarms: users will tell you about outages
== Security
  MED   role try-wa-role of try-intake allows actions on every resource (policy app)
== Cost optimization
  HIGH  no budget: nobody is told when spending jumps
== Performance efficiency and sustainability
  info  function try-intake: 128 MB, x86_64
  info  function try-notify: 1024 MB, x86_64

A script will not answer the questions for you - "how do you know when a deployment is bad?" is a conversation - but it turns the review from "I think we have alarms" into "there are zero alarms". Run the same script after the fixes and save it as the milestone.

From findings to an improvement plan

Rank by risk x effort, give each an owner and a date, and fix the cheap high risks first. For the try- workload, with where this course taught the fix:

FindingPillarFixTaught
try-orders-in has no DLQreliabilityredrive policy + DLQ + an alarm on its depththis chapter, decoupling
no alarmsoperational excellencealarms on queue age, function errors and throttles, API 5XX; to a topic that pagesAWS III
failed async events droppedreliabilityon-failure destinationthis chapter, event sources
no budgetcosta monthly budget with forecast alerts, cost allocation tagsAWS III
bucket without versioningreliability, securityversioning, lifecycle for old versions, Block Public AccessAWS I, AWS V
x86 functionssustainability, costarm64 (Graviton), right-sized memorythis chapter, Lambda
function with sqs:* on *securityone role per function, scoped actions and resourcesthis chapter, security

Then the questions only people can answer, the ones reviews exist for:

  • Operational excellence: How do you deploy - and roll back? What does on-call see first when this breaks? When did you last run the runbook?
  • Security: Who can change production? How are secrets rotated? What would tell you about a leaked key?
  • Reliability: What are the RPO and RTO, who agreed them, and when was a restore last tested? What happens when the payment provider is down for an hour?
  • Performance: What is the expected peak, and has it been load-tested? Where is the bottleneck at 10x?
  • Cost: Who looks at this workload's bill, and how often? What is idle at night?
  • Sustainability: What data can be deleted or archived? Does anything run that nobody uses?

Making reviews stick

  • Review before launch and at least once a year, and after a major incident or redesign.
  • Keep it short: one workload, the people who build and run it, 2-4 hours, an improvement plan of the top 5 risks - not 60.
  • Turn findings into automation where you can: Config rules and conformance packs (AWS IV), Security Hub controls, Trusted Advisor checks, budgets, so the next review starts from a dashboard.
  • Pair with an architecture decision record for the trade-offs you accept on purpose ("one AZ for the dev environment, accepted").

On the exam (SAA-C03 all domains; SOA-C03 1.3 and 2.x; CLF-C02 1.1 "the AWS Well-Architected Framework"): the six pillars by name and what each covers (sustainability is the sixth, added in 2021), design principles such as "stop guessing capacity", "test recovery procedures", "apply security at all layers", "go global in minutes"; the Well-Architected Tool, lenses, HRI/MRI and milestones.

In an interview: "How would you review an existing AWS architecture you just inherited?" - "Walk it with the team pillar by pillar, using the Well-Architected questions but with evidence from the account: CloudTrail and Config for what changes and how, IAM Access Analyzer for who can do what, the alarms that exist, the backup and restore history, Compute Optimizer and Cost Explorer for waste. Then rank the risks by impact and effort, fix the cheap high risks first, record what we consciously accept, and repeat the review as a milestone."

You can now: name the six pillars and what each asks, run a review as a conversation backed by evidence from the account, classify findings as high or medium risk, turn them into a ranked improvement plan with owners, use the Well-Architected Tool's lenses and milestones, and keep reviews short and recurring.

Why it helps

Most real design problems are missing defaults, not exotic mistakes: no alarms, no DLQ, one Availability Zone, roles that allow everything, no budget. A structured review with evidence from the account finds them before an incident does. The six pillars of the Well-Architected Framework are also named directly on the Cloud Practitioner exam and underlie the reasoning of both Associate exams.

Commands in this lesson

cd sed bash

FAQ

What are the six pillars?

Operational excellence, security, reliability, performance efficiency, cost optimization and sustainability. Each has design principles and a set of questions with best practices; sustainability was added in 2021. A review walks a workload through all of them, not only the pillar a team happens to care about most.

What is a high risk issue?

A best practice whose absence could cause serious harm to the business, such as no backups that have been restored, no alarms, or public data. Medium risk issues matter but are less urgent. The improvement plan ranks them by impact and effort so the cheap high risks are fixed first.

What are lenses?

Extra sets of questions for a kind of workload, added to a review in the Well-Architected Tool: for example the Serverless, Containers, SaaS, Financial Services and Generative AI lenses. They make the general framework specific, such as asking about DLQs and concurrency for serverless applications.

Who should take part in a review?

The people who build and operate the workload, with someone who knows the framework guiding the conversation. It works best for one workload at a time, in a few hours, with access to the account so claims can be checked against evidence instead of memory.

How often should a workload be reviewed?

Before launch, then at least once a year, and after a major incident or redesign. Saving each review as a milestone lets the team compare progress, and turning recurring findings into automated checks such as Config rules, Security Hub controls and budgets makes the next review start from a dashboard.

In an interview Mid

How would you review an existing AWS architecture you just inherited?

Pillar by pillar with the team, using the Well-Architected questions but with evidence from the account: CloudTrail and Config for how things change, IAM Access Analyzer for who can do what, the alarms that exist, backup and restore history, Compute Optimizer and Cost Explorer for waste. I would record each high risk issue, rank them by impact and effort, fix the cheap high risks first, write down the trade-offs we accept on purpose, and repeat the review as a milestone.

Also asked: What are the six pillars of the Well-Architected Framework? · What makes a finding a high risk issue? · How do you keep a review from becoming a one-off exercise?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.