Route 53 routing policies and health checks: DNS that answers per query
AWS II pointed a name at a load balancer with an alias record. A DNS name can do more than hold one answer: Route 53 can answer differently per query. It can split traffic by weight, send each user to the closest Region, or stop returning an endpoint whose health check fails. That makes DNS a failover mechanism between Regions, with one big caveat: clients cache answers for the TTL, so DNS failover is never instant.
Need to know: records with a routing policy come in sets with the same name and type, each with a SetIdentifier: weighted (shares), latency (lowest latency Region for the resolver), failover (PRIMARY / SECONDARY), geolocation (by continent or country, with a default), geoproximity (by distance with a bias), multivalue answer (up to 8 healthy values), IP-based (by the resolver's CIDR). Health checks probe an endpoint from many locations (HTTP, HTTPS, TCP, string match), or combine others (calculated), or follow a CloudWatch alarm (for private endpoints). A record attached to a failing health check is left out of answers.
Two web servers, one name
The zone shop.oncall-lab.example has two servers in different subnets (web-a, web-b) and a health check on each:
$ Z=$(aws route53 list-hosted-zones-by-name --dns-name shop.oncall-lab.example --query 'HostedZones[0].Id' --output text)
$ aws route53 list-resource-record-sets --hosted-zone-id $Z --query "ResourceRecordSets[?Type=='A'].[Name,SetIdentifier,Failover,TTL,ResourceRecords[0].Value,HealthCheckId]" --output text
app.shop.oncall-lab.example. primary PRIMARY 60 3.66.184.122 94a3cab6-745d-48a6-a843-7913c91ddd11
app.shop.oncall-lab.example. secondary SECONDARY 60 3.75.97.100 None
$ aws route53 list-health-checks --query 'HealthChecks[].[Id,HealthCheckConfig.Type,HealthCheckConfig.IPAddress,HealthCheckConfig.ResourcePath,HealthCheckConfig.FailureThreshold,HealthCheckConfig.RequestInterval]' --output text
94a3cab6-745d-48a6-a843-7913c91ddd11 HTTP 3.66.184.122 /healthz 3 10
65a15379-eef2-472a-8470-88e62d2c6605 HTTP 3.75.97.100 /healthz 3 10
app.shop.oncall-lab.example is a failover pair: primary (web-a, with a health check) and secondary (web-b). Route 53 answers with the primary while its health check passes, and with the secondary otherwise. The health checker probes /healthz every 10 seconds and marks it unhealthy after 3 failures in a row: about 30 seconds to detect, plus the record's TTL (60 s) before every client has asked again.
$ aws route53 get-health-check-status --health-check-id $(aws route53 list-health-checks --query "HealthChecks[?HealthCheckConfig.IPAddress=='$(aws ec2 describe-instances --filters Name=tag:Name,Values=web-a Name=instance-state-name,Values=running --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)'].Id" --output text) --query 'HealthCheckObservations[0].[Region,StatusReport.Status]' --output text
us-east-1 Success: HTTP Status Code 200, OK
$ aws route53 test-dns-answer --hosted-zone-id $Z --record-name app.shop.oncall-lab.example --record-type A --query '[RecordData,ResponseCode]' --output text
NOERROR
3.66.184.122
$ dig +noall +answer app.shop.oncall-lab.example
app.shop.oncall-lab.example. 60 IN A 3.66.184.122
test-dns-answer asks Route 53 what it would answer, without any resolver cache in the way: the first thing to run when "DNS failover did not work".
A failover, end to end
$ aws ec2 stop-instances --instance-ids $(aws ec2 describe-instances --filters Name=tag:Name,Values=web-a Name=instance-state-name,Values=running --query 'Reservations[0].Instances[0].InstanceId' --output text) --query 'StoppingInstances[0].CurrentState.Name' --output text
stopping
$ sleep 45
$ aws route53 get-health-check-status --health-check-id $(aws route53 list-health-checks --query 'HealthChecks[?HealthCheckConfig.ResourcePath==`/healthz`]|[0].Id' --output text) --query 'HealthCheckObservations[0].StatusReport.Status' --output text
Failure: Connection timed out. The endpoint or the internet connection is down, or requests are being blocked by your firewall. See https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-router-firewall-rules.html
$ aws route53 test-dns-answer --hosted-zone-id $Z --record-name app.shop.oncall-lab.example --record-type A --query RecordData --output text
3.75.97.100
$ dig +short app.shop.oncall-lab.example
3.75.97.100
The checker saw the failure, and the answer is now web-b's address. In production the next minute matters. Resolvers and clients that cached the old answer keep using it until the TTL runs out, and some clients (old JVMs, connection pools, mobile apps) keep it much longer. Keep TTLs on failover records short (30-60 s), and make sure the secondary can take the full load. Failover moves traffic, not capacity.
Weighted and latency: splitting and steering
$ aws route53 change-resource-record-sets --hosted-zone-id $Z --change-batch file://$HOME/oncall-lab/labs/aws/try5/canary.json --query 'ChangeInfo.Status' --output text
PENDING
$ for i in $(seq 1 50); do dig +short new.shop.oncall-lab.example; done | sort | uniq -c
44 192.0.2.10
6 192.0.2.20
$ aws route53 change-resource-record-sets --hosted-zone-id $Z --change-batch '{"Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"x.shop.oncall-lab.example","Type":"A","Weight":10,"TTL":60,"ResourceRecords":[{"Value":"192.0.2.1"}]}}]}'
aws: [ERROR]: An error occurred (InvalidInput) when calling the ChangeResourceRecordSets operation: Invalid request: Missing field 'SetIdentifier' in Change with [Action=UPSERT, Name=x.shop.oncall-lab.example., Type=A, SetIdentifier=null]
The canary file sends weight 90 to the current version and 10 to the new one: about one answer in ten goes to the canary. A weight of 0 takes an endpoint out without deleting it. Every record in a routing set needs a SetIdentifier (the error says so). Latency records (one per Region, "Region": "eu-central-1") answer with the Region that has the lowest measured latency from the user's resolver, the basis of active-active multi-Region designs. Geolocation answers by where the user is (continent, country, US state). It is for compliance and localisation, not for performance, and it needs a default (CountryCode: "*") or users elsewhere get no answer. Multivalue answer returns up to eight healthy values: a simple client-side load spreading, not a load balancer.
Health checks for what the internet cannot reach
Route 53's health checkers run on the internet. A private endpoint (an internal ALB, an instance in a private subnet) cannot be probed directly. The pattern is a CloudWatch alarm on a metric that proves health (the ALB's HealthyHostCount, a canary's success metric) and a health check of type CLOUDWATCH_METRIC that follows the alarm. A calculated health check combines up to 256 others ("healthy when at least 2 of 3 are"). For alias records to AWS resources (load balancers, CloudFront, S3 websites), EvaluateTargetHealth: true uses the target's own health instead of a separate check: an ALB with no healthy targets makes the record unhealthy.
Other Route 53 parts on the exams:
- Private hosted zones answer only inside the VPCs associated with them (
enableDnsSupportandenableDnsHostnamesmust be on), and win over public zones with the same name there. - Route 53 Resolver endpoints connect VPC DNS with on-premises DNS: inbound endpoints let on-premises servers resolve private zones, outbound endpoints plus forwarding rules let instances resolve
corp.exampleagainst on-premises servers. Resolver DNS Firewall blocks lookups of bad domains from the VPC. - Query logging records the queries a public zone receives (to CloudWatch Logs in us-east-1); Resolver query logging records what the VPC asks.
- Route 53 Application Recovery Controller adds readiness checks and routing controls for Region failover that does not depend on health checks guessing.
On the exam (SOA-C03 5.2, SAA-C03 2.2 and 3.4): "active-passive DR across Regions" = failover routing + health check on the primary; "send users to the Region with the lowest latency" = latency routing; "blue/green or canary by percentage" = weighted; "serve content by country (compliance, language)" = geolocation (with a default record); "route by distance and shift traffic with a bias" = geoproximity; "several healthy IPs, no load balancer" = multivalue answer; "health check a private resource" = CloudWatch alarm health check. Aliases are free, work at the zone apex, and can evaluate target health.
In an interview: "DNS failover did not work during a Region outage. Where do you look?" - "Was the health check actually failing (get-health-check-status from its Regions; does it check a page that really depends on the backend)? What does Route 53 answer right now (test-dns-answer)? Then the caches: the record's TTL, resolvers that ignore it, clients and connection pools that never re-resolve. And whether the secondary could take the load, or quietly failed too."
You can now: read routing sets and health checks, explain failover timing (detection plus TTL plus client caching), use test-dns-answer and dig to see what Route 53 answers, split traffic with weights, and choose between failover, latency, geolocation, geoproximity, weighted and multivalue routing, including health checks for private endpoints.