OnCallReadyStart learning

AWS Lesson 38.35 · AWS V: Data, Resilience & the Edge · 12 min read

AWS WAF and Shield: filtering requests before they reach you

In plain words

A security group is a doorman who checks which street you came from and lets you through or not. AWS WAF is a doorman who also reads the letter you are carrying: is there something nasty written in it, like an SQL injection, and have you knocked 500 times in the last minute? Then it allows you in, blocks you, just counts you, or asks you to prove you are human.

Shield is the crowd barrier outside, which stops a stampede from flattening the door before anyone gets to the doorman.

AWS WAF and Shield: filtering requests before they reach you

A security group decides which addresses and ports may connect. It cannot tell a normal search from an SQL injection in the query string, or a customer from a bot sending 500 requests a minute to the login page. AWS WAF inspects HTTP requests at CloudFront, an ALB, API Gateway (and a few others) and allows, blocks, counts or challenges them by rules. AWS Shield protects against DDoS at the network layer. This lesson puts a web ACL in front of the shop's CloudFront distribution.

Need to know: a web ACL has rules evaluated by priority and a default action. Rules use statements (IP sets, geo match, string / regex match, SQLi and XSS detection, size limits) or managed rule groups (AWS's, or Marketplace sellers'). Rate-based rules block a key (an IP, a header, a combination) that exceeds N requests in a window. A web ACL's scope is CLOUDFRONT (created in us-east-1) or REGIONAL (ALB, API Gateway, in their Region). Count mode lets you see what a rule would block before it blocks.

The rules

$ aws wafv2 list-available-managed-rule-groups --scope CLOUDFRONT --region us-east-1 --query 'ManagedRuleGroups[].Name' --output text
AWSManagedRulesCommonRuleSet	AWSManagedRulesKnownBadInputsRuleSet	AWSManagedRulesSQLiRuleSet	AWSManagedRulesAmazonIpReputationList	AWSManagedRulesAnonymousIpList
$ cat ~/oncall-lab/labs/aws/try5/rules.json | jq '.[] | {name: .Name, priority: .Priority, statement: (.Statement | keys[0]), action: (.Action // .OverrideAction)}'
{
  "name": "sqli",
  "priority": 0,
  "statement": "ManagedRuleGroupStatement",
  "action": {
    "None": {}
  }
}
{
  "name": "bad-inputs",
  "priority": 1,
  "statement": "ManagedRuleGroupStatement",
  "action": {
    "None": {}
  }
}
{
  "name": "per-ip",
  "priority": 2,
  "statement": "RateBasedStatement",
  "action": {
    "Block": {}
  }
}
$ aws wafv2 create-web-acl --name try-edge --scope CLOUDFRONT --default-action Allow={} --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=try-edge
aws: [ERROR]: An error occurred (WAFInvalidParameterException) when calling the CreateWebACL operation: Error reason: The scope is not valid., field: SCOPE_VALUE, parameter: CLOUDFRONT
$ ACL=$(aws wafv2 create-web-acl --region us-east-1 --name try-edge --scope CLOUDFRONT --default-action Allow={} --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=try-edge --rules file://$HOME/oncall-lab/labs/aws/try5/rules.json --query Summary.ARN --output text)
$ echo $ACL
arn:aws:wafv2:us-east-1:111122223333:global/webacl/try-edge/eea99743-4339-4548-9471-3cbf499d114a

The first create-web-acl was refused: a CLOUDFRONT web ACL lives in us-east-1, like CloudFront's certificates. The rules: AWS's SQL injection rule group (with OverrideAction: None, which keeps the group's own block actions; Count would turn them all into counts), the known bad inputs group (exploit patterns such as Log4Shell), and a rate-based rule: more than 10 requests from one IP in 60 seconds (a lab-sized number; production limits are in the hundreds or thousands) and that IP is blocked until it slows down.

Attaching it, and watching it work

$ D=$(aws cloudfront list-distributions --query "DistributionList.Items[?Comment=='shop assets'].Id" --output text)
$ DOM=$(aws cloudfront get-distribution --id $D --query Distribution.DomainName --output text)
$ aws cloudfront get-distribution-config --id $D > /tmp/cfg.json && E=$(jq -r .ETag /tmp/cfg.json) && jq --arg a "$ACL" '.DistributionConfig | .WebACLId = $a' /tmp/cfg.json > /tmp/new.json && aws cloudfront update-distribution --id $D --distribution-config file:///tmp/new.json --if-match $E --query 'Distribution.[Status,DistributionConfig.WebACLId]' --output text
InProgress	arn:aws:wafv2:us-east-1:111122223333:global/webacl/try-edge/eea99743-4339-4548-9471-3cbf499d114a
$ curl -s -o /dev/null -w '%{http_code}\n' "https://$DOM/index.html?id=1%27%20OR%20%271%27=%271"
403
$ for i in $(seq 1 14); do curl -s -o /dev/null -w '%{http_code} ' https://$DOM/index.html; done; echo
200 200 200 200 200 200 200 200 200 200 403 403 403 403
$ curl -s https://$DOM/index.html | head -3
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>

A web ACL is attached to CloudFront through the distribution (WebACLId in its config, with the If-Match ETag of the current config). An ALB or API Gateway uses aws wafv2 associate-web-acl --resource-arn. The injection was blocked by the managed group; after ten normal requests the rate rule took over. Blocked requests get 403 from CloudFront ("Request blocked."), or a custom response you define.

$ aws wafv2 get-sampled-requests --web-acl-arn $ACL --rule-metric-name per-ip --scope CLOUDFRONT --region us-east-1 --time-window StartTime=$(date -u -d '-10 min' +%s),EndTime=$(date -u -d '+1 min' +%s) --max-items 3 --query 'SampledRequests[].[Request.ClientIP,Request.URI,Action]' --output text
198.51.100.24	/index.html	BLOCK
198.51.100.24	/index.html	BLOCK
198.51.100.24	/index.html	BLOCK
$ aws wafv2 get-rate-based-statement-managed-keys --scope CLOUDFRONT --region us-east-1 --web-acl-name try-edge --web-acl-id ${ACL##*/} --rule-name per-ip --query 'ManagedKeysIPV4.Addresses' --output text
198.51.100.24/32

Sampled requests show what each rule matched (up to three hours back). The managed keys of a rate-based rule are the addresses it is blocking right now: here, the lab box itself. For the full picture, WAF logs (to CloudWatch Logs, S3 or Firehose; log group names must start with aws-waf-logs-) and the CloudWatch metrics AllowedRequests, BlockedRequests, CountedRequests per rule.

Shield, Firewall Manager and the rest

  • Shield Standard is on for everyone at no cost: it absorbs common layer 3/4 attacks (SYN floods, UDP reflection) at CloudFront, Route 53 and the Global Accelerator edge.
  • Shield Advanced (a monthly fee per organization) adds protection and visibility for ELB, CloudFront, Global Accelerator, Route 53 and Elastic IPs, the Shield Response Team, WAF at no extra cost for protected resources, and cost protection (credits for scaling caused by an attack).
  • Firewall Manager applies WAF rules, Shield Advanced, security groups and Network Firewall policies across every account of an organization.
  • AWS Network Firewall is a stateful firewall inside the VPC (domain lists, Suricata rules), for traffic that never passes CloudFront or an ALB.
  • Layering: CloudFront + WAF at the edge, an ALB that only accepts CloudFront (its managed prefix list in the ALB's security group, or a secret header), security groups and NACLs inside.

On the exam (SAA-C03 1.2, SOA-C03 4.2 and 5.1, CLF-C02 2.4): "SQL injection / cross-site scripting" = WAF (managed rule groups); "block an IP range or a country at the application layer" = WAF IP set / geo match (or CloudFront geo restriction); "throttle abusive clients" = rate-based rule; "DDoS protection with a response team and cost protection" = Shield Advanced; "manage WAF across accounts" = Firewall Manager. WAF attaches to CloudFront, ALB, API Gateway, AppSync, Cognito user pools, App Runner and Verified Access, never to an NLB or an EC2 instance.

In an interview: "Your login page gets a credential-stuffing attack. What do you put in place?"

  • "At the edge: a WAF rate-based rule scoped to the login path, keyed by IP (or IP plus a header),

the IP reputation and anonymous-IP managed groups, and a CAPTCHA or challenge action for suspicious traffic, started in Count mode to check false positives. In the app: account lockouts with backoff, MFA, and alarms on failed-login rates. Then the WAF logs to find the sources and tune the rules."

You can now: build a web ACL with managed rule groups and a rate-based rule in the right scope and Region, attach it to a CloudFront distribution (and know how ALBs differ), read sampled requests and a rate rule's blocked keys, roll rules out in Count mode first, and place Shield, Firewall Manager and Network Firewall around it.

Why it helps

Network rules cannot tell a normal search from an SQL injection, or a customer from a bot hammering the login page. WAF works on the HTTP request itself, at CloudFront, an ALB and a few other services.

It can also break checkout when a managed rule matches legitimate traffic, which is why rules go in with Count first. The lesson builds a web ACL in the right scope and Region, attaches it, and reads what it blocked and why, which is what you need on the day it blocks the wrong thing.

Commands in this lesson

aws cat echo curl

FAQ

Why was my create-web-acl for CloudFront refused?

A web ACL with scope CLOUDFRONT must be created in us-east-1, like CloudFront's certificates; add --region us-east-1. A REGIONAL web ACL, for an ALB or other regional resources, lives in that resource's Region. The scope is fixed when you create the web ACL.

How do I attach a web ACL?

To CloudFront, through the distribution: set WebACLId in the distribution config and call update-distribution with the If-Match ETag of the current config. An ALB uses aws wafv2 associate-web-acl --resource-arn instead. WAF never attaches to an NLB or directly to an EC2 instance.

What do OverrideAction None and Count mean for a rule group?

They apply to managed rule groups. None keeps the group's own actions, so its block rules block. Count turns every rule in the group into a count: requests are only labelled and counted, which lets you see in sampled requests and logs what would be blocked before you enforce it.

How do I see what WAF blocked?

get-sampled-requests shows what a rule matched, up to three hours back. For a rate-based rule, get-rate-based-statement-managed-keys lists the addresses it is blocking right now. For the full picture, send WAF logs to CloudWatch Logs (group names must start with aws-waf-logs-), S3 or Firehose, and watch BlockedRequests and CountedRequests per rule.

What does Shield Advanced add to Shield Standard?

Shield Standard is on for everyone at no cost and absorbs common layer 3 and 4 attacks such as SYN floods and UDP reflection. Shield Advanced, a monthly fee per organization, adds protection and visibility for ELB, CloudFront, Global Accelerator, Route 53 and Elastic IPs, the Shield Response Team, WAF at no extra cost for protected resources, and cost protection.

In an interview Mid

Your login page gets a credential-stuffing attack. What do you put in place?

At the edge, a WAF rate-based rule scoped to the login path and keyed by IP, or IP plus a header, together with the IP reputation and anonymous-IP managed rule groups, and a CAPTCHA or challenge action for suspicious traffic. I start the new rules in Count mode to check for false positives, then enforce. In the application: account lockouts with backoff, MFA, and alarms on the failed-login rate. Then I read the WAF logs and get-sampled-requests to find the sources and tune the rules.

Also asked: What is the difference between a security group and AWS WAF? · How would you roll out a new managed rule group without breaking customers? · What does Shield Advanced give you over Shield Standard?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.