OnCallReadyStart learning

AWS Lesson 37.19 · AWS IV: Operations Automation · 23 min read

EventBridge: rules, patterns, targets and the auto-remediation loop

In plain words

EventBridge is the building's alarm panel. Every door, window and machine sends a little message when something happens: this instance stopped, that setting changed, someone opened a port. You write rules that say which messages you care about, and where to send them - a pager, a log book, or a robot that fixes the problem.

The panel never complains if a bell refuses to ring. If the pager does not accept messages from the panel, the message is simply dropped, and only a counter on the panel goes up.

EventBridge: rules, patterns, targets and the auto-remediation loop

Everything in an AWS account produces events: an instance stops, an alarm changes state, a Config rule finds something, a backup job fails, and every write API call recorded by CloudTrail. Amazon EventBridge is the router: a rule on an event bus matches events with a pattern and sends each match to up to five targets - an SNS topic that pages you, a log group, an SQS queue, a Lambda function, a Systems Manager runbook that fixes the problem. Rules can also run on a schedule. That last hop - event in, runbook out - is how an account starts to fix itself.

Need to know: an event is JSON with a fixed envelope - source (aws.ec2), detail-type (EC2 Instance State-change Notification), account, region, time, resources, and a service-specific detail. A pattern is JSON of the same shape where every leaf is an array of allowed values (or matchers like prefix and anything-but); fields you leave out match anything. A target needs permission to receive: a resource policy on the target (SNS topics, SQS queues, Lambda functions) or a role EventBridge assumes (RoleArn: Systems Manager, Kinesis, Step Functions). A refused delivery is not an error anywhere you would see it - only the rule's FailedInvocations metric goes up.

Events and patterns

A real EC2 event and three patterns. test-event-pattern tells you whether a pattern matches, without creating anything - use it before every new rule:

$ cd ~/oncall-lab/labs/aws/try4
$ cat ev-stopped.json
{
  "version": "0",
  "id": "7bf73129-1428-4cd3-a780-95db273d1602",
  "detail-type": "EC2 Instance State-change Notification",
  "source": "aws.ec2",
  "account": "111122223333",
  "time": "2026-10-15T08:30:00Z",
  "region": "eu-central-1",
  "resources": [
    "arn:aws:ec2:eu-central-1:111122223333:instance/i-0abcd1234efgh5678"
  ],
  "detail": {
    "instance-id": "i-0abcd1234efgh5678",
    "state": "stopped"
  }
}
$ aws events test-event-pattern --event-pattern file://p-stopped.json --event file://ev-stopped.json
{
    "Result": true
}
$ cat p-not-running.json
{
  "source": [
    "aws.ec2"
  ],
  "detail": {
    "state": [
      {
        "anything-but": [
          "running",
          "pending"
        ]
      }
    ]
  }
}
$ aws events test-event-pattern --event-pattern file://p-not-running.json --event file://ev-stopped.json
{
    "Result": true
}

The matchers you will use: exact values (["stopped", "terminated"] - any of them), prefix, suffix, anything-but, numeric ([">", 80]), exists (true / false), wildcard, cidr for IP addresses, equals-ignore-case, and $or across fields. Matching is case-sensitive and exact on strings - "Stopped" never matches "stopped", and a number in the event never matches a string in the pattern.

Mistakes in the pattern itself are refused when you create the rule - the classic one is a value that is not an array:

$ aws events put-rule --name try-bad --event-pattern '{"source": "aws.ec2"}'
aws: [ERROR]: An error occurred (InvalidEventPatternException) when calling the PutRule operation: Event pattern is not valid. Reason: "source" must be an object or an array at [Source: (String)"{"source":"aws.ec2"}"; line: 1, column: 11]

A rule that pages, and the delivery nobody sees fail

Page the team when an instance stops: a rule with the pattern, the topic as target, then stop the instance and look at the inbox (pager.log is the lab's stand-in for the email subscription):

$ aws events put-rule --name try-instance-stopped --event-pattern file://p-stopped.json --query RuleArn --output text
arn:aws:events:eu-central-1:111122223333:rule/try-instance-stopped
$ aws events put-targets --rule try-instance-stopped --targets Id=page,Arn=arn:aws:sns:eu-central-1:111122223333:try-alerts
{
    "FailedEntryCount": 0,
    "FailedEntries": []
}
$ I=$(aws ec2 describe-instances --filters Name=tag:Name,Values=try-web-1 Name=instance-state-name,Values=running --query 'Reservations[0].Instances[0].InstanceId' --output text)
$ aws ec2 stop-instances --instance-ids $I --query 'StoppingInstances[0].CurrentState.Name' --output text
stopping
$ sleep 60
$ grep -c 'EC2 Instance State-change' ../pager.log
0
$ aws cloudwatch get-metric-statistics --namespace AWS/Events --metric-name FailedInvocations --dimensions Name=RuleName,Value=try-instance-stopped --start-time $(date -u -d '-10 min' +%FT%TZ) --end-time $(date -u +%FT%TZ) --period 600 --statistics Sum --query 'Datapoints[0].Sum'
1

put-targets said "FailedEntryCount": 0, the rule matched, and nothing arrived. The topic's default access policy lets only the account's own principals publish; EventBridge is a service principal and needs a statement of its own (with aws:SourceArn, so only your rules qualify):

$ cat topic-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowEventBridge",
      "Effect": "Allow",
      "Principal": {
        "Service": "events.amazonaws.com"
      },
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:eu-central-1:111122223333:try-alerts",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:events:eu-central-1:111122223333:rule/try-*"
        }
      }
    }
  ]
}
$ aws sns set-topic-attributes --topic-arn arn:aws:sns:eu-central-1:111122223333:try-alerts --attribute-name Policy --attribute-value file://topic-policy.json
$ aws ec2 start-instances --instance-ids $I >/dev/null
$ sleep 60
$ aws ec2 stop-instances --instance-ids $I >/dev/null
$ sleep 60
$ grep -c 'EC2 Instance State-change' ../pager.log
1

The same rule applies to every resource-policy target: an SQS queue needs a queue policy, a Lambda function a resource-based permission (lambda add-permission --principal events.amazonaws.com). When a rule "does nothing", check in this order: does test-event-pattern match a real event, is the rule ENABLED and on the right bus and Region, did MatchedEvents / Invocations go up, did FailedInvocations go up (then it is the target's permission).

Pages a person can read

By default the target gets the whole event JSON. An InputTransformer picks fields (InputPathsMap) and builds the message (InputTemplate):

$ aws events put-targets --rule try-instance-stopped --targets '[{"Id":"page","Arn":"arn:aws:sns:eu-central-1:111122223333:try-alerts","InputTransformer":{"InputPathsMap":{"id":"$.detail.instance-id","state":"$.detail.state","at":"$.time"},"InputTemplate":"\"<id> is <state> since <at> - check whether this was planned.\""}}]' --query FailedEntryCount
0
$ aws ec2 start-instances --instance-ids $I >/dev/null
$ sleep 60
$ aws ec2 stop-instances --instance-ids $I >/dev/null
$ sleep 60
$ tail -3 ../pager.log
Subject: (no subject)

i-0e51940f154311175 is stopped since 2026-09-22T20:04:20Z - check whether this was planned.

The loop: an API call fixes itself

CloudTrail sends every write API call to EventBridge as detail-type: "AWS API Call via CloudTrail", with the CloudTrail record as detail. So a rule can react to what someone did: here, anyone opening a security group. The target is a Systems Manager runbook that removes SSH from the group again - the role EventBridge uses (try-events-ssm, trusting events.amazonaws.com) may start the runbook and pass the runbook's own role (try-remediate, trusting ssm.amazonaws.com, allowed to revoke rules). The whole loop first:

$ cat p-ssh.json
{
  "source": [
    "aws.ec2"
  ],
  "detail-type": [
    "AWS API Call via CloudTrail"
  ],
  "detail": {
    "eventSource": [
      "ec2.amazonaws.com"
    ],
    "eventName": [
      "AuthorizeSecurityGroupIngress"
    ]
  }
}
$ aws events put-rule --name try-close-ssh --event-pattern file://p-ssh.json --query RuleArn --output text
arn:aws:events:eu-central-1:111122223333:rule/try-close-ssh
$ aws events put-targets --rule try-close-ssh --targets '[{"Id":"fix","Arn":"arn:aws:ssm:eu-central-1:111122223333:automation-definition/AWS-DisablePublicAccessForSecurityGroup:$DEFAULT","RoleArn":"arn:aws:iam::111122223333:role/try-events-ssm","InputTransformer":{"InputPathsMap":{"sg":"$.detail.requestParameters.groupId"},"InputTemplate":"{\"GroupId\":[<sg>],\"AutomationAssumeRole\":[\"arn:aws:iam::111122223333:role/try-remediate\"]}"}}]' --query FailedEntryCount
0
$ SG=$(aws ec2 describe-instances --instance-ids $I --query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
$ aws ec2 authorize-security-group-ingress --group-id $SG --protocol tcp --port 22 --cidr 0.0.0.0/0 --query 'SecurityGroupRules[0].[FromPort,CidrIpv4]' --output text; sleep 30
22	0.0.0.0/0
$ aws ssm describe-automation-executions --filters Key=DocumentNamePrefix,Values=AWS-DisablePublicAccessForSecurityGroup --query 'AutomationExecutionMetadataList[0].[AutomationExecutionStatus,ExecutedBy]' --output text
Success	arn:aws:sts::111122223333:assumed-role/try-events-ssm/events
$ aws ec2 describe-security-groups --group-ids $SG --query 'SecurityGroups[0].IpPermissions[?FromPort==`22`]' --output json
[]

Thirty seconds from "someone opened SSH to the world" to "it is closed again", with the who and when in CloudTrail. The same shape covers most auto-remediation: GuardDuty finding -> isolate the instance; Config rule NON_COMPLIANT -> runbook; alarm -> restart. Two warnings that come with it: a remediation that fights another tool (Terraform puts the rule back, the runbook removes it again, forever - an incident later in this chapter), and a rule whose target triggers the same rule again (a Lambda that writes to the bucket that triggers it). Scope patterns tightly.

Schedules: rules and EventBridge Scheduler

A rule with --schedule-expression 'rate(5 minutes)' or 'cron(0 2 * * ? *)' fires on its own, in UTC. EventBridge Scheduler is the newer, separate service for that job and is better at it: millions of schedules, a time zone per schedule, one-time at() schedules that delete themselves, a flexible window, retries and a dead-letter queue, and universal targets - any API action (arn:aws:scheduler:::aws-sdk:ec2:stopInstances) without writing a function. Stop the dev instance at 20:00 Berlin time on weekdays:

$ aws scheduler create-schedule --name try-stop-dev-evening --schedule-expression 'cron(0 20 ? * MON-FRI *)' --schedule-expression-timezone Europe/Berlin --flexible-time-window Mode=OFF --target "{\"Arn\":\"arn:aws:scheduler:::aws-sdk:ec2:stopInstances\",\"RoleArn\":\"arn:aws:iam::111122223333:role/try-scheduler\",\"Input\":\"{\\\"InstanceIds\\\":[\\\"$I\\\"]}\"}" --query ScheduleArn --output text
arn:aws:scheduler:eu-central-1:111122223333:schedule/default/try-stop-dev-evening
$ aws scheduler get-schedule --name try-stop-dev-evening --query '[ScheduleExpression,ScheduleExpressionTimezone,State,Target.Arn]' --output text
cron(0 20 ? * MON-FRI *)	Europe/Berlin	ENABLED	arn:aws:scheduler:::aws-sdk:ec2:stopInstances
$ aws scheduler create-schedule --name try-once --schedule-expression 'rate(1 hour)' --flexible-time-window Mode=OFF --target Arn=arn:aws:sns:eu-central-1:111122223333:try-alerts
aws: [ERROR]: An error occurred (ValidationException) when calling the CreateSchedule operation: 1 validation error detected: Value null at 'target.roleArn' failed to satisfy constraint: Member must not be null

The role is not optional for Scheduler: every target is called with a role that trusts scheduler.amazonaws.com. The old "cron rule in UTC that runs an hour late half the year" is the reason teams move schedules to Scheduler.

The rest of the picture

  • Buses: the default bus gets AWS service events; custom buses take your own events (put-events with your own source - an aws. source is refused); a bus policy can accept events from other accounts, which is how an organization sends every account's findings to one security account. Partner buses receive SaaS events.
  • Reliability: EventBridge retries a failed delivery for up to 24 hours (185 attempts by default) and can send what still fails to a dead-letter queue (an SQS queue on the target); an archive keeps events so you can replay them after fixing a target.
  • S3 Event Notifications are the older, bucket-level way to react to object events (to SNS, SQS or Lambda, one configuration per event type and prefix); turning on "send to EventBridge" on the bucket lets rules filter on keys, sizes and buckets with patterns instead.

On the exam (SOA-C03 1.2 and 3.2, SAA-C03 2.1): "use EventBridge to route, enrich, and deliver events, and troubleshoot event bus rules": patterns, targets and their permissions (the SNS topic policy is the favourite trap), InputTransformer, FailedInvocations, scheduled rules vs EventBridge Scheduler, and event-driven remediation (EventBridge -> SSM Automation or Lambda). SAA asks the decoupling side: EventBridge vs SNS vs SQS.

In an interview: "Your EventBridge rule matches but the SNS topic never gets anything. Why?" - "The topic's access policy doesn't allow events.amazonaws.com to sns:Publish. EventBridge delivers as a service principal, so resource-policy targets - SNS, SQS, Lambda - need a statement for it. The rule's FailedInvocations metric shows it; I'd add the statement with an aws:SourceArn condition for the rule, then test with a real event."

You can now: read an event's envelope, write patterns and test them before creating a rule, route matches to SNS with the right topic policy, find a silent delivery failure in FailedInvocations, shape the message with an InputTransformer, react to API calls through CloudTrail events, close the loop with an SSM runbook and two roles, schedule work in a time zone with EventBridge Scheduler, and place buses, DLQs, archives and S3 notifications.

Why it helps

EventBridge is how AWS reacts: instance state changes, alarm transitions, Config findings, GuardDuty findings and every write API call recorded by CloudTrail arrive as events that rules can match. The lesson covers the event envelope, writing and testing patterns, target permissions - the SNS topic policy that silently refuses deliveries - InputTransformer for readable pages, the full auto-remediation loop from an API call to a runbook with two roles, and EventBridge Scheduler with time zones and universal targets. It is the wiring behind everything automatic in the rest of the chapter.

Commands in this lesson

cd cat aws sleep grep tail

FAQ

Why does my rule match in test-event-pattern but never fire?

Check the rule is ENABLED, on the right event bus and in the Region where the events happen. Then compare the pattern with a real event, not an example from the documentation: field names, string case and number versus string types must match exactly. MatchedEvents in CloudWatch shows whether it matched at all.

How do I know a delivery failed?

The AWS/Events FailedInvocations metric for the rule goes up, and nothing else tells you. The usual cause is the target's permission: a topic, queue or function policy that does not allow events.amazonaws.com, or a RoleArn that EventBridge cannot assume. A dead-letter queue on the target keeps the failed events for inspection.

Can a rule react to something someone did in the console?

Yes. CloudTrail sends every write management API call to EventBridge as detail-type AWS API Call via CloudTrail, with the full record in detail - who, what, which parameters. Console actions are API calls too. Read-only calls and data events need their own trail settings and are not sent by default.

What is the difference between EventBridge and SNS?

SNS is a topic that fans one message out to subscribers. EventBridge routes events by content: rules with patterns choose which events go to which targets, from AWS services, other accounts and SaaS partners, with schedules, archives and replay. They are often used together: an EventBridge rule targeting an SNS topic.

Why use EventBridge Scheduler instead of a scheduled rule?

Scheduler supports a time zone per schedule, so local times survive daylight saving, one-time at() schedules that can delete themselves, retries and dead-letter queues, flexible windows, and universal targets that call an API directly. Scheduled rules only run in UTC and are meant for a few periodic jobs.

In an interview Mid

Your EventBridge rule matches, but the SNS topic never receives anything. Why?

The topic's access policy does not allow events.amazonaws.com to sns:Publish. EventBridge delivers as a service principal, so resource-policy targets - SNS, SQS, Lambda - need a statement for it, ideally with an aws:SourceArn condition for the rule. Nothing errors where you would look: the rule's FailedInvocations metric is the signal. I'd add the statement and then test with a real event, not just test-event-pattern.

Also asked: How would you automatically close SSH opened to the world on a security group? · What is an InputTransformer used for? · When would you use EventBridge Scheduler instead of a cron rule?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.