OnCallReadyStart learning

AWS Lesson 37.1 · AWS IV: Operations Automation · 22 min read

CloudFormation: templates, stacks and events

In plain words

A CloudFormation template is a shopping list for AWS: these buckets, this topic, that role, with these settings. You hand the list over, and AWS builds everything on it as one parcel called a stack, writing down each step as it goes. If any item cannot be bought, it puts everything back on the shelves.

Later you change the list and hand it over again, and AWS works out what to add, change or remove. The stack remembers what it built, so nobody needs to keep a separate note of it.

CloudFormation: templates, stacks and events

AWS II built the shop's network with Terraform; most teams that run AWS also meet the other tool for the same job, the one AWS ships itself. AWS CloudFormation takes a template - a YAML or JSON file that lists resources and their properties - and turns it into a stack: one unit that creates, updates and deletes those resources together, records every step as an event, and puts everything back the way it was when a step fails. Landing zones, vendor products, AWS's own quick starts, the CDK and SAM all end up as CloudFormation stacks, so even a Terraform shop reads stack events during incidents.

Need to know: a template has Resources (required: a logical ID, a Type, Properties) and optional Parameters (inputs), Mappings (lookup tables), Conditions (create this only in prod), and Outputs (values to show or export to other stacks). Intrinsic functions wire them together: !Ref, !GetAtt, !Sub, !FindInMap, !If, !Equals, !ImportValue. create-stack returns at once; the work happens afterwards and the stack's status (CREATE_IN_PROGRESS -> CREATE_COMPLETE) and events tell you how it went. There is no state file to keep: the stack is the state, stored by AWS.

A template, read top to bottom

The lab's starter template sets up what a small service needs - a log group, a topic for its alerts (in prod only), and a parameter that tells the app where to log:

$ cd ~/oncall-lab/labs/aws/try4
$ cat app.yaml
AWSTemplateFormatVersion: '2010-09-09'
Description: try - the shop's settings, logs and alerts
Parameters:
  Env:
    Type: String
    AllowedValues: [dev, prod]
    Default: dev
Mappings:
  Retention:
    dev: { Days: 7 }
    prod: { Days: 30 }
Conditions:
  IsProd: !Equals [!Ref Env, prod]
Resources:
  Logs:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub '/try/${Env}/app'
      RetentionInDays: !FindInMap [Retention, !Ref Env, Days]
  Alerts:
    Type: AWS::SNS::Topic
    Condition: IsProd
    Properties:
      TopicName: !Sub 'try-${Env}-alerts'
  Setting:
    Type: AWS::SSM::Parameter
    Properties:
      Name: !Sub '/try/${Env}/log-group'
      Type: String
      Value: !Ref Logs
Outputs:
  LogGroup:
    Value: !Ref Logs
    Export:
      Name: !Sub '${AWS::StackName}-LogGroup'

Section by section:

  • Parameters - Env has AllowedValues: CloudFormation refuses test before it creates anything. Other parameter types validate more: AWS::EC2::VPC::Id must be a real VPC, AWS::SSM::Parameter::Value<String> reads Parameter Store at deploy time, NoEcho: true hides the value in every describe call;
  • Mappings - a two-level lookup table, read with !FindInMap [Retention, !Ref Env, Days]. The classic use is an AMI per Region; here, retention per environment;
  • Conditions - IsProd is true when Env is prod; a resource with Condition: IsProd is simply not part of the stack otherwise (!If [IsProd, a, b] picks a property value instead);
  • Resources - each has a logical ID (Logs) you use inside the template, and gets a physical ID (/try/prod/app) when it is made. !Ref Logs returns the physical ID (for a log group, its name); !GetAtt Logs.Arn returns an attribute;
  • Outputs - values describe-stacks shows, and with Export a name other stacks in the same account and Region can !ImportValue.

The order of the resources in the file means nothing. CloudFormation builds a dependency graph from !Ref / !GetAtt / !Sub (and explicit DependsOn) and creates what it can in parallel.

Validate, create, wait

validate-template checks the syntax and the references - not whether the properties are valid for the service:

$ aws cloudformation validate-template --template-body file://app.yaml
{
    "Parameters": [
        {
            "ParameterKey": "Env",
            "DefaultValue": "dev",
            "NoEcho": false
        }
    ],
    "Description": "try - the shop's settings, logs and alerts"
}
$ aws cloudformation create-stack --stack-name try-app --template-body file://app.yaml --parameters ParameterKey=Env,ParameterValue=prod
{
    "StackId": "arn:aws:cloudformation:eu-central-1:111122223333:stack/try-app/19639a63-bbe4-4af2-9683-0ab28217a1b6"
}
$ aws cloudformation describe-stacks --stack-name try-app --query 'Stacks[0].StackStatus' --output text
CREATE_IN_PROGRESS

create-stack answers with the stack ID and nothing else - the stack is still being built. In a script, wait for it: the waiter polls every 30 seconds and exits 255 when the stack lands in a failure state, so && after it is safe:

$ aws cloudformation wait stack-create-complete --stack-name try-app && echo created
created

Events: the stack's own log

Every step of every resource is an event. Newest first, as the API returns them - reverse() turns them into a timeline:

$ aws cloudformation describe-stack-events --stack-name try-app --query 'reverse(StackEvents)[].[LogicalResourceId,ResourceStatus,ResourceStatusReason]' --output text
try-app	CREATE_IN_PROGRESS	User Initiated
Logs	CREATE_IN_PROGRESS	None
Alerts	CREATE_IN_PROGRESS	None
Logs	CREATE_IN_PROGRESS	Resource creation Initiated
Alerts	CREATE_IN_PROGRESS	Resource creation Initiated
Logs	CREATE_COMPLETE	None
Setting	CREATE_IN_PROGRESS	None
Setting	CREATE_IN_PROGRESS	Resource creation Initiated
Alerts	CREATE_COMPLETE	None
Setting	CREATE_COMPLETE	None
try-app	CREATE_COMPLETE	None

Read it like a build log: Logs and Alerts start together (nothing links them), Setting waits for Logs because its value is !Ref Logs, and the stack itself is the first and the last line. None is how the text output prints an empty reason. When something fails, the ResourceStatusReason of the first CREATE_FAILED event is the cause; everything after it is CloudFormation cleaning up - the next lesson but one is about exactly that.

What the stack made, and what it reports:

$ aws cloudformation describe-stack-resources --stack-name try-app --query 'StackResources[].[LogicalResourceId,ResourceType,PhysicalResourceId]' --output table
---------------------------------------------------------------------------------------------
|                                  DescribeStackResources                                   |
+---------+-----------------------+---------------------------------------------------------+
|  Logs   |  AWS::Logs::LogGroup  |  /try/prod/app                                          |
|  Alerts |  AWS::SNS::Topic      |  arn:aws:sns:eu-central-1:111122223333:try-prod-alerts  |
|  Setting|  AWS::SSM::Parameter  |  /try/prod/log-group                                    |
+---------+-----------------------+---------------------------------------------------------+
$ aws cloudformation describe-stacks --stack-name try-app --query 'Stacks[0].Outputs' --output table
----------------------------------------------------
|                  DescribeStacks                  |
+------------------+-------------+-----------------+
|    ExportName    |  OutputKey  |   OutputValue   |
+------------------+-------------+-----------------+
|  try-app-LogGroup|  LogGroup   |  /try/prod/app  |
+------------------+-------------+-----------------+
$ aws ssm get-parameter --name /try/prod/log-group --query Parameter.Value --output text
/try/prod/app

The resources are ordinary AWS resources - aws logs, aws sns, aws ssm see them like anything else - with three tags CloudFormation adds where the service supports tags: aws:cloudformation:stack-name, aws:cloudformation:logical-id and aws:cloudformation:stack-id. In the console, "which stack made this?" is that tag.

The setup also left a second stack from the same template, made with the default Env:

Errors before anything is created

Two classes of errors stop a stack before it starts. Parameters are checked against the template:

$ aws cloudformation create-stack --stack-name try-app2 --template-body file://app.yaml --parameters ParameterKey=Env,ParameterValue=test
aws: [ERROR]: An error occurred (ValidationError) when calling the CreateStack operation: Parameter 'Env' must be one of AllowedValues

And references are checked by validate-template (and again by create-stack): a typo in a logical ID is a template format error, which is the cheap kind - it costs nothing:

$ sed 's/Value: !Ref Logs$/Value: !Ref Log/' app.yaml > typo.yaml
$ aws cloudformation validate-template --template-body file://typo.yaml
aws: [ERROR]: An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unresolved resource dependencies [Log] in the Resources block of the template

Everything else - a property the service rejects, a name that is taken, a permission you do not have - only shows up while the stack is being built, as a CREATE_FAILED event. Validation is not a dry run.

Capabilities: CloudFormation asks before it touches IAM

A template that creates IAM roles, users or policies can give itself (or anyone) more access than you have reviewed, so CloudFormation wants an explicit acknowledgement:

$ aws cloudformation validate-template --template-body file://role.yaml --query '[Capabilities,CapabilitiesReason]'
[
    [
        "CAPABILITY_NAMED_IAM"
    ],
    "The following resource(s) require capabilities: [AWS::IAM::Role]"
]
$ aws cloudformation create-stack --stack-name try-role --template-body file://role.yaml
aws: [ERROR]: An error occurred (InsufficientCapabilitiesException) when calling the CreateStack operation: Requires capabilities : [CAPABILITY_NAMED_IAM]

CAPABILITY_IAM for IAM resources with generated names, CAPABILITY_NAMED_IAM when you name them yourself (as here: RoleName), CAPABILITY_AUTO_EXPAND for macros and transforms (SAM). Pipelines pass the capability explicitly; a person reading the change should see why.

Deleting a stack

Deleting a stack deletes its resources, in reverse dependency order, and leaves a DELETE_COMPLETE stack in list-stacks for 90 days:

$ aws cloudformation delete-stack --stack-name try-app && aws cloudformation wait stack-delete-complete --stack-name try-app && echo deleted
deleted
$ aws cloudformation describe-stacks --stack-name try-app
aws: [ERROR]: An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id try-app does not exist
$ aws cloudformation list-stacks --stack-status-filter DELETE_COMPLETE --query 'StackSummaries[?StackName==`try-app`].[StackName,DeletionTime]' --output text
try-app	2026-09-22T20:00:41.800000+00:00

describe-stacks by name only finds live stacks; the deleted one is still there by its stack ID (the ARN). A bucket that still has objects, or an export another stack imports, stops the delete (DELETE_FAILED) - the change sets lesson shows how to keep data on purpose with DeletionPolicy.

CloudFormation and Terraform

Both are declarative: you describe the result, the tool works out the steps. The differences that matter on call:

CloudFormationTerraform
statekept by AWS, per stacka state file you store and lock (S3 + lockfile)
previewchange setsterraform plan
failure mid-wayautomatic rollback to the last good statestops; what was created stays, state records it
scopeAWS (plus registry extensions)any provider: AWS, Kubernetes, GitHub, DNS...
driftdetect-stack-drift on demandevery plan compares with reality
many accounts / RegionsStackSetsa provider per account/Region, or a wrapper
new AWS featuresusually on launch daywhen the provider adds them

Neither is "the right one"; mixing them on the same resource is wrong. Whatever tool made a resource owns it - a change made outside it (the console, a script, another tool) is drift, and the next run of the owner either reverts it or trips over it.

On the exam (SOA-C03 3.1, SAA-C03 3.x): "create and manage resources by using CloudFormation": the template sections, Ref vs GetAtt, parameters with AllowedValues and AWS-specific types, mappings for per-Region AMIs, conditions for per-environment resources, outputs and exports, CAPABILITY_NAMED_IAM, and that a stack's resources are created in dependency order, not file order. The AWS CDK writes the same templates from code (cdk synth prints the template, cdk deploy creates the stack).

In an interview: "What is a CloudFormation stack and how do you find out why one failed?" - "A stack is the set of resources one template made, managed as a unit: created in dependency order, updated together, rolled back together when a step fails. To find the cause I read describe-stack-events oldest first and look for the first CREATE_FAILED or UPDATE_FAILED: its ResourceStatusReason is the service's own error. Everything after it is the rollback."

You can now: read a template's parameters, mappings, conditions, resources and outputs, validate and create a stack, wait for it, read its events as a timeline, find what it made and what it exports, acknowledge IAM capabilities, delete it, and say where CloudFormation and Terraform differ.

Why it helps

CloudFormation is the infrastructure as code AWS ships itself, and much of what runs in real accounts was built by it - landing zones, vendor products, CDK and SAM applications. Reading a template (parameters, mappings, conditions, resources, outputs) and a stack's event log is a basic on-call skill even in a Terraform shop. The lesson also shows the cheap failures that stop a stack before anything exists - template format errors, AllowedValues and capabilities - and how CloudFormation and Terraform differ in state, previews and rollback.

Commands in this lesson

cd cat aws sed

FAQ

What is the difference between a logical ID and a physical ID?

The logical ID is the name inside the template, such as Logs, that other resources reference. The physical ID is what AWS created, such as the log group /try/prod/app or an instance ID. Ref returns the physical ID for most types; describe-stack-resources lists both side by side.

Does validate-template prove that the stack will create?

No. It checks YAML or JSON syntax, the structure, and that every Ref, GetAtt and Sub points at something that exists. It does not ask the services whether a name is taken, a property is valid or you have the permissions. Those failures appear during creation as CREATE_FAILED events.

Why does CloudFormation want CAPABILITY_NAMED_IAM?

A template that creates IAM roles or policies can grant itself, or anyone, more access than the person deploying it reviewed. CloudFormation therefore wants an explicit acknowledgement: CAPABILITY_IAM for IAM resources with generated names, CAPABILITY_NAMED_IAM when the template names them, CAPABILITY_AUTO_EXPAND for macros and transforms.

In which order are resources created?

In dependency order, not file order. CloudFormation builds a graph from Ref, GetAtt, Sub and explicit DependsOn, creates independent resources in parallel, and waits where one needs another. Deletion runs the graph backwards, which is why a stack that exports a value cannot be deleted while another stack imports it.

What happens to a stack that failed to create?

CloudFormation rolls back by deleting what it made and leaves the stack in ROLLBACK_COMPLETE, which cannot be updated. Read the first CREATE_FAILED event for the cause, delete the stack, fix the template and create it again. While debugging, --on-failure DO_NOTHING keeps the failed resources to inspect.

In an interview Mid

What is a CloudFormation stack, and how do you find out why one failed?

A stack is the set of resources one template made, managed as a unit: created in dependency order, updated together and rolled back together when a step fails. To find the cause I read describe-stack-events oldest first - reverse(StackEvents) - and look for the first CREATE_FAILED or UPDATE_FAILED whose reason is not "cancelled": its ResourceStatusReason is the service's own error. Everything after it is the rollback. A template format error or a missing capability fails earlier, at create-stack.

Also asked: What is the difference between Ref and GetAtt in a template? · How do parameters, mappings and conditions differ in a template? · When would you choose CloudFormation over Terraform?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.