OnCallReady

Lesson 12.11 · Terraform: Language & Workflow · 32 min read

Expressions: references, operators, for, splat and templates

In plain words

An expression is like a sentence you say to a calculator that also understands lists. "Take every name on this list and shout it" is a for expression: [for s in var.names : upper(s)]. "Give me the phone number of everyone in the class" is a splat: people[*].phone. "Write 'Hello' plus the name" is a template: "Hello, ${var.name}".

Terraform expressions compute values from references (var.env, local.prefix, azurerm_subnet.app.id), operators, conditionals (cond ? a : b), for expressions, splats and string templates. terraform console is where you try them, the same way you would use the browser devtools console to test a JavaScript expression before putting it in code.

Why this matters

What you need to know already: variables, locals and outputs (12.5), types (12.8), CIDR (8.3), heredocs (6.18), JS template literals and Array.map from your frontend work.

Real configurations are not lists of literal values. Names are built from the environment, sizes depend on dev vs prod, one list of subnets becomes many resources. Expressions are the small bits of logic that compute a value - the right-hand side of every name = ... line.

The console is your REPL

A REPL (read-eval-print loop) is a prompt where you type code and see the result at once, like the browser's JS console. Every expression in this lesson can be tried in terraform console. It loads the configuration in the current directory - variables, locals, and resource attributes from state - and evaluates whatever you type:

$ terraform console
> 1 + 2 * 3
7
> "rg-${upper("dev")}"
"rg-DEV"
> exit

> is the console's prompt; the line under it is the result. exit (or Ctrl+D) leaves. It also reads standard input (a pipe, 1.7), which is how you script it and check an answer quickly:

$ echo 'cidrsubnet("10.20.0.0/16", 8, 3)' | terraform console
"10.20.3.0/24"

(cidrsubnet cuts a smaller network out of a bigger one - 12.14.) Use it constantly. Writing an expression into main.tf, running a plan and reading a diff is a slow way to discover that split returns a list.

References

A reference names a value defined somewhere else. The full list - you will meet each one in this chapter or the next two:

var.env                              an input variable
local.prefix                         a local value
azurerm_subnet.app                   a resource - the whole object
azurerm_subnet.app.id                one attribute of it
azurerm_subnet.app[0].id             an instance of a count resource
azurerm_subnet.app["web"].id         an instance of a for_each resource
data.azurerm_client_config.me.tenant_id   a data source (12.18)
module.network.vnet_id               an output of a child module (Ch 13)
path.module                          directory of the module this expression is in
path.root                            directory of the root module
path.cwd                             where terraform was run from
terraform.workspace                  the current workspace name (Ch 14)
count.index  each.key  each.value    inside count / for_each (12.20)
self                                 inside postconditions and provisioners

[0] and ["web"] pick one copy of a resource that was created several times (12.20). path.module is the one to reach for when reading files that sit next to your code: file("${path.module}/scripts/init.sh") works wherever the module is called from; file("scripts/init.sh") only works from one directory.

Operators

The usual maths and logic, with this order of evaluation (precedence):

precedence (highest first)
  !   - (unary)        logical not, negation
  *   /   %            arithmetic
  +   -                arithmetic
  >   >=   <   <=      numeric comparison
  ==  !=               equality (any types)
  &&                   logical and
  ||                   logical or
> 1 + 2 * 3
7
> (1 + 2) * 3
9
> 10 / 4
2.5
> 10 % 4
2
> 2 > 1 && "a" == "a"
true

Things that surprise people coming from JavaScript:

> "rg-" + "dev"
╷
│ Error: Invalid operand
│
│ Unsuitable value for left operand: a number is required.
╵

+ is arithmetic only. Strings are joined with interpolation ("rg-${var.env}") or join/format. And == never converts: 1 == "1" is false, because a number and a string are different values - even though a number variable would happily accept "1" as input.

&& and || require bools on both sides. Up to Terraform 1.11 both sides are always evaluated, so var.cfg != null && var.cfg.enabled still fails when var.cfg is null - use try(var.cfg.enabled, false) instead. Terraform 1.12 added short-circuiting (the exam version); the lab runs 1.9, so write it the portable way.

Conditional expressions

cond ? a : b means "if cond then a else b" - the same ternary as in JS.

sku        = var.env == "prod" ? "Premium" : "Standard"
node_count = var.env == "prod" ? 3 : 1
count      = var.enable_monitoring ? 1 : 0

Both results must have the same type (or be convertible to one). cond ? "a" : 1 becomes a string; cond ? ["a"] : "a" is an error: "Inconsistent conditional result types".

A very common idiom: a value that is set only in some environments.

zones = var.env == "prod" ? ["1", "2", "3"] : null

null for an argument means "not set" - the provider uses its default, exactly as if the line were not there.

for expressions

A for expression transforms one collection into another - like JS array.map(). Square brackets make a tuple (list), braces make an object (map).

> [for s in ["web", "api"] : upper(s)]
[
  "WEB",
  "API",
]

> { for s in ["web", "api"] : s => length(s) }
{
  "api" = 3
  "web" = 3
}

Over a map you get the key and the value; over a list, the index and the value:

> { for name, cidr in { web = "10.0.1.0/24", db = "10.0.2.0/24" } : name => cidrhost(cidr, 4) }
{
  "db" = "10.0.2.4"
  "web" = "10.0.1.4"
}

> [for i, s in ["a", "b"] : "${i}:${s}"]
[
  "0:a",
  "1:b",
]

An if clause filters:

> [for s in var.subnets : s.name if s.public]

(var.subnets here is a list of objects with name and public.) And ... after the value switches on grouping, for when several items produce the same key:

> { for p in ["web:1", "api:2", "web:3"] : split(":", p)[0] => split(":", p)[1]... }
{
  "api" = [
    "2",
  ]
  "web" = [
    "1",
    "3",
  ]
}

Without the ..., duplicate keys are an error: "Two different items produced the key "web" in this 'for' expression."

Maps are always iterated in lexical key order, regardless of how you wrote them. That is why the outputs above are sorted.

Splat expressions

A splat [*] is shorthand for "this attribute of every element":

> [{ name = "web", port = 443 }, { name = "api", port = 8080 }][*].name
[
  "web",
  "api",
]

It is the same as [for o in list : o.name]. The common real use:

output "subnet_ids" {
  value = azurerm_subnet.app[*].id          # a count resource
}

Two details:

The older .* form (azurerm_subnet.app.*.id) still works but behaves differently on some edge cases; prefer [*].

Strings and templates

name  = "rg-${var.project}-${var.env}"        # interpolation
note  = "a literal $${not_interpolated}"      # $${ escapes
path  = "C:\\temp\\file"                      # backslash escapes: \n \t \" \\

A string that is only one interpolation is not a string conversion - it yields the value itself. "${var.subnets}" is the list, not a string. (It is also redundant; write var.subnets.)

Heredocs for multi-line strings, the same idea as bash heredocs (6.18). This one is a start-up script for a virtual machine (custom_data, which Azure wants base64-encoded - base64 is a way to write any bytes as plain letters):

custom_data = base64encode(<<-EOT
  #!/bin/bash
  echo "env=${var.env}" > /etc/app.conf
  systemctl restart app
EOT
)

<<EOT keeps the text exactly; <<-EOT removes the indentation common to all lines, so the heredoc can be indented with the code around it. EOT is convention - any identifier works, as long as the closing line matches.

Template directives put logic (loops, ifs) inside a string. This builds an /etc/hosts-style file (8.16) from a map of name to IP:

locals {
  hosts = <<-EOT
    %{ for name, ip in var.hosts ~}
    ${ip} ${name}
    %{ endfor ~}
  EOT

  greeting = "Hello, %{ if var.name != "" }${var.name}%{ else }stranger%{ endif }!"
}

Templates longer than a few lines belong in a file. (cloud-init is the standard program that runs such a start-up file on a new Linux VM.)

custom_data = base64encode(templatefile("${path.module}/cloud-init.yaml.tftpl", {
  env   = var.env
  hosts = var.hosts
}))

templatefile evaluates the file as a template with only the variables you pass - no var., no local.. The .tftpl extension is the convention.

dynamic blocks

Nested blocks (like security_rule or blob_properties) are not values, so you cannot put a for expression in them. dynamic generates them. The example is a network security group (Azure's list of firewall rules, 12.8) with one security_rule per entry of var.rules:

resource "azurerm_network_security_group" "app" {
  name                = "nsg-app"
  location            = var.location
  resource_group_name = azurerm_resource_group.main.name

  dynamic "security_rule" {
    for_each = var.rules
    content {
      name                       = security_rule.key
      priority                   = security_rule.value.priority
      direction                  = "Inbound"
      access                     = security_rule.value.access
      protocol                   = "Tcp"
      destination_port_range     = security_rule.value.port
      source_address_prefix      = "*"
      source_port_range          = "*"
      destination_address_prefix = "*"
    }
  }
}

for_each walks the map; content { } is the block to write for each entry. The iterator is named after the block (security_rule.key, security_rule.value) unless you set iterator = rule. Use dynamic sparingly: it makes a resource harder to read, and a block that is always present should just be written out.

Putting expressions together

A realistic local that you should be able to read at a glance by the end of this chapter:

locals {
  # every (vnet, subnet) pair as a flat map, keyed "vnet/subnet"
  subnets = merge([
    for vnet_name, vnet in var.vnets : {
      for sub_name, cidr in vnet.subnets :
      "${vnet_name}/${sub_name}" => {
        vnet = vnet_name
        name = sub_name
        cidr = cidr
      }
    }
  ]...)
}

Read it from the inside out: for each subnet of each VNet, build an object keyed vnet/subnet; the outer for produces a list of such maps; merge(...) with ... expands that list into separate arguments and merges them into one map. (merge(a, b, ...) combines maps into one.) That map can feed a for_each directly - the flattening pattern from 12.20 and 12.22.

What you can now do:

Why it helps

Real configurations are full of expressions like a merge([for ...]...) that flattens networks into a subnet map. In PR review you need to read those at a glance and tell whether the output shape is right, or you are approving blind. The JavaScript habits you bring are exactly the ones that bite: + does not join strings, == never converts types, and before Terraform 1.12 && does not short-circuit, so var.cfg != null && var.cfg.enabled still fails on null. dynamic blocks come up when generating firewall rules (an NSG's security_rule blocks). The exam includes reading for and splat output, and terraform console with a pipe (echo 'expr' | terraform console) is the fastest way to check yourself.

Commands in this lesson

terraform echo

FAQ

Why does "rg-" + var.env fail?

In HCL, + is arithmetic only, so Terraform tries to turn "rg-" into a number and fails with "a number is required". Strings are joined with interpolation, "rg-${var.env}", or with join and format. Related trap: == never converts, so 1 == "1" is false even though a number variable would happily accept "1" as input.

When do I get a list and when do I get a map from a for expression?

Square brackets produce a tuple (list-like): [for s in list : upper(s)]. Braces produce an object (map-like) and need a key => value: { for s in list : s => length(s) }. Duplicate keys in the brace form are an error unless you add ... after the value to group them into lists. Maps are always iterated in lexical key order, whatever order you wrote them in.

Can I use splat on a for_each resource?

Not directly. A for_each resource is a map of instance objects, and [*] works on lists (count resources). Use values(azurerm_subnet.s)[*].id for a list of IDs, [for s in azurerm_subnet.s : s.id], or { for k, s in azurerm_subnet.s : k => s.id } to keep the keys. On a single value, [*] wraps it into a one-element list and turns null into an empty list.

What is the difference between <<EOT and <<-EOT?

<<EOT keeps the text exactly, including indentation. <<-EOT strips the indentation common to all lines, so the heredoc can be indented along with the surrounding code. Inside either, ${...} interpolates and %{ for } / %{ if } directives add logic, with ~ trimming whitespace next to them. Longer templates belong in a .tftpl file rendered with templatefile, which only sees the variables you pass it.

When should I use a dynamic block?

When a nested block (like security_rule or ip_rules) must repeat based on input. Nested blocks are not values, so a for expression cannot generate them; dynamic "security_rule" { for_each = var.rules content { ... } } can. The iterator is named after the block (security_rule.value) unless you set iterator. Use it sparingly: a block that is always present should just be written out, because dynamic makes resources harder to read.

In an interview Junior

Explain for expressions and splat expressions, and when you would use each.

A for expression transforms one collection into another, like JavaScript's array.map(); square brackets make a list, braces make a map, and an if filters:

[for s in var.subnets : s.name if s.public]
{ for name, cidr in var.subnets : name => cidrhost(cidr, 4) }

A splat [*] is shorthand for "this attribute of every element": azurerm_subnet.app[*].id is the same as [for s in azurerm_subnet.app : s.id].

Use a splat for the simple case on a list (a count resource). Use a for expression when you filter, reshape, build a map, or work on a for_each resource (which is a map: values(azurerm_subnet.app)[*].id or a for).

Try either in terraform console before writing it down. And for JavaScript habits: + is arithmetic only - build strings with interpolation ("rg-${var.env}") - and == never converts types.

Also asked: How do you build strings in Terraform? · What does a dynamic block do? · What is the difference between templatefile and interpolation?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.