Lesson 12.14 · Terraform: Language & Workflow · 27 min read

Functions: the ones you use every day

Q: What is the difference between try and coalesce?

try(a, b) returns the first argument that evaluates without an error; null is a valid value, so try(null, "x") is null. coalesce(a, b) returns the first argument that is not null and not an empty string, so coalesce(null, "", "x") is "x". Use try for structure that might be missing (an optional attribute), coalesce for values that might be empty.

Q: Why does toset reorder my list?

A set has no order and no duplicates, so toset(["b", "a", "b"]) becomes the set of "a" and "b", displayed sorted. You use toset mainly because foreach accepts a set of strings but not a list. If the list's order carried meaning (priority, for example), converting to a set throws that away; in that case build a map with explicit keys instead.

Q: Can file() read a file that a resource creates during apply?

No. file() and templatefile() read at plan time from disk, relative to where you point them, so a file generated by a resource in the same run does not exist yet. Use path.module so the path works wherever the module is called from. For content produced during apply, pass the resource attribute directly instead of writing and re-reading a file.

Q: Which Terraform functions do you use most, and for what?

The everyday ones, all testable in terraform console: - merge(a, b) - combine maps, later arguments win: layering tags (merge(local.commontags, var.extratags)). - lookup(map, key, default) - a safe map index: per-environment sizes. - cidrsubnet(prefix, newbits, netnum) / cidrsubnets - carve subnets without overlaps: cidrsubnet("10.20.0.0/16", 8, 3) is 10.20.3.0/24. cidrhost picks an address in a subnet. - toset - turn a list into the set of strings foreach accepts. - flatten - turn nested lists into one, for nested foreach. - try, coalesce, one - missing values: try returns the first expression that does not error, coalesce the first not null or empty, one(x[].id) reads a conditional count resource. - format, lower, replace, substr - names that follow the cloud's naming rules. - jsonencode, templatefile, file - generated settings and start-up scripts. There are no user-defined functions; locals and modules are how you reuse logic.

In plain words

Imagine a toolbox where every tool is already made for you and you are not allowed to build new ones. There is a label maker, a scissors, a tape measure, a tool that glues two lists together. You get very good by knowing which tool to grab, not by inventing tools.

Terraform has about 120 built-in functions and no user-defined ones. The daily tools are lookup and merge for maps, concat, flatten and toset for lists, try, can, coalesce and one for maybe-missing values, format, replace and regex for Azure naming rules, cidrsubnet and cidrhost for carving VNets, and jsonencode and templatefile for building content. Every one of them can be tried in terraform console.

Why this matters

What you need to know already: expressions and the console (12.11), types (12.8), CIDR and subnetting by hand (8.3, 8.6), regular expressions (7.3), JSON (7.11).

Cloud names have strict rules (lowercase, no dashes, max 24 characters), subnets must be carved without overlaps, and tags must be layered from several sources. Doing that by hand in every file is how typos reach production. Terraform's built-in functions do it in one line - name(arguments), like calling a JS function.

Built-in functions, and only those

Terraform has about 120 built-in functions and no user-defined functions. You cannot write function slugify(x) in HCL. What you have instead: locals for reuse, modules for encapsulation, and (since 1.8) provider-defined functions called as provider::<name>::<function>(...) - for example the azurerm provider's provider::azurerm::parse_resource_id(id). Everything else in this lesson is built in, and every example runs in terraform console.

The families, as the docs group them:

numeric          abs ceil floor max min pow log signum parseint
string           format formatlist join split replace regex regexall lower upper
                 title trim trimspace trimprefix trimsuffix substr strrev
                 startswith endswith strcontains chomp indent
collection       length element index lookup merge concat flatten distinct
                 keys values contains zipmap setproduct setunion setintersection
                 setsubtract slice reverse sort range compact coalesce
                 coalescelist one chunklist transpose matchkeys alltrue anytrue sum
encoding         jsonencode jsondecode yamlencode yamldecode base64encode
                 base64decode urlencode csvdecode
filesystem       file fileexists templatefile filebase64 abspath dirname
                 basename pathexpand fileset
date and time    timestamp timeadd timecmp formatdate plantimestamp
hash and crypto  md5 sha1 sha256 sha512 bcrypt uuid filemd5 filesha256
ip network       cidrhost cidrnetmask cidrsubnet cidrsubnets
type conversion  tostring tonumber tobool tolist toset tomap try can
                 sensitive nonsensitive type (console only)

You will not memorise all of them. You should know the ones below without looking, because they are in every real configuration and every exam.

Looking things up

The map values below are Azure VM sizes (B2s small, D4s_v5 bigger - think "t-shirt sizes" for rented machines).

> lookup({ dev = "B2s", prod = "D4s_v5" }, "prod", "B1s")
"D4s_v5"
> lookup({ dev = "B2s", prod = "D4s_v5" }, "test", "B1s")
"B1s"
> lookup({ dev = "B2s" }, "test")
╷
│ Error: Error in function call
│
│ Call to function "lookup" failed: lookup failed to find key "test".
╵

lookup(map, key, default) is the safe index. With a default it never fails; without one it is just map[key] with a worse error message. Plain indexing fails the same way:

> { dev = "B2s" }["test"]
╷
│ Error: Invalid index
│
│ The given key does not identify an element in this collection value.
╵

element(list, i) wraps around (element(["a","b"], 3) is "b") - handy for spreading copies over availability zones (separate data centres inside one region): zone = element(var.zones, count.index). index(list, value) is the reverse: the position of a value, or an error.

Combining maps and lists

> merge({ env = "dev", owner = "platform" }, { owner = "orders", cost = "42" })
{
  "cost" = "42"
  "env" = "dev"
  "owner" = "orders"
}

merge - later arguments win. That is the standard way to layer tags:

tags = merge(local.common_tags, var.extra_tags, { component = "api" })

> concat(["a", "b"], ["c"])
[
  "a",
  "b",
  "c",
]
> flatten([["a", "b"], [], ["c", ["d"]]])
[
  "a",
  "b",
  "c",
  "d",
]
> distinct(["a", "b", "a"])
tolist([
  "a",
  "b",
])
> zipmap(["web", "api"], [443, 8080])
{
  "api" = 8080
  "web" = 443
}
> setproduct(["dev", "prod"], ["weu", "neu"])
[
  [
    "dev",
    "weu",
  ],
  [
    "dev",
    "neu",
  ],
  ...
]

flatten is the one that unlocks nested for_each (12.20, 12.22). setproduct is every combination - environments times regions, for example.

keys and values of a map come back in lexical key order, which matters when you zip them back together:

> keys({ web = 1, api = 2 })
[
  "api",
  "web",
]

When a value might be missing: try, can, coalesce, one

> try({ a = 1 }.b, "fallback")
"fallback"
> try(null, "fallback")
null
> coalesce(null, "", "fallback")
"fallback"
> can(regex("^st[a-z0-9]{3,22}$", "stOrders"))
false
> one([])
null
> one(["only"])
"only"

The distinction the exam tests and people get wrong:

try(a, b) returns the first argument that evaluates without an error. null is not an error, so try(null, "x") is null.
coalesce(a, b) returns the first argument that is not null and not an empty string.
can(expr) is true/false for "does this evaluate" - use it inside validation conditions.
one(list) turns a zero-or-one element list into the element or null - the tidy way to read the single instance of a conditional count resource (12.20): one(azurerm_public_ip.bastion[*].id) (a public IP address that may or may not exist).

try hides real mistakes if you overuse it. Use it for genuinely optional structure (an attribute a caller may omit), not to paper over typos.

Strings

> format("vm-%s-%03d", "web", 7)
"vm-web-007"
> formatlist("%s.internal.corp", ["web", "api"])
tolist([
  "web.internal.corp",
  "api.internal.corp",
])
> join(",", ["10.0.1.4", "10.0.1.5"])
"10.0.1.4,10.0.1.5"
> split(",", "a,b,c")
tolist([
  "a",
  "b",
  "c",
])
> replace("rg_orders_dev", "_", "-")
"rg-orders-dev"
> replace("stOrders-Dev", "/[^a-z0-9]/", "")
"strdersev"
> lower(replace("stOrders-Dev", "-", ""))
"stordersdev"
> regex("^rg-([a-z]+)-([a-z]+)$", "rg-orders-dev")
[
  "orders",
  "dev",
]
> trimprefix("rg-orders", "rg-")
"orders"
> substr("oncall-lab", 0, 5)
"sysop"

replace treats its pattern as a regular expression when it is wrapped in slashes - "/[^a-z0-9]/" - otherwise as a literal. format uses Go verbs: %s string, %d integer, %03d zero-padded, %-10s left-aligned, %q quoted, %.2f two decimals (the same as bash printf, 1.7). Azure naming rules make these daily tools: storage accounts are 3-24 lowercase letters and digits, key vaults 3-24 with dashes, and so on.

IP network functions - the ones you will use in every VNet

cidrsubnet(prefix, newbits, netnum) carves a subnet out of a range:

newbits is how many bits to add to the prefix length: a /16 plus 8 is a /24.
netnum is which of the resulting subnets: 0 is the first.

> cidrsubnet("10.20.0.0/16", 8, 0)
"10.20.0.0/24"
> cidrsubnet("10.20.0.0/16", 8, 3)
"10.20.3.0/24"
> cidrsubnet("10.20.0.0/16", 4, 1)
"10.20.16.0/20"
> cidrsubnet("10.20.0.0/16", 6, 2)
"10.20.8.0/22"

Do the last one by hand once. A /16 plus 6 bits is a /22; a /22 is 1024 addresses = 4 in the third octet. Subnet 0 is 10.20.0.0, subnet 1 is 10.20.4.0, subnet 2 is 10.20.8.0/22. The rule of thumb: the step between consecutive subnets is 2^(32 - new prefix length) addresses.

Asking for a subnet number that does not fit is an error, not a wrap-around:

> cidrsubnet("10.20.0.0/24", 2, 4)
╷
│ Error: Error in function call
│
│ Call to function "cidrsubnet" failed: prefix extension of 2 does not
│ accommodate a subnet numbered 4.
╵

cidrsubnets(prefix, newbits...) allocates several consecutive subnets of different sizes without overlaps, aligning each to its own size:

> cidrsubnets("10.20.0.0/16", 6, 8, 8, 10)
tolist([
  "10.20.0.0/22",
  "10.20.4.0/24",
  "10.20.5.0/24",
  "10.20.6.0/26",
])

That is the function for "a big /22 for a pool of machines, an app subnet and a data subnet (/24 each) and a small management /26" in one line. Order the largest first to waste the least space - alignment of a big subnet after small ones leaves a gap.

> cidrhost("10.20.1.0/24", 4)
"10.20.1.4"
> cidrhost("10.20.1.0/24", -1)
"10.20.1.255"
> cidrnetmask("10.20.1.0/24")
"255.255.255.0"

Azure reserves the first four and the last address of every subnet (.0 network, .1 gateway, .2 and .3 Azure DNS, .255 broadcast), so the first usable static IP in a /24 is cidrhost(cidr, 4) and a /24 has 251 usable addresses. Ch 8 (8.6) did that maths; here it becomes code.

Encoding and files

azurerm_linux_web_app is Azure's hosting for a web application (you give it code or an image, Azure runs it); app_settings become the app's environment variables.

locals {
  settings = jsondecode(file("${path.module}/settings.json"))
}

resource "azurerm_linux_web_app" "api" {
  # ...
  app_settings = {
    FEATURE_FLAGS = jsonencode({ newCheckout = true, betaSearch = false })
  }
}

file(path) reads a file at plan time from the configuration directory. It cannot read a file a resource creates during the same apply.
jsonencode produces minified JSON; use it for policy documents and app settings rather than hand-writing JSON strings with escaped quotes.
templatefile(path, vars) renders a template file (12.11).
base64encode is what VM custom_data wants.

Type conversion

> tostring(5)
"5"
> tonumber("5")
5
> tonumber("five")
╷
│ Error: Error in function call
│
│ Call to function "tonumber" failed: Invalid value for "v" parameter: cannot
│ convert "five" to number; given string must be a decimal representation of a
│ number.
╵
> toset(["b", "a", "b"])
toset([
  "a",
  "b",
])

toset is the conversion you use most - for_each (12.20) accepts a set of strings but not a list. It removes duplicates and sorts; if the order of a list mattered, that information is gone.

Reading an expression you did not write

locals {
  app_subnet_ids = [for k, s in azurerm_subnet.this : s.id if startswith(k, "app")]
  first_app_ip   = cidrhost(one([for k, s in var.subnets : s.cidr if k == "app"]), 4)
  vm_names       = formatlist("vm-%s-%02d", var.env, range(1, var.vm_count + 1))
}

Work outward: identify each function's input type, then its output type. The first is a list of IDs of subnets whose key starts with "app". The second finds the single subnet keyed "app" and takes its first usable IP. The third is ["vm-dev-01", "vm-dev-02", ...]. If you cannot tell, paste pieces into the console until you can.

What you can now do:

Look up, merge and reshape maps and lists (lookup, merge, flatten, toset).
Handle missing values with try, coalesce, can, one.
Carve subnets with cidrsubnet / cidrsubnets and check them in the console.

Why it helps

Functions are where Azure's rules meet code. A storage account name must be 3-24 lowercase letters and digits, so lower(replace(...)) plus a can(regex(...)) validation is standard. Carving a /22, two /24s and a /26 out of a network without overlaps is one cidrsubnets call, and you will review subnet plans where someone got newbits wrong. The try vs coalesce distinction (null is not an error) is a favourite exam trick and a real bug in modules. When a reviewer writes "use one(x[*]) instead of x[0]", this lesson is why a disabled feature flag stops crashing the plan.

FAQ

What is the difference between try and coalesce?

try(a, b) returns the first argument that evaluates without an error; null is a valid value, so try(null, "x") is null. coalesce(a, b) returns the first argument that is not null and not an empty string, so coalesce(null, "", "x") is "x". Use try for structure that might be missing (an optional attribute), coalesce for values that might be empty.

How do I read cidrsubnet("10.20.0.0/16", 6, 2)?

The second argument is how many bits to add to the prefix: /16 plus 6 is a /22. The third is which of those subnets: 0 is the first. A /22 is 1024 addresses, which is 4 in the third octet, so subnet 0 is 10.20.0.0, 1 is 10.20.4.0 and 2 is 10.20.8.0/22. A netnum that does not fit is an error, not a wrap-around.

Can I write my own function in HCL?

No. Terraform has no user-defined functions. You reuse logic with locals (a named expression) and modules (a reusable group of resources with inputs and outputs). Since Terraform 1.8, providers can ship functions called as provider::azurerm::parse_resource_id(id), which covers some cases like parsing Azure IDs. If you need genuinely complex logic, that is often a sign the input data should be prepared differently.

Why does toset reorder my list?

A set has no order and no duplicates, so toset(["b", "a", "b"]) becomes the set of "a" and "b", displayed sorted. You use toset mainly because for_each accepts a set of strings but not a list. If the list's order carried meaning (priority, for example), converting to a set throws that away; in that case build a map with explicit keys instead.

Can file() read a file that a resource creates during apply?

No. file() and templatefile() read at plan time from disk, relative to where you point them, so a file generated by a resource in the same run does not exist yet. Use path.module so the path works wherever the module is called from. For content produced during apply, pass the resource attribute directly instead of writing and re-reading a file.

In an interview Junior

Which Terraform functions do you use most, and for what?

The everyday ones, all testable in terraform console:

merge(a, b) - combine maps, later arguments win: layering tags (merge(local.common_tags, var.extra_tags)).
lookup(map, key, default) - a safe map index: per-environment sizes.
cidrsubnet(prefix, newbits, netnum) / cidrsubnets - carve subnets without overlaps: cidrsubnet("10.20.0.0/16", 8, 3) is 10.20.3.0/24. cidrhost picks an address in a subnet.
toset - turn a list into the set of strings for_each accepts.
flatten - turn nested lists into one, for nested for_each.
try, coalesce, one - missing values: try returns the first expression that does not error, coalesce the first not null or empty, one(x[*].id) reads a conditional count resource.
format, lower, replace, substr - names that follow the cloud's naming rules.
jsonencode, templatefile, file - generated settings and start-up scripts.

There are no user-defined functions; locals and modules are how you reuse logic.

Also asked: What is the difference between try and coalesce? · How would you split a /16 into subnets of different sizes in Terraform? · How do you safely reference a resource created with count = var.enabled ? 1 : 0?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.