OnCallReadyStart learning

Infrastructure as code Lesson 16.2 · Terraform in Real Life & the Associate Exam · 24 min read

The platform module: network, identity, cluster (AKS), Key Vault

In plain words

Think of a starter kit for opening a new shop: the building plot (a resource group), the road and car park (a VNet and a subnet), a fence with gates (an NSG), a manager who carries a badge instead of a key (a managed identity), the shop itself, and a safe for valuables (Key Vault). You order the same kit for the test shop and the real shop, only in different sizes.

The platform module is that kit. The "shop" is a cluster: a group of machines that run containers for you, which Azure manages when you ask for one azurerm_kubernetes_cluster resource. The module wires the pieces together: the identity may manage only its own subnet, the cluster waits for that permission with depends_on, and the vault is private. envs/dev asks for one small machine, envs/prod for three bigger ones spread over zones.

The problem: a platform that every environment gets the same way

In Chapters 12-13 you ran containers on one machine with Docker. Real teams run hundreds of containers on many machines, and they do not want to manage those machines by hand. They use a cluster: a group of machines that run containers for you - it decides which machine runs which container, restarts ones that die, and gives them network addresses.

AKS (Azure's managed container-cluster service) is Azure's managed cluster. You ask for one with a single Terraform resource, azurerm_kubernetes_cluster, and Azure runs the management side for you. You do not need to know how the cluster works inside for this lesson - treat it as one more Azure resource with arguments.

What you do need is everything around it: a network for its machines, a firewall, a login identity with the right permissions, a place for logs, and a safe for secrets. Built by hand, that is an afternoon of clicking and easy to get subtly different between dev and prod. Built as one module called from envs/dev and envs/prod (lesson 16.1), every environment gets the same platform, only sized differently.

This lesson is the design of that module. The mission after it builds it. The Terraform ideas it practises: module composition, hidden dependencies, identity wiring, outputs, and per-environment sizing.

What you need to know already: containers (12.3), subnets as address ranges and /16 vs /22 (10.3, 10.6), cidrsubnet (14.14), references and depends_on (14.18), validation blocks (14.8), sensitive outputs (14.5), modules and their files (15.34), Azure login methods incl. managed identity (14.3), the environments layout (16.1).

Later (Ch 17): you will learn what runs inside a cluster - pods, nodes, the API server. This chapter only builds the box it runs in.

The pieces, in plain words

pieceAzure resourcewhat it is
resource groupazurerm_resource_groupa named folder that holds related Azure resources
virtual network (VNet)azurerm_virtual_networka private network in Azure with an address range, like a /16
subnetazurerm_subneta slice of the VNet's range where the cluster's machines get addresses
NSGazurerm_network_security_groupnetwork security group: a list of firewall rules (allow/deny by port and source)
managed identityazurerm_user_assigned_identitya login for Azure resources, with no password to store (14.3)
role assignmentazurerm_role_assignmenta permission: this identity may do this role on this resource
Log workspaceazurerm_log_analytics_workspaceAzure's store for logs, which the cluster sends its logs to
the clusterazurerm_kubernetes_clusterAKS: machines that run containers, managed by Azure
Key Vaultazurerm_key_vaultAzure's safe for secrets such as passwords and keys

(A log workspace has nothing to do with Terraform workspaces from 16.1 - same word, different product.)

The module's files:

modules/aks/
  versions.tf     required_providers azurerm >= 4.0
  variables.tf    env, location, vnet_cidr, node_count, vm_size, sku_tier, zones, tags
  network.tf      resource group, VNet, AKS subnet, NSG + association
  identity.tf     user-assigned identity, role assignment on the subnet
  aks.tf          azurerm_kubernetes_cluster
  keyvault.tf     Key Vault (RBAC mode) + role assignment for the cluster
  monitoring.tf   log workspace
  outputs.tf      cluster id/name, kubelet identity, key vault uri, subnet id

One file per concern keeps each file short enough to review. Terraform reads every .tf file in the folder as one configuration, so the split is only for humans. (In the mission you may put all resources in one main.tf; the checks only need versions.tf, variables.tf, main.tf and outputs.tf.)

Networking

resource "azurerm_resource_group" "this" {
  name     = "rg-aks-${var.env}"
  location = var.location
  tags     = local.tags
}

resource "azurerm_virtual_network" "this" {
  name                = "vnet-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  address_space       = [var.vnet_cidr]
}

resource "azurerm_subnet" "aks" {
  name                 = "snet-aks"
  resource_group_name  = azurerm_resource_group.this.name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = [cidrsubnet(var.vnet_cidr, 6, 0)]      # a /22 from a /16
}

resource "azurerm_network_security_group" "aks" {
  name                = "nsg-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

resource "azurerm_subnet_network_security_group_association" "aks" {
  subnet_id                 = azurerm_subnet.aks.id
  network_security_group_id = azurerm_network_security_group.aks.id
}

Reading it top to bottom:

  • The resource group is named after the environment (rg-aks-dev, rg-aks-prod). Everything else lives inside it and copies its location.
  • The VNet gets the whole range passed in (var.vnet_cidr, e.g. 10.110.0.0/16).
  • The subnet takes the first /22 of that range: cidrsubnet(var.vnet_cidr, 6, 0) adds 6 bits to the /16 prefix (16 + 6 = 22) and picks block number 0 (14.14). A /22 holds 1,024 addresses.
  • The NSG starts empty (Azure's default rules apply).

Why a /22? The cluster's machines take addresses from this subnet, and depending on a setting called the network plugin (below), the containers may take addresses from it too. Running out of addresses is painful to fix later, so the module leaves room. Subnet sizing is the arithmetic from 10.6.

The association is its own resource. The NSG is attached to the subnet by a third resource, azurerm_subnet_network_security_group_association, that holds just the two IDs. Azure models many links this way (NSG to subnet, route table to subnet). Because the link is its own object in state, it can be added or removed without replacing the subnet or the NSG.

Identity: who the cluster logs in as

The cluster itself needs to change things in Azure - for example, create a load balancer (11.23) and claim addresses in its subnet. To do that it needs a managed identity: a login Azure gives to a resource, with no password or secret for anyone to store or rotate.

There are two kinds:

  • system-assigned: created together with the resource and deleted with it.
  • user-assigned: a separate resource you create yourself and attach.

The module uses a user-assigned identity, because it exists before the cluster. You can grant it permissions first, and it survives if the cluster is ever recreated:

resource "azurerm_user_assigned_identity" "aks" {
  name                = "id-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

resource "azurerm_role_assignment" "aks_subnet" {
  scope                = azurerm_subnet.aks.id
  role_definition_name = "Network Contributor"
  principal_id         = azurerm_user_assigned_identity.aks.principal_id
}

A role assignment answers three questions:

argumentquestionhere
principal_idwho?the identity (principal_id is its ID in Azure's login system)
role_definition_namemay do what?Network Contributor: a built-in role that may manage network things
scopeon what?only this subnet

Scope it to the subnet, not the resource group or the whole subscription (your Azure account area). Giving the smallest permission that works is called least privilege, and it is the default reviewers expect.

The cluster

resource "azurerm_kubernetes_cluster" "this" {
  name                = "aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  dns_prefix          = "aks-${var.env}"
  sku_tier            = var.sku_tier                 # "Free" in dev, "Standard" (SLA) in prod

  default_node_pool {
    name           = "system"
    node_count     = var.node_count
    vm_size        = var.vm_size
    vnet_subnet_id = azurerm_subnet.aks.id
    zones          = var.zones                        # null in dev, ["1", "2", "3"] in prod
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.aks.id]
  }

  network_profile {
    network_plugin      = "azure"
    network_plugin_mode = "overlay"
    network_policy      = "cilium"
    network_data_plane  = "cilium"
  }

  oms_agent {
    log_analytics_workspace_id = azurerm_log_analytics_workspace.this.id
  }

  local_account_disabled            = true
  role_based_access_control_enabled = true
  automatic_upgrade_channel         = "patch"

  depends_on = [azurerm_role_assignment.aks_subnet]
}

You do not need to understand clusters to read this - each argument is one plain decision:

argumentplain meaning
name, dns_prefixthe cluster's name, and the start of its web address in Azure
sku_tierFree: no uptime promise. Standard: Azure promises (an SLA, 0.8) the management side stays up, for a fee
default_node_poolthe group of virtual machines (nodes) the containers run on
node_count, vm_sizehow many machines, and which machine size (Standard_B2s small, Standard_D4s_v5 4 CPUs)
vnet_subnet_idput those machines in our subnet
zonesspread the machines over availability zones - separate datacentre buildings in the region - so one building failing does not stop everything. null = do not spread
identitylog in as our user-assigned identity
network_profilehow containers get addresses. azure + overlay: containers get addresses from a private range inside the cluster, so they do not use up subnet addresses. cilium: the software that enforces firewall rules between containers (network policy)
oms_agentsend logs to the log workspace
local_account_disabledno built-in admin password for the cluster; people log in with their Microsoft accounts (Entra ID, Microsoft's login service)
role_based_access_control_enabledpermissions inside the cluster are role-based (RBAC: roles granted to people, like the role assignment above)
automatic_upgrade_channellet Azure install patch-level updates of the cluster software automatically

What a reviewer asks about

  • name and dns_prefix force replacement. Changing either one means Azure cannot update the cluster in place: the plan says must be replaced, i.e. destroy it and build a new one (15.46). Never build them from anything that might change, like a team name.
  • sku_tier = "Standard" buys the uptime SLA. prod needs it; dev can run "Free".
  • depends_on on the role assignment - a hidden dependency. Terraform orders resources by references (14.18). The cluster references the identity, and the role assignment references the identity - but nothing in the cluster block references the role assignment. So Terraform may create the cluster and the role assignment at the same time, and the cluster then fails because its identity has no permission yet. depends_on is the textbook fix for a dependency Terraform cannot see.
  • local_account_disabled + RBAC means nobody has a shared admin password to the cluster; access goes through each person's own login.
  • Provider version names. These are the azurerm 4.x argument names. automatic_upgrade_channel was called automatic_channel_upgrade in 3.x - a rename in the 4.0 upgrade guide (you do that upgrade in 16.24).

Key Vault: the safe

data "azurerm_client_config" "current" {}

resource "azurerm_key_vault" "this" {
  name                          = "kv-aks-${var.env}-${random_string.kv.result}"
  location                      = azurerm_resource_group.this.location
  resource_group_name           = azurerm_resource_group.this.name
  tenant_id                     = data.azurerm_client_config.current.tenant_id
  sku_name                      = "standard"
  purge_protection_enabled      = var.env == "prod"
  soft_delete_retention_days    = 90
  public_network_access_enabled = false
  enable_rbac_authorization     = true

  network_acls {
    default_action = "Deny"
    bypass         = "AzureServices"
  }
}
  • data "azurerm_client_config" "current" is a data source (14.18) that reads who Terraform is logged in as. Its tenant_id is the ID of your organisation's Microsoft login directory, which a vault must belong to.
  • public_network_access_enabled = false and network_acls { default_action = "Deny" }: the vault cannot be reached from the internet; bypass = "AzureServices" still lets trusted Azure services in.
  • RBAC authorization (enable_rbac_authorization = true): who may read secrets is decided by ordinary role assignments - the same mechanism as the subnet permission above - reviewed like everything else. The older alternative, "access policies", is a separate permission list inside the vault. Newer 4.x releases than the lab's 4.14 rename the argument to rbac_authorization_enabled; the old name then warns and is removed in azurerm 5.0. terraform validate after a provider upgrade tells you which one your version wants.
  • Soft delete and purge protection. A deleted vault is kept recoverable for soft_delete_retention_days (90 days). Purge protection means nobody can wipe it for good during that time - which also means the vault's name stays taken for those 90 days. That is why the name has a random suffix, and why dev often leaves purge protection off (var.env == "prod" is true only in prod).
  • Key Vault names are globally unique (across all Azure customers) and 3-24 characters. kv-aks-prod-x7f2 fits; kv-aks-production-westeurope does not.

The cluster's identity then gets permission to read secrets from this one vault - another role assignment, with the built-in role Key Vault Secrets User and scope = azurerm_key_vault.this.id. Lesson 16.20 is about putting secrets in it.

Different sizing per environment

The module takes the sizes as inputs; each environment's terraform.tfvars sets them:

# envs/dev/terraform.tfvars
node_count = 1
vm_size    = "Standard_B2s"
sku_tier   = "Free"
zones      = null

# envs/prod/terraform.tfvars
node_count = 3
vm_size    = "Standard_D4s_v5"
sku_tier   = "Standard"
zones      = ["1", "2", "3"]

zones = null means "leave this argument out entirely" - null unsets an argument (14.16). Leaving zones out of the file has the same effect when its default is null.

Validate the inputs in the module, so a typo fails at plan time instead of deep inside an apply (14.8):

variable "sku_tier" {
  type    = string
  default = "Free"
  validation {
    condition     = contains(["Free", "Standard", "Premium"], var.sku_tier)
    error_message = "sku_tier must be Free, Standard or Premium."
  }
}

variable "node_count" {
  type = number
  validation {
    condition     = var.node_count >= 1 && var.node_count <= 20
    error_message = "node_count must be between 1 and 20."
  }
}

Outputs: what the module hands back

output "cluster_name"   { value = azurerm_kubernetes_cluster.this.name }
output "cluster_id"     { value = azurerm_kubernetes_cluster.this.id }
output "key_vault_uri"  { value = azurerm_key_vault.this.vault_uri }
output "aks_subnet_id"  { value = azurerm_subnet.aks.id }
output "kube_config" {
  value     = azurerm_kubernetes_cluster.this.kube_config_raw
  sensitive = true
}

Outputs are the module's public interface (15.35): the roots print cluster_name, other teams wire key_vault_uri or aks_subnet_id into their own code.

kube_config_raw is the cluster's login file (a "kubeconfig") - a credential. It is marked sensitive, so Terraform hides it in output, but it is still stored in state in plain text (15.1). With local_account_disabled = true it is much less powerful, because people still need their own Microsoft login - one more reason for that setting.

What the lab simulates

The lab's azurerm provider knows these resource types and their IDs, forces replacement for the arguments Azure really cannot change, and records everything in state. It does not build a real cluster - there are no machines behind the cluster ID - and it does not check every argument's allowed values (tflint's azurerm ruleset, lesson 16.6, and the real provider do that). (simulator)

What you can now do:

  • Read a module that composes a network, an identity, a cluster and a vault, and say what each resource is for.
  • Spot a hidden dependency and fix it with depends_on.
  • Wire an identity to a resource with a narrowly scoped role assignment.
  • Size environments through validated inputs and tfvars.

Why it helps

This module is the centre of the chapter's build and a portfolio piece you can walk an interviewer through. Every design choice in it is a review comment you will one day write or receive: grant the identity a role on the subnet, not the whole subscription; never build name or dns_prefix from something that changes, because changing them replaces the whole cluster; add depends_on when a resource needs another one it never references; give the Key Vault a random name suffix because purge protection keeps a deleted vault's name taken for 90 days. It is also the best practice you will get at composing many resources into one reusable module with clean inputs and outputs.

FAQ

Do I need to understand clusters for this lesson?

No. For Terraform, the cluster is one more resource with arguments: how many machines, which size, which subnet, which identity, which tier. The lesson explains each argument in one line. What the cluster does inside - scheduling and running containers - comes later in the course; here the point is building the network, identity, logs and vault around it the same way in every environment.

Why a user-assigned identity instead of system-assigned?

A user-assigned identity is its own resource, so it exists before the cluster. You can grant it roles (Network Contributor on the subnet) first, and the cluster finds them ready when it is created. It also survives if the cluster is ever recreated, so the role assignments do not have to be redone. A system-assigned identity is created and deleted together with its resource.

Why is the NSG attached with a separate association resource?

Azure models many links as objects of their own, NSG to subnet and route table to subnet among them, and the azurerm provider mirrors that with azurerm_subnet_network_security_group_association. Because the link is its own entry in state, it can be added, changed or removed without replacing either the subnet or the NSG.

What does sku_tier Standard buy?

The Standard tier adds an uptime SLA: Azure promises, with money back, that the cluster's management side stays available. Free has no such promise, which is fine for dev. The security scanner's check CKV_AZURE_170 flags the Free tier; for dev that is exactly the case for a documented skip comment with a reason.

Why does the Key Vault name have a random suffix?

Key Vault names are globally unique across all Azure customers and 3-24 characters long. With purge protection, a deleted vault cannot be wiped for its retention period, which also keeps its name taken. Recreating a vault with the same name would fail for up to 90 days. A random_string suffix avoids the clash; dev often leaves purge protection off for the same reason.

In an interview Mid

You are reviewing a platform module that creates a network, a managed identity, a container cluster and a Key Vault. What do you look for?

  • Hidden dependencies: the cluster references the identity, and the role assignment references the identity, but nothing links the cluster to the role assignment. Terraform may create them at the same time and the cluster fails without permission - depends_on = [azurerm_role_assignment.aks_subnet] is the textbook fix.
  • Least privilege: role assignments scoped to the subnet or the one vault, not the resource group or subscription.
  • Arguments that force replacement: name and dns_prefix must never be built from anything that could change.
  • Sizing through validated inputs: node_count, vm_size, sku_tier, zones set per environment in tfvars, with validation blocks so a typo fails at plan.
  • Key Vault: globally unique name (hence a random suffix), purge protection in prod, no public network access, RBAC authorization.
  • Outputs: IDs and names callers wire up; the cluster login file marked sensitive - and remembered to still be in state.
  • Provider version: argument names match the pinned major version.

Also asked: What is a managed identity, and why use a user-assigned one? · What is least privilege, and how does it apply to role assignments? · Why does a subnet NSG association exist as its own resource?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.