The platform module: network, identity, cluster (AKS), Key Vault

The problem: a platform that every environment gets the same way

In Chapters 10-11 you ran containers on one machine with Docker. Real teams run hundreds of containers on many machines, and they do not want to manage those machines by hand. They use a cluster: a group of machines that run containers for you - it decides which machine runs which container, restarts ones that die, and gives them network addresses.

AKS (Azure's managed container-cluster service) is Azure's managed cluster. You ask for one with a single Terraform resource, azurerm_kubernetes_cluster, and Azure runs the management side for you. You do not need to know how the cluster works inside for this lesson - treat it as one more Azure resource with arguments.

What you do need is everything around it: a network for its machines, a firewall, a login identity with the right permissions, a place for logs, and a safe for secrets. Built by hand, that is an afternoon of clicking and easy to get subtly different between dev and prod. Built as one module called from envs/dev and envs/prod (lesson 14.1), every environment gets the same platform, only sized differently.

This lesson is the design of that module. The mission after it builds it. The Terraform ideas it practises: module composition, hidden dependencies, identity wiring, outputs, and per-environment sizing.

What you need to know already: containers (10.3), subnets as address ranges and /16 vs /22 (8.3, 8.6), cidrsubnet (12.14), references and depends_on (12.18), validation blocks (12.8), sensitive outputs (12.5), modules and their files (13.34), Azure login methods incl. managed identity (12.3), the environments layout (14.1).

Later (Ch 15): you will learn what runs inside a cluster - pods, nodes, the API server. This chapter only builds the box it runs in.

The pieces, in plain words

(A log workspace has nothing to do with Terraform workspaces from 14.1 - same word, different product.)

The module's files:

modules/aks/
  versions.tf     required_providers azurerm >= 4.0
  variables.tf    env, location, vnet_cidr, node_count, vm_size, sku_tier, zones, tags
  network.tf      resource group, VNet, AKS subnet, NSG + association
  identity.tf     user-assigned identity, role assignment on the subnet
  aks.tf          azurerm_kubernetes_cluster
  keyvault.tf     Key Vault (RBAC mode) + role assignment for the cluster
  monitoring.tf   log workspace
  outputs.tf      cluster id/name, kubelet identity, key vault uri, subnet id

One file per concern keeps each file short enough to review. Terraform reads every .tf file in the folder as one configuration, so the split is only for humans. (In the mission you may put all resources in one main.tf; the checks only need versions.tf, variables.tf, main.tf and outputs.tf.)

Networking

resource "azurerm_resource_group" "this" {
  name     = "rg-aks-${var.env}"
  location = var.location
  tags     = local.tags
}

resource "azurerm_virtual_network" "this" {
  name                = "vnet-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  address_space       = [var.vnet_cidr]
}

resource "azurerm_subnet" "aks" {
  name                 = "snet-aks"
  resource_group_name  = azurerm_resource_group.this.name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = [cidrsubnet(var.vnet_cidr, 6, 0)]      # a /22 from a /16
}

resource "azurerm_network_security_group" "aks" {
  name                = "nsg-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

resource "azurerm_subnet_network_security_group_association" "aks" {
  subnet_id                 = azurerm_subnet.aks.id
  network_security_group_id = azurerm_network_security_group.aks.id
}

Reading it top to bottom:

• The resource group is named after the environment (rg-aks-dev, rg-aks-prod). Everything else lives inside it and copies its location.
• The VNet gets the whole range passed in (var.vnet_cidr, e.g. 10.110.0.0/16).
• The subnet takes the first /22 of that range: cidrsubnet(var.vnet_cidr, 6, 0) adds 6 bits to the /16 prefix (16 + 6 = 22) and picks block number 0 (12.14). A /22 holds 1,024 addresses.
• The NSG starts empty (Azure's default rules apply).

Why a /22? The cluster's machines take addresses from this subnet, and depending on a setting called the network plugin (below), the containers may take addresses from it too. Running out of addresses is painful to fix later, so the module leaves room. Subnet sizing is the arithmetic from 8.6.

The association is its own resource. The NSG is attached to the subnet by a third resource, azurerm_subnet_network_security_group_association, that holds just the two IDs. Azure models many links this way (NSG to subnet, route table to subnet). Because the link is its own object in state, it can be added or removed without replacing the subnet or the NSG.

Identity: who the cluster logs in as

The cluster itself needs to change things in Azure - for example, create a load balancer (9.23) and claim addresses in its subnet. To do that it needs a managed identity: a login Azure gives to a resource, with no password or secret for anyone to store or rotate.

There are two kinds:

• system-assigned: created together with the resource and deleted with it.
• user-assigned: a separate resource you create yourself and attach.

The module uses a user-assigned identity, because it exists before the cluster. You can grant it permissions first, and it survives if the cluster is ever recreated:

resource "azurerm_user_assigned_identity" "aks" {
  name                = "id-aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
}

resource "azurerm_role_assignment" "aks_subnet" {
  scope                = azurerm_subnet.aks.id
  role_definition_name = "Network Contributor"
  principal_id         = azurerm_user_assigned_identity.aks.principal_id
}

A role assignment answers three questions:

Scope it to the subnet, not the resource group or the whole subscription (your Azure account area). Giving the smallest permission that works is called least privilege, and it is the default reviewers expect.

The cluster

resource "azurerm_kubernetes_cluster" "this" {
  name                = "aks-${var.env}"
  location            = azurerm_resource_group.this.location
  resource_group_name = azurerm_resource_group.this.name
  dns_prefix          = "aks-${var.env}"
  sku_tier            = var.sku_tier                 # "Free" in dev, "Standard" (SLA) in prod

  default_node_pool {
    name           = "system"
    node_count     = var.node_count
    vm_size        = var.vm_size
    vnet_subnet_id = azurerm_subnet.aks.id
    zones          = var.zones                        # null in dev, ["1", "2", "3"] in prod
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.aks.id]
  }

  network_profile {
    network_plugin      = "azure"
    network_plugin_mode = "overlay"
    network_policy      = "cilium"
    network_data_plane  = "cilium"
  }

  oms_agent {
    log_analytics_workspace_id = azurerm_log_analytics_workspace.this.id
  }

  local_account_disabled            = true
  role_based_access_control_enabled = true
  automatic_upgrade_channel         = "patch"

  depends_on = [azurerm_role_assignment.aks_subnet]
}

You do not need to understand clusters to read this - each argument is one plain decision:

What a reviewer asks about

• name and dns_prefix force replacement. Changing either one means Azure cannot update the cluster in place: the plan says must be replaced, i.e. destroy it and build a new one (13.46). Never build them from anything that might change, like a team name.
• sku_tier = "Standard" buys the uptime SLA. prod needs it; dev can run "Free".
• depends_on on the role assignment - a hidden dependency. Terraform orders resources by references (12.18). The cluster references the identity, and the role assignment references the identity - but nothing in the cluster block references the role assignment. So Terraform may create the cluster and the role assignment at the same time, and the cluster then fails because its identity has no permission yet. depends_on is the textbook fix for a dependency Terraform cannot see.
• local_account_disabled + RBAC means nobody has a shared admin password to the cluster; access goes through each person's own login.
• Provider version names. These are the azurerm 4.x argument names. automatic_upgrade_channel was called automatic_channel_upgrade in 3.x - a rename in the 4.0 upgrade guide (you do that upgrade in 14.24).

Key Vault: the safe

data "azurerm_client_config" "current" {}

resource "azurerm_key_vault" "this" {
  name                          = "kv-aks-${var.env}-${random_string.kv.result}"
  location                      = azurerm_resource_group.this.location
  resource_group_name           = azurerm_resource_group.this.name
  tenant_id                     = data.azurerm_client_config.current.tenant_id
  sku_name                      = "standard"
  purge_protection_enabled      = var.env == "prod"
  soft_delete_retention_days    = 90
  public_network_access_enabled = false
  enable_rbac_authorization     = true

  network_acls {
    default_action = "Deny"
    bypass         = "AzureServices"
  }
}

• data "azurerm_client_config" "current" is a data source (12.18) that reads who Terraform is logged in as. Its tenant_id is the ID of your organisation's Microsoft login directory, which a vault must belong to.
• public_network_access_enabled = false and network_acls { default_action = "Deny" }: the vault cannot be reached from the internet; bypass = "AzureServices" still lets trusted Azure services in.
• RBAC authorization (enable_rbac_authorization = true): who may read secrets is decided by ordinary role assignments - the same mechanism as the subnet permission above - reviewed like everything else. The older alternative, "access policies", is a separate permission list inside the vault. Newer 4.x releases than the lab's 4.14 rename the argument to rbac_authorization_enabled; the old name then warns and is removed in azurerm 5.0. terraform validate after a provider upgrade tells you which one your version wants.
• Soft delete and purge protection. A deleted vault is kept recoverable for soft_delete_retention_days (90 days). Purge protection means nobody can wipe it for good during that time - which also means the vault's name stays taken for those 90 days. That is why the name has a random suffix, and why dev often leaves purge protection off (var.env == "prod" is true only in prod).
• Key Vault names are globally unique (across all Azure customers) and 3-24 characters. kv-aks-prod-x7f2 fits; kv-aks-production-westeurope does not.

The cluster's identity then gets permission to read secrets from this one vault - another role assignment, with the built-in role Key Vault Secrets User and scope = azurerm_key_vault.this.id. Lesson 14.20 is about putting secrets in it.

Different sizing per environment

The module takes the sizes as inputs; each environment's terraform.tfvars sets them:

# envs/dev/terraform.tfvars
node_count = 1
vm_size    = "Standard_B2s"
sku_tier   = "Free"
zones      = null

# envs/prod/terraform.tfvars
node_count = 3
vm_size    = "Standard_D4s_v5"
sku_tier   = "Standard"
zones      = ["1", "2", "3"]

zones = null means "leave this argument out entirely" - null unsets an argument (12.16). Leaving zones out of the file has the same effect when its default is null.

Validate the inputs in the module, so a typo fails at plan time instead of deep inside an apply (12.8):

variable "sku_tier" {
  type    = string
  default = "Free"
  validation {
    condition     = contains(["Free", "Standard", "Premium"], var.sku_tier)
    error_message = "sku_tier must be Free, Standard or Premium."
  }
}

variable "node_count" {
  type = number
  validation {
    condition     = var.node_count >= 1 && var.node_count <= 20
    error_message = "node_count must be between 1 and 20."
  }
}

Outputs: what the module hands back

output "cluster_name"   { value = azurerm_kubernetes_cluster.this.name }
output "cluster_id"     { value = azurerm_kubernetes_cluster.this.id }
output "key_vault_uri"  { value = azurerm_key_vault.this.vault_uri }
output "aks_subnet_id"  { value = azurerm_subnet.aks.id }
output "kube_config" {
  value     = azurerm_kubernetes_cluster.this.kube_config_raw
  sensitive = true
}

Outputs are the module's public interface (13.35): the roots print cluster_name, other teams wire key_vault_uri or aks_subnet_id into their own code.

kube_config_raw is the cluster's login file (a "kubeconfig") - a credential. It is marked sensitive, so Terraform hides it in output, but it is still stored in state in plain text (13.1). With local_account_disabled = true it is much less powerful, because people still need their own Microsoft login - one more reason for that setting.

What the lab simulates

The lab's azurerm provider knows these resource types and their IDs, forces replacement for the arguments Azure really cannot change, and records everything in state. It does not build a real cluster - there are no machines behind the cluster ID - and it does not check every argument's allowed values (tflint's azurerm ruleset, lesson 14.6, and the real provider do that). (simulator)

What you can now do:

• Read a module that composes a network, an identity, a cluster and a vault, and say what each resource is for.
• Spot a hidden dependency and fix it with depends_on.
• Wire an identity to a resource with a narrowly scoped role assignment.
• Size environments through validated inputs and tfvars.