Secrets: Key Vault references, not tfvars

The problem: a password in five places

An app needs a database password. The quick way: put it in terraform.tfvars, pass it into the app's settings, done. Six months later that password is in git history, in two CI logs, in the state file, in a saved plan attached to a ticket, and in the app's configuration where anyone with read access to the app can see it. Changing it means finding every one of those places.

This lesson is how to design secrets so they pass through as few places as possible - ideally none.

What you need to know already: sensitive = true and variable precedence incl. TF_VAR_ (12.5), what state stores (13.1: every attribute, in plain text), random_password (13.1), -replace (12.26), managed identities (12.3, 14.2), role assignments and Key Vault (14.2), plan artifacts (14.15), secrets in Docker images (10.40).

A secret is any value that grants access: a password, an API key (a long random string an app uses to log in to another service), a certificate's private key, a token. Rotation means replacing a secret with a new one, so the old one stops working.

Where secrets leak in a Terraform setup

terraform.tfvars committed with db_password = "..."      git history, forever
-var db_password=... in a pipeline                       CI logs, shell history
output "connection_string" without sensitive = true      plan output, CI logs
state                                                    every attribute, plaintext
saved plan files                                         every value
TF_LOG=TRACE                                             request bodies

Row by row:

• A committed tfvars file stays in git history even after you delete it.
• -var db_password=... on a command line is printed by pipelines that echo their commands, and stays in shell history (the history command).
• An output without sensitive = true is printed at the end of every apply.
• State stores every attribute of every resource in plain text JSON.
• A saved plan (14.15) contains every value too.
• TF_LOG=TRACE (12.24), the most verbose debug log, prints the raw requests Terraform sends to Azure - secrets included.

sensitive = true fixes exactly one line of that table (the CLI output). The rest need design.

Rule 1: never put secrets in tfvars

tfvars files are configuration, and configuration is committed. A secret in terraform.tfvars is in git history forever - rewriting history does not un-leak it, because clones, forks and caches already have it. The secret must be rotated.

*.auto.tfvars "just on my machine" is the same mistake one git add . away. If a secret must be passed in at all, it comes from the pipeline's secret store (a setting in the pipeline service that is masked in logs) as the environment variable TF_VAR_db_password - and even then it lands in state.

Rule 2: let Terraform create secrets where they live

The pattern that keeps secrets out of tfvars and pipelines entirely: Terraform generates the value and writes it straight into Key Vault, Azure's safe for secrets (14.2):

resource "random_password" "db" {
  length  = 32
  special = true
}

resource "azurerm_key_vault_secret" "db_password" {
  name            = "orders-db-password"
  value           = random_password.db.result
  key_vault_id    = azurerm_key_vault.this.id
  content_type    = "password"
  expiration_date = "2027-03-31T00:00:00Z"
}

• random_password (from the hashicorp/random provider) generates a random 32-character string with special characters, once, and keeps it in state.
• azurerm_key_vault_secret stores it in the vault under the name orders-db-password, with a label (content_type) and an expiry date - the two settings checkov's CKV_AZURE_114 and CKV_AZURE_41 want (14.10).

The password is generated, stored in Key Vault, and no human ever sees it. It is in state (random_password.db.result and the secret's value) - which is why state lives in a locked-down backend (13.4).

Rule 3: apps read from Key Vault at runtime - references, not values

The examples below use App Service: Azure's service that runs a web app for you (you give it code or a container image, it runs it). Its app settings are the environment variables the app sees when it starts.

The worst pattern: Terraform reads the secret and writes it into the app settings.

# DON'T: the plaintext value ends up in the app's configuration and in state twice
app_settings = {
  DB_PASSWORD = random_password.db.result
}

The right pattern: a Key Vault reference. The setting holds a pointer to the secret; App Service fetches the real value when the app starts, logging in with the app's managed identity:

resource "azurerm_linux_web_app" "orders" {
  # ...
  identity {
    type = "SystemAssigned"
  }

  app_settings = {
    DB_PASSWORD = "@Microsoft.KeyVault(SecretUri=${azurerm_key_vault_secret.db_password.versionless_id})"
  }
}

resource "azurerm_role_assignment" "orders_kv" {
  scope                = azurerm_key_vault.this.id
  role_definition_name = "Key Vault Secrets User"
  principal_id         = azurerm_linux_web_app.orders.identity[0].principal_id
}

• identity { type = "SystemAssigned" } gives the app its own managed identity (14.2). In the mission you use a user-assigned one instead; the app then also needs key_vault_reference_identity_id to know which identity to use.
• @Microsoft.KeyVault(SecretUri=...) is App Service's reference syntax. The app reads an ordinary environment variable DB_PASSWORD and never knows Key Vault was involved.
• versionless_id is the secret's address without a version number. Key Vault keeps every version of a secret; the versionless address always means "the current one", so a rotated secret reaches the app without a Terraform change.
• The role assignment gives the app's identity Key Vault Secrets User (may read secret values) on this vault only - not Contributor, not the whole subscription.
• Other platforms have the same idea. A container cluster, for example, can use a driver that mounts Key Vault secrets as files into containers using the workload's identity. Terraform's only job is to grant access.

Now a rotation is a Key Vault operation, and Terraform never hands the app the value.

Rule 4: identities instead of passwords

Many "secrets" exist only because one thing needs to log in to another: an app to a database, a pipeline to Azure, a cluster to a container registry (10.54). Managed identities remove the secret entirely:

pipeline -> Azure          OIDC federated credential (no client secret)
cluster -> registry        cluster identity + AcrPull role assignment
app -> Key Vault / Storage managed identity + a data-plane role
app -> Azure SQL           Entra ID authentication with the app's identity

Row by row:

• pipeline -> Azure: OIDC (12.3, 14.15) - Azure trusts the pipeline's short-lived token; there is no password.
• cluster -> registry: the cluster's machines log in to ACR (Azure Container Registry) with their managed identity, which gets the built-in AcrPull role (may download images).
• app -> Key Vault or storage: the app's identity gets a data-plane role - a role about the data inside (read secrets, read files) rather than about managing the resource.
• app -> Azure SQL (Azure's managed database): the database accepts Entra ID (Microsoft's login service) logins, so the app logs in as its identity.

The best secret is the one that does not exist.

Rule 5: protect what is unavoidable

• State in a backend with RBAC, encryption and versioning, readable only by the pipeline identities (13.4).
• sensitive = true on every output and variable that carries a secret, so plans and logs show (sensitive value).
• Short retention on plan artifacts; no TRACE logs in pipelines.
• Rotate on exposure. If a value reached a log or git, it is compromised. With Terraform-generated secrets, rotation is one command:

terraform apply -replace=random_password.db

-replace=ADDRESS forces Terraform to destroy and recreate that one resource (12.26). A new random password is generated, the Key Vault secret gets a new version, and apps using versionless references pick it up.

Newer Terraform: ephemeral values and write-only arguments

Terraform 1.10 added ephemeral values and 1.11 write-only arguments - the first language features that keep secrets out of state and plan files altogether. The exam (version 1.12) includes them; the lab runs 1.9 and does not:

variable "db_admin_password" {
  type      = string
  sensitive = true
  ephemeral = true          # 1.10+: available during the run, never persisted
}

ephemeral "azurerm_key_vault_secret" "admin" {   # 1.10+: an ephemeral resource
  name         = "sql-admin"
  key_vault_id = azurerm_key_vault.this.id
}

resource "azurerm_mssql_server" "this" {
  # ...
  administrator_login_password_wo         = ephemeral.azurerm_key_vault_secret.admin.value  # 1.11+ write-only
  administrator_login_password_wo_version = 1
}

• An ephemeral variable or resource exists only during the plan/apply that uses it. Terraform refuses to store it in state or in the plan. Here, the ephemeral block reads a secret from Key Vault for the length of the run.
• A write-only argument (the _wo suffix, where the provider offers one) is sent to Azure but never recorded. Because Terraform does not remember the value, it cannot tell when it changed - so you bump _wo_version (1 -> 2) when you want it to send a new one.

Know they exist, what problem they solve, and that they need both a new Terraform and provider support. Which resources offer ephemeral variants and _wo arguments varies by provider version, so treat the snippet above as the shape of the feature and check the provider docs for the exact names.

Sensitive vs secret, one more time

sensitive = true      redacts CLI output                  still in state and plan files
ephemeral = true      (1.10+) never in state or plan      only usable in ephemeral contexts
Key Vault reference   the value never passes through the app configuration or Terraform
managed identity      there is no secret at all

From weakest to strongest: hiding the value on screen, keeping it out of state, keeping it out of the app's configuration, and not having a secret at all.

What you can now do:

• List the places a Terraform secret leaks, and which one sensitive fixes.
• Generate a secret into Key Vault and give an app a Key Vault reference plus a read-only role on that vault.
• Rotate a Terraform-generated secret with -replace.