Providers, versions and the lock file

Why this matters

What you need to know already: the block types and the workflow (12.1), environment variables (export, 6.1 and 2.21), and that an HTTP API is a web address programs send requests to (9.21).

Two engineers run the same Terraform code a week apart and get different plans. Nothing in the code changed - but a new version of the plugin that talks to the cloud came out in between, and one of them downloaded it. This lesson is about controlling exactly which plugin, which version, and with which credentials.

What a provider is

A provider is a plugin (a separate program Terraform starts) that knows one API. hashicorp/azurerm turns azurerm_subnet blocks into calls to Azure's management API (Azure Resource Manager, "ARM"). hashicorp/random generates values locally and calls nothing at all.

Terraform core downloads the provider binary during init, starts it as a child process (3.1) for every plan/apply, and talks to it over gRPC (a standard way for two programs to call each other; you never touch it).

Every provider has a source address, like a package name in npm:

registry.terraform.io / hashicorp / azurerm
      hostname           namespace    type

• hostname: the registry, the website providers are downloaded from. It defaults to registry.terraform.io, so hashicorp/azurerm is the short form.
• namespace: who publishes it (like a GitHub user or org).
• type: the provider's name.

Providers come in three tiers on the registry:

official    published by HashiCorp          hashicorp/azurerm, hashicorp/random
partner     published by the vendor         datadog/datadog, cloudflare/cloudflare
community   anyone                           read the source before you trust it

Declaring providers: required_providers

This is the provider's entry in the terraform { } block - the equivalent of dependencies in a package.json.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.14"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

• source: which provider (the address above).
• version: which versions are acceptable (the version constraint, explained below).
• The key (azurerm) is the local name - what your resource type prefixes and provider blocks use. It is almost always the same as the type, but it does not have to be.

If you use a resource without declaring its provider, Terraform assumes hashicorp/<prefix>: an azurerm_resource_group with no required_providers still works because hashicorp/azurerm exists. That fallback is a convenience, not a practice - it gives you no version constraint at all, and it breaks for any provider outside the hashicorp namespace.

Configuring a provider

The provider block sets how the provider behaves and which account it works in.

provider "azurerm" {
  features {}
  subscription_id = "00000000-1111-2222-3333-444444444444"
}

• features {} is required by azurerm, even empty. It holds behaviour switches (for example whether deleting a resource group that still contains resources is allowed).
• subscription_id picks the Azure subscription - the billing account that everything is created under. A company has several (dev, prod). Since azurerm 4.0 it is required - in the block, or in the ARM_SUBSCRIPTION_ID environment variable. In 3.x it was taken from whatever subscription your command-line login happened to be using, which is how people used to apply to the wrong one. (The lab behaves as if ARM_SUBSCRIPTION_ID is set - simulator.)

Credentials never go in the provider block. The block ends up in git, and anything in git is readable by everyone with the repo. Instead, azurerm finds credentials itself, from environment variables. The options, from best to worst for automation:

OIDC / workload identity federation   CI jobs: short-lived token, no secret at all
managed identity                      code running inside Azure (a VM): Azure vouches for it
service principal + certificate       a robot account that proves itself with a certificate
service principal + client secret     a robot account with a password (ARM_CLIENT_SECRET)
Azure CLI (az login)                  your laptop only: uses your own login

In plain words:

• A service principal is a robot user account for programs.
• A managed identity is a robot account Azure attaches to a machine, so there is no password to store at all.
• OIDC (OpenID Connect) lets a CI system (the server that runs your tests on every push) swap its own short-lived signed token for an Azure token - again, no stored password.

Every one of those is configured through ARM_* environment variables or use_oidc = true / use_msi = true - never by writing a secret into a .tf file.

More than one configuration of the same provider: alias

One provider block per provider is the default. When you need two - two regions, two subscriptions - give the extra ones an alias (a second name):

provider "azurerm" {
  features {}
}

provider "azurerm" {
  alias = "dr"
  features {}
  subscription_id = var.dr_subscription_id
}

resource "azurerm_resource_group" "primary" {
  name     = "rg-orders-weu"
  location = "westeurope"
}

resource "azurerm_resource_group" "dr" {
  provider = azurerm.dr
  name     = "rg-orders-neu"
  location = "northeurope"
}

(dr = disaster recovery: a second copy in another region, for when the first region is down. var.dr_subscription_id is an input variable - 12.5.)

• The block without an alias is the default; every resource uses it unless it says otherwise.
• provider = azurerm.dr is a meta-argument - an argument Terraform itself understands on any resource, not one the provider defines. It is a reference, not a string, so no quotes.

Later (Ch 13): reusable groups of resources (modules) receive providers from their caller with providers = { azurerm = azurerm.dr }, and never contain a provider block themselves.

Many providers in one configuration

This is what "multi-cloud / service-agnostic workflow" means in practice (exam objective 1c and 2c): one plan that spans several APIs, wired together by references.

resource "random_password" "db" {
  length  = 24
  special = true
}

resource "azurerm_key_vault_secret" "db" {
  name         = "db-password"
  value        = random_password.db.result
  key_vault_id = azurerm_key_vault.main.id
}

• random_password makes a 24-character password (special = true: include symbols). .result is the generated value.
• An Azure Key Vault is a locked box for secrets in the cloud; azurerm_key_vault_secret stores one value in it.

random generates the value, azurerm stores it. Terraform orders the two because one references the other; it does not care that they are different providers.

What init does with providers

terraform init prepares a directory: it downloads what the code needs. Run it first, and again whenever you add a provider or change a version.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "~> 4.14"...
- Installing hashicorp/azurerm v4.14.0...
- Installed hashicorp/azurerm v4.14.0 (signed by HashiCorp)
- Finding hashicorp/random versions matching "~> 3.6"...
- Installing hashicorp/random v3.6.3...
- Installed hashicorp/random v3.6.3 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Read it line by line:

1. It collects every provider requirement - from your required_providers, and from resource types used without a declaration.
2. It combines the constraints and asks the registry for the newest version that satisfies all of them ("Finding ... matching").
3. It downloads the binary into .terraform/providers/registry.terraform.io/hashicorp/azurerm/4.14.0/linux_arm64/ and checks it was really published by HashiCorp ("signed by HashiCorp").
4. It records the choice in .terraform.lock.hcl.

$ ls .terraform/providers/registry.terraform.io/hashicorp/azurerm/
4.14.0

On the next init (a teammate, or CI), the lock file wins:

- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Using previously-installed hashicorp/azurerm v4.14.0

To save downloading the same 200 MB provider into every directory, set a plugin cache: plugin_cache_dir = "$HOME/.terraform.d/plugin-cache" in ~/.terraformrc (Terraform's settings file in your home directory), or the TF_PLUGIN_CACHE_DIR environment variable. Networks with no internet access use terraform providers mirror (copy providers to a folder) and a provider_installation block pointing at that folder. (The lab's provider source is such a mirror, carrying azurerm up to 4.14.0 - simulator.)

Version constraints

Versions look like 4.14.0 = major.minor.patch (semantic versioning, as in npm): patch = bug fixes, minor = new features, major = may break things.

= 4.14.0        exactly this (a bare "4.14.0" means the same)
!= 4.10.0       anything but this one (a known-bad release)
>= 4.0          this or newer - no upper bound, so 5.0 would be accepted
< 5.0           below this
>= 4.0, < 5.0   both: an explicit range
~> 4.14         >= 4.14.0 and < 5.0.0     the rightmost number may grow
~> 4.14.0       >= 4.14.0 and < 4.15.0    patch releases only

The pessimistic operator ~> allows only the rightmost given component to increase. Two components (~> 4.14) means minor and patch bumps; three (~> 4.14.0) means patch only. It is like npm's ^ and ~, but decided by how many numbers you write. That one line is on every Terraform exam.

Where each style belongs:

root configuration     ~> 4.14     you own the upgrade; take minors, review majors
reusable module        >= 4.0      a minimum only - let the ROOT choose the version
terraform itself       ~> 1.9      in required_version

(The root configuration is the directory you run terraform in. A reusable module is shared code other configurations call - Ch 13.) A module that pins = 4.5.0 forces every caller onto 4.5.0; combined with another module pinning = 4.8.0, init fails because nothing satisfies both.

The lock file

.terraform.lock.hcl is like package-lock.json: the exact versions chosen, with checksums.

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/hashicorp/azurerm" {
  version     = "4.14.0"
  constraints = "~> 4.14"
  hashes = [
    "h1:...",
    "zh:...",
  ]
}

• version: what was picked. constraints: what the code allowed.
• hashes: checksums of the download, so a tampered or different binary is refused.

Facts worth having cold:

• It records providers only. Module versions are not locked here - a module taken from a git branch changes under you (Ch 13 has that incident).
• Commit it. Without it, two engineers resolving ~> 4.14 a week apart can get 4.14.0 and 4.15.0 and see different plans from identical code.
• h1: hashes are per platform (operating system + CPU type). A lock created on a Mac contains only darwin_arm64 hashes; a Linux CI machine then fails to verify. Fix once with terraform providers lock -platform=linux_amd64 -platform=darwin_arm64 (compute hashes for each listed platform).
• It is only changed by init (and providers lock). terraform init -upgrade is the deliberate "move to the newest allowed version".

When the constraint and the lock disagree

Change ~> 4.5.0 to ~> 4.10 in the code, and plan without re-running init:

# the providers mission's config after editing ~> 4.5.0 to ~> 4.10
terraform plan
╷
│ Error: Inconsistent dependency lock file
│
│ The following dependency selections recorded in the lock file are
│ inconsistent with the current configuration:
│   - provider registry.terraform.io/hashicorp/azurerm: locked version selection 4.5.0 doesn't match the updated version constraints "~> 4.10"
│
│ To update the locked dependency selections to match a changed
│ configuration, run:
│   terraform init -upgrade
╵

The lock says 4.5.0, the code now demands 4.10 or newer: they cannot both be true. Plain init refuses too, because the lock is only changed on purpose:

│ Could not retrieve the list of available versions for provider
│ hashicorp/azurerm: locked provider registry.terraform.io/hashicorp/azurerm
│ 4.5.0 does not match configured version constraint ~> 4.10; must use
│ terraform init -upgrade to allow selection of new versions

terraform init -upgrade resolves again, installs 4.14.0 and rewrites the lock:

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

The lock-file diff then goes into the same PR as the constraint change, and the plan in that PR is what proves the upgrade is harmless.

Seeing what you have

Two commands answer "which providers?":

$ terraform version
Terraform v1.9.8
on linux_arm64
+ provider registry.terraform.io/hashicorp/azurerm v4.14.0
+ provider registry.terraform.io/hashicorp/random v3.6.3

$ terraform providers

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/azurerm]
├── provider[registry.terraform.io/hashicorp/random]

• terraform version: the Terraform binary's version, then the providers actually installed (from the lock).
• terraform providers: the providers your code asks for, as a tree (. is the current directory).

When they disagree, you have found your problem.

Major upgrades

A major version (3.x to 4.0) is allowed to break things. azurerm 4.0, for example, made subscription_id mandatory and renamed one of its provider settings. The procedure is always the same:

1. Read the provider's upgrade guide for that major version.
2. Bump the constraint in a branch, init -upgrade, fix what terraform validate (the offline syntax check) complains about.
3. Plan against every environment. The goal is no changes - an upgrade that wants to replace things is a bug in your migration, not something to apply.
4. Merge the constraint, the lock file and the fixes together.

What you can now do:

• Declare providers with a sensible version constraint and explain ~>.
• Read init output and the lock file; upgrade on purpose with init -upgrade.
• Use two configurations of one provider with alias.