HCP Terraform: the exam's objective 8

Why a lesson on a product the lab does not have

Everything you built in this chapter - remote state, a pipeline, approvals, policy checks, a drift job - you built from parts: a storage account, shell scripts, jq, checkov. HashiCorp sells all of that as one hosted product, HCP Terraform. Many companies use it; many others (often Azure-heavy ones) build their own like you did.

Either way you need its model, for two reasons: the Associate exam's objective 8 is entirely about it, and being able to map "their" feature onto "our" pipeline is a skill interviewers test. (The lab has no HCP Terraform - this lesson is concepts, simulator.)

What you need to know already: backends and the cloud block (13.4), state locking (13.11), CLI workspaces (14.1), the pipeline, approvals and OIDC (14.15), policy as code and Sentinel/OPA by name (14.10), drift jobs (14.19), check blocks (12.10), registry modules (13.38).

What HCP Terraform is

HCP Terraform (formerly Terraform Cloud; HCP = HashiCorp Cloud Platform) is HashiCorp's hosted service for running Terraform. Terraform Enterprise is the same product installed on your own servers.

What it provides, and what you built instead:

remote state        stored, versioned and locked per workspace
remote runs         plan and apply execute on HashiCorp's (or your own) agents
run history         every plan/apply with its logs, who triggered it, who approved
variables           per workspace, and variable sets shared across workspaces
policy              Sentinel or OPA policies evaluated between plan and apply
private registry    modules and providers for your organisation
VCS integration     plans on pull requests, applies on merge
health              drift detection and continuous validation (check blocks)
teams & permissions who can read, plan, approve, apply - per workspace/project
run tasks           call external tools (e.g. a security scanner) during a run

• A run is one plan (and maybe apply) executed by HCP Terraform - the equivalent of one pipeline run.
• VCS means version control system - in practice your git host, like GitHub or Azure DevOps Repos.
• The rest map onto pieces of this chapter: state storage (13.4), the approval gate and artifacts (14.15), checkov and jq gates (14.10, 14.15), the drift job (14.19).

Organisation, projects, workspaces

HCP Terraform arranges everything in three levels:

organization  acme                         billing, SSO, teams, the registry
  project     payments                     groups workspaces; permissions can be set here
    workspace payments-network-dev         ONE state + variables + run history + settings
    workspace payments-network-prod
    workspace payments-app-prod
  project     platform
    workspace platform-aks-prod

• Organization - the top: your company's account, its users and teams, billing, single sign-on (SSO: logging in with the company account), and its private registry.
• Project - a group of workspaces, usually one per team or product. Team access is usually granted here: the payments team gets write on the payments project and read on platform. (Projects were added in 2023.)
• Workspace - one state, plus its variables, permissions, settings and run history.

An HCP workspace is not a CLI workspace (14.1). It is closer to a directory with its own backend key, variables and pipeline. The usual design is one workspace per configuration per environment - directory-per-environment in HCP's vocabulary.

Three workflows: how a run gets started

VCS-driven      the workspace is linked to a repo/branch/directory; a PR triggers a
                speculative plan (plan-only, cannot be applied); a merge triggers a
                run that waits for confirmation before apply
CLI-driven      you run terraform plan/apply locally; the run executes remotely and
                streams its output; state stays in HCP
API-driven      a pipeline uploads a configuration version and triggers runs via API

• VCS-driven: HCP watches a git repo. Opening a PR starts a speculative plan - a plan-only run that can never be applied, just shown on the PR. Merging starts a real run, which plans and then waits for someone to click Confirm & apply.
• CLI-driven: you type terraform plan on your laptop as usual, but the plan actually runs in HCP and its output streams back to your terminal.
• API-driven: your own pipeline uploads the code (a "configuration version") and starts runs through HCP's API (web interface for programs).

The CLI-driven workflow is set up with the cloud block (Terraform 1.1+), which replaces the backend block:

terraform {
  cloud {
    organization = "acme"
    workspaces {
      name = "payments-network-prod"      # or: tags = ["payments", "network"]
    }
  }
}

Then:

terraform login          # stores an API token in ~/.terraform.d/credentials.tfrc.json
terraform init
terraform plan           # "Running plan in HCP Terraform. Output will stream here..."

• terraform login opens a browser, lets you create an API token (a long secret string that proves who you are to HCP), and saves it in ~/.terraform.d/credentials.tfrc.json. Treat that file like a password.
• terraform init connects the folder to the workspace.
• terraform plan runs in HCP; the message says so.

workspaces { tags = [...] } instead of name maps one directory to several HCP workspaces (all with those tags), and you pick one with terraform workspace select - the one place CLI workspace commands meet HCP workspaces.

Moving an existing state into HCP: add the cloud block and run terraform init. It notices the old local or backend state and offers to migrate it into the workspace - the same idea as moving state between backends (13.7).

Execution modes: where the run happens

remote     plan/apply run on HCP infrastructure (the default)
local      HCP only stores state; runs happen where you invoke Terraform
agent      runs execute on self-hosted HCP agents inside your network

An agent is a small program you run on your own machine, which picks up runs from HCP and executes them there. Agents matter for private resources: an agent inside your Azure network can reach a private endpoint or a cluster management address that HashiCorp's shared machines on the internet cannot reach.

Variables and variable sets

Workspace variables come in two kinds:

• Terraform variables - like entries in terraform.tfvars.
• Environment variables - set in the shell of the run, like ARM_CLIENT_ID or TF_LOG.

Either kind can be marked sensitive: once saved, nobody can read it back in the UI or API (write-only), and it is hidden in logs.

Variable sets share variables across many workspaces - for example one set with the Azure login settings, applied to every workspace in a project.

Which value wins? A workspace's own variable overrides a value from a variable set - unless the set is marked priority. A priority set wins over everything, including workspace variables and values passed on the command line, which is how an organisation enforces a setting nobody may override.

With dynamic provider credentials, HCP uses OIDC (12.3, 14.15) to log in to Azure for each run, so no long-lived password is stored in HCP at all.

Governance: policies and run tasks

Governance means the organisation's rules and who may do what. Between the plan and the apply, HCP evaluates policy sets - groups of policies attached to workspaces:

• Sentinel - HashiCorp's policy language, with three enforcement levels: · advisory: warn, but let the run continue; · soft-mandatory: fail the run, unless someone with the right permission overrides it; · hard-mandatory: fail, no override possible.
• OPA (Open Policy Agent, written in the Rego language) - supported as an alternative, with advisory and mandatory levels.

# a Sentinel policy (shape)
import "tfplan/v2" as tfplan
main = rule {
  all tfplan.resource_changes as _, rc {
    rc.type is not "azurerm_storage_account" or
    rc.change.after.min_tls_version is "TLS1_2"
  }
}

Read it as: import the plan data; the policy passes if, for all resource changes, either it is not a storage account, or its new min_tls_version is TLS1_2. It reads the same resource_changes your jq gate read in 14.15.

Run tasks call external services - a security scanner like checkov, a cost estimate, a ticketing system - at set points of a run, and can block it.

Private registry

The organisation can publish its own modules (and providers):

module "network" {
  source  = "app.terraform.io/acme/network/azurerm"
  version = "~> 2.1"
}

The source reads <host>/<organisation>/<name>/<provider> - app.terraform.io is HCP's address. Modules are published from git repositories named terraform-<PROVIDER>-<NAME> (here terraform-azurerm-network) with version tags like v2.1.0, or through the API. Consumers get versions, docs and inputs/outputs shown like on the public registry, visible only inside the organisation.

Health and drift

HCP can run drift detection on a schedule - a refresh-only plan (12.26) that reports if reality no longer matches state - and continuous validation, which re-checks check blocks and postconditions (12.10) regularly. Workspaces that fail are flagged. It is the managed version of your nightly -detailed-exitcode job.

Where HCP fits for you

For the exam: the workflows, the organisation/project/workspace model, remote runs, state, variable sets and priority, policies and the Sentinel levels, the private registry, the cloud block and terraform login.

At work, the same ideas show up in different products: pipeline environments for approvals, a storage account for state, checkov for policy. Being able to map one onto the other is the useful skill.

What you can now do:

• Explain organisations, projects and workspaces, and why an HCP workspace is not a CLI workspace.
• Describe the VCS-, CLI- and API-driven workflows, including speculative plans and the cloud block with terraform login.
• Name the Sentinel enforcement levels and what a priority variable set does.