Chapter 25 CI/CD Pipelines & Helm

Build once, deploy many: git as the delivery ledger, Azure Pipelines and GitLab CI on a self-hosted agent, variables and secrets, templates, promotion by digest with approvals, then Helm in depth - charts, values, templates, hooks, releases and how they get stuck.

In plain words

Think of a bakery that supplies ten shops. The baker tests one cake in the kitchen, then seals it in a box with a numbered sticker. The same sealed box travels to each shop; the shops only decide the price label and which shelf it sits on. If a shop baked its own copy "from the same recipe", nobody could promise it tastes like the one that was tested.

CI is the kitchen: every push compiles, tests and produces one sealed image with a digest (sha256:...) as its sticker. CD is the delivery van: it moves that exact image into dev, test and prod, with an approval before prod. Helm is the packaging: a chart plus per-environment values, installed as a numbered release. This chapter builds the whole route, from git push to a pod running payments-api.

Why it matters on call

Delivery is the part of platform work everyone touches and few understand end to end. When prod breaks after a release, the first questions are "what exactly is running?" and "what changed?", and you answer them with imageID on the pod, helm history and git log on the config repo. When a teammate's pipeline goes green while tests failed, you spot the multi-line script: without set -e. When a Helm upgrade hangs on another operation is in progress, you know it is a stuck pending-upgrade Secret, not a cluster problem.

This is also where most DevOps interviews spend their time: build once deploy many, CI vs CD, how secrets reach a pipeline, Helm values precedence and rollback. And you will feel the limits of a pipeline that runs helm upgrade itself: it needs cluster credentials, and anyone with kubectl can drift what it deployed.

Lessons

26 hands-on labs (missions, incidents and drills) run in the terminal: Open this chapter in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.

Questions people ask

What is the difference between continuous delivery and continuous deployment?

Continuous delivery means every build that passes CI is releasable: prod is one approval away. Continuous deployment removes the human: every green build goes to prod automatically. Regulated companies such as banks almost always run delivery, because a change record or an approver must sign off on prod. The pipeline shape is the same in both; the difference is whether the prod stage has an approval check on its environment.

Why not just rebuild the image for each environment from the same commit?

Because the same commit does not give the same bytes. The base image tag may have moved, Maven may resolve a newer transitive dependency, the compiler may differ. The image prod runs would then never have been tested, and "it passed in test" describes another artifact. Build once, record the digest, and let environments differ only in values files.

Why do I need both git and a pipeline tool? Isn't the pipeline the source of truth?

The pipeline is the machine that does the work; git is the ledger. The app repo holds the code and the pipeline YAML, the config repo holds the chart and the per-environment values. A deploy that is a commit in the config repo can be reviewed, audited with git log and undone with git revert. Pipeline run history expires and is hard to diff; git history is permanent and readable.

Is Helm required, or could the pipeline just run kubectl apply?

You can deploy with kubectl apply -f, and many teams do. Helm adds two things: templating (one chart, many environments via values) and release tracking (numbered revisions stored as Secrets, so helm rollback and pruning of removed objects work). Without release tracking, deleting a resource from your manifests leaves it running in the cluster.

Is Azure Pipelines knowledge useful if the next job uses GitLab or GitHub Actions?

Yes. The concepts map almost one to one: stages, jobs on runners or agents, artifacts between jobs, caches keyed on lockfiles, secrets injected as masked variables, environments with approvals. The syntax and some defaults differ, for example GitLab fails on the first failing script line while an Azure multi-line script only checks the last command. Learn one deeply and translate; interviewers ask about the concepts, not the YAML keywords.