Chapter 26 GitOps, Argo CD & Delivery
Git as the only way into the cluster: kustomize overlays, Argo CD Applications, sync policies, prune and self-heal, drift and diffing, app-of-apps, promotion by commit, secrets that can live in git, and rollout strategies.
In plain words
Imagine a thermostat. You do not walk to the radiator and turn it up or down all day; you set the number you want on the wall, and the thermostat keeps checking the room and heating or cooling until it matches. If someone opens a window, it notices and corrects. To change the temperature, you change the number, and there is a note of who changed it and when.
GitOps is a thermostat for the cluster. The number on the wall is the config repo, payments-gitops, with a base and an overlay per environment. The thermostat is Argo CD running inside the cluster, pulling git and reconciling live objects to match. The pipeline no longer deploys: it commits a digest. Promotion is a pull request, rollback is git revert, and a kubectl edit in prod is an open window that self-heal closes.
Why it matters on call
In a bank, "the CI system can kubectl apply anything to prod" is an audit finding; "a controller applies what was merged to a protected branch after review" is a control. That is why platform teams move to GitOps, and why you will likely run Argo CD or Flux in your next role. When the page says "payments is OutOfSync and Degraded", you need to read both statuses, run argocd app diff, and know whether to fix git, fix the cluster, or add a narrow ignoreDifferences.
The chapter also covers the parts GitOps does not solve by itself: secrets (Sealed Secrets or External Secrets with Key Vault), release strategies (rolling, blue-green, canary with Argo Rollouts), and multi-tenant setups with AppProjects. These are staple interview topics for platform roles, and they build directly on the promotion-by-digest discipline from the previous chapter.
Lessons
- GitOps: git as the only way in
- kustomize: a base and overlays, without templates
- Argo CD: how it works, and the Application
- Sync policies: automated, prune, self-heal, waves and hooks
- How Argo CD diffs, and ignoreDifferences
- App of apps, ApplicationSet and projects
- Promotion in a GitOps world
- Secrets when git is the source of truth
- Delivery strategies: rolling, blue-green, canary
21 hands-on labs (missions, incidents and drills) run in the terminal: Open this chapter in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.
Questions people ask
Is GitOps just CI/CD with extra steps?
No, the direction changes. In push-based CD the pipeline holds cluster credentials and applies changes; the cluster is whatever the last run left. In GitOps the pipeline only commits to git, and an agent inside the cluster pulls and continuously reconciles. That removes cluster credentials from CI, detects drift within minutes, and makes git log the deployment history. CI still builds, tests and pushes images exactly as before.
What is the difference between Argo CD and Flux?
Both are mature open-source GitOps controllers (graduated projects of the CNCF, the foundation that also looks after Kubernetes) that pull from git and reconcile. Argo CD centres on the Application object, has a strong web UI, multi-cluster management from one instance, SSO and RBAC built in, and ApplicationSets. Flux is a set of controllers (source, kustomize, helm, image automation) configured with CRDs, no built-in UI, and native SOPS decryption. The concepts transfer; banks often pick Argo CD for the UI and RBAC.
Does GitOps mean nobody may use kubectl anymore?
Humans keep read access: kubectl get, logs, describe and exec for debugging are fine. Write access to managed namespaces is removed or limited, because any change outside git is drift that self-heal reverts, or that silently disappears on the next sync. Emergencies use a documented break-glass procedure, and the fix is then committed to git. The incident in this chapter shows what happens when that is skipped.
Why environments as directories and not branches?
With branch per environment, promotion is a merge that drags every other change along, branches drift apart, and "what differs between staging and prod" is a merge question. With directories on one branch (overlays/dev, overlays/prod), the difference is diff -r, promotion is a one-line change to one directory, and every Argo CD Application points at main with a different path.
Where do secrets go if git is the source of truth?
Not in git as plain Secrets: base64 is encoding, not encryption. Either encrypt them for the cluster with Sealed Secrets (the ciphertext lives in git, only the in-cluster controller can decrypt) or keep the value in a secret manager like Azure Key Vault and commit only a pointer, an ExternalSecret that the External Secrets Operator syncs. SOPS and the Secrets Store CSI driver are other common options.