OnCallReady

Lesson 33.11 · Ansible: Configuration as Code · 21 min read

Templates (Jinja2) and files

In plain words

A template is a letter with blanks: "Dear ___, your table for ___ is at ___ o'clock". You write the letter once and a helper fills the blanks differently for each guest. In Ansible the letter is a config file, the blanks are Jinja2 expressions such as {{ app_port }}, and the helper fills them per host from variables and facts.

Jinja2 can also repeat a line for every item in a list (one server line per web server) and choose between lines (four workers in prod, one in dev). The result is a normal file on each host, different where it needs to be, identical everywhere else.

The problem

Configuration files are where variables pay off. The load balancer's config must list every web server's address; the web servers' config differs only by port; the app's config needs a worker count that depends on the host's CPUs. Writing those files by hand per host is the snowflake-server problem again. Ansible writes them for you from templates: the file with holes in it, filled per host.

What you need to know already: the variables lesson (variables, facts, hostvars), 4.7 (file permissions and modes), 7.6 (sed - the thing lineinfile replaces).

Four ways to put a file on a host

ansible.builtin.copy        a file as it is: from files/, or inline content:
ansible.builtin.template    a Jinja2 template from templates/, rendered per host
ansible.builtin.file        no content: directories, links, modes, owners, deletion
ansible.builtin.lineinfile  one line in a file someone else owns (and blockinfile: a block)

A useful rule: whoever owns the whole file templates it. If your team owns /etc/nginx/sites-available/default, template the whole thing - the template in git is then the complete truth. Use lineinfile / blockinfile only for files you share with the package or another team (/etc/sysctl.conf, /etc/hosts), where you own one line.

Relative src: paths are looked up in files/ (copy) or templates/ (template) next to the playbook - inside a role, in the role's own directories (two lessons on).

- name: Site config
  ansible.builtin.template:
    src: site.conf.j2                  # templates/site.conf.j2 on the control node
    dest: /etc/nginx/sites-available/default
    owner: root
    group: root
    mode: "0644"
    backup: true                       # keep the old file as default.<pid>.<date>~
    validate: nginx -t -c %s           # test the new file before it replaces the old one

copy and template compare checksums: they write (and report changed) only when the content, owner or mode differs. That is what makes them idempotent - and what makes the "notify a handler when the config changed" pattern of the next lesson possible.

validate: runs a command on a temporary copy (%s is its path) and refuses to put the file in place if the command fails. For anything a daemon parses (nginx -t, visudo -cf, sshd -t -f), it turns "the new config broke the service" into "the task failed and the old config is still there".

Jinja2 in one screen

Templates (and every {{ }} in a playbook) are Jinja2:

{{ expression }}               print a value
{% statement %}                logic: if, for, set
{# comment #}                  dropped from the output

{{ app_port }}                 a variable
{{ ansible_facts['hostname'] }}            a fact
{{ users[0].name }}            list index, then a key (also users[0]['name'])
{{ app_port + 1 }}  {{ 'a' ~ 'b' }}        arithmetic, ~ joins strings
{{ x if x > 0 else 1 }}        inline if

Filters transform a value with |, and chain:

{{ name | upper }}                         WEB-1
{{ timeout | default(30) }}                30 when timeout is not set
{{ pkgs | join(', ') }}                    nginx, htop
{{ pkgs | length }}                        2
{{ '42' | int + 1 }}                       43
{{ path | basename }}                      the last part of a path
{{ text | regex_replace('^v', '') }}       sed-like replace (Python regex)
{{ data | to_nice_yaml }}  {{ data | to_json }}
{{ secret | b64encode }}
{{ 'S3cret' | password_hash('sha512') }}   a /etc/shadow-style hash for the user module
{{ users | map(attribute='name') | list }} one attribute of every item

A variable that does not exist is an error, not an empty string - Ansible stops the task and says 'app_port' is undefined. | default(...) is how you make a value optional.

Try expressions without writing a file, on the implicit localhost:

$ cd ~/oncall-lab/labs/1a-ansible/try/templates
$ ansible localhost -m ansible.builtin.debug -a 'msg={{ ["web-2","lb-1","web-1"] | sort | join(",") }}'
localhost | SUCCESS => {
    "msg": "lb-1,web-1,web-2"
}
$ ansible localhost -m ansible.builtin.debug -a 'msg={{ missing_var | default("fallback") }}'
localhost | SUCCESS => {
    "msg": "fallback"
}

Statements loop and branch:

{% for h in groups['web'] %}
    server {{ hostvars[h]['ansible_facts']['default_ipv4']['address'] }}:80;
{% endfor %}

{% if env == 'prod' %}
worker_processes 4;
{% else %}
worker_processes 1;
{% endif %}

Inside a for, loop.index (1, 2, ...), loop.first and loop.last help with commas and separators. The template module sets trim_blocks: the newline after a {% ... %} line is removed, so a line holding only a statement does not leave a blank line behind. A - inside the braces ({%- ... -%}) strips the whitespace on that side too.

A template that uses other hosts' facts

The lab project renders the load balancer's list of web servers. The template reads each web host's IP from hostvars - which only has facts for hosts that gathered them, so the play first gathers facts on the web group:

$ cat templates/upstream.conf.j2
# {{ ansible_managed }}
upstream web_pool {
{% for h in groups['web'] %}
    server {{ hostvars[h]['ansible_facts']['default_ipv4']['address'] }}:{{ upstream_port }};
{% endfor %}
}
$ cat play.yml
- name: Facts from the web hosts
  hosts: web
  gather_facts: true
  tasks: []

- name: The load balancer's list
  hosts: lb
  gather_facts: false
  become: true
  vars:
    upstream_port: 80
  tasks:
    - name: Config directory
      ansible.builtin.file:
        path: /etc/lab
        state: directory
        mode: "0755"

    - name: Render the upstream list
      ansible.builtin.template:
        src: upstream.conf.j2
        dest: /etc/lab/upstream.conf
        mode: "0644"
$ ansible-playbook play.yml

PLAY [Facts from the web hosts] ************************************************

TASK [Gathering Facts] *********************************************************
ok: [web-1]
ok: [web-2]

PLAY [The load balancer's list] ************************************************

TASK [Config directory] ********************************************************
changed: [lb-1]

TASK [Render the upstream list] ************************************************
changed: [lb-1]

PLAY RECAP *********************************************************************
lb-1                       : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
web-1                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
web-2                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
$ ssh lb-1 cat /etc/lab/upstream.conf
# Ansible managed
upstream web_pool {
    server 10.0.5.11:80;
    server 10.0.5.12:80;
}

# {{ ansible_managed }} at the top of a templated file tells the next person "do not edit this by hand, it will be overwritten" - set a more helpful text with ansible_managed = Managed by Ansible - edit roles/web in the platform repo in ansible.cfg.

Run it again: ok, not changed. Change a variable (-e upstream_port=8080) and the template task reports changed - the rendered file differs. --diff shows how:

$ ansible-playbook play.yml -e upstream_port=8080 --diff

PLAY [Facts from the web hosts] ************************************************

TASK [Gathering Facts] *********************************************************
ok: [web-1]
ok: [web-2]

PLAY [The load balancer's list] ************************************************

TASK [Config directory] ********************************************************
ok: [lb-1]

TASK [Render the upstream list] ************************************************
changed: [lb-1]
--- before: /etc/lab/upstream.conf
+++ after: /home/learner/oncall-lab/labs/1a-ansible/try/templates/templates/upstream.conf.j2
@@ -1,5 +1,5 @@
 # Ansible managed
 upstream web_pool {
-    server 10.0.5.11:80;
-    server 10.0.5.12:80;
+    server 10.0.5.11:8080;
+    server 10.0.5.12:8080;
 }


PLAY RECAP *********************************************************************
lb-1                       : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
web-1                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
web-2                      : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The file module

ansible.builtin.file manages everything about a path except its content:

- name: App directory
  ansible.builtin.file:
    path: /srv/app
    state: directory          # also: file (must exist), touch, link, hard, absent
    owner: app
    group: app
    mode: "0750"

- name: Current release link
  ansible.builtin.file:
    src: /srv/app/releases/1.4.2
    dest: /srv/app/current
    state: link

- name: Old release gone
  ansible.builtin.file:
    path: /srv/app/releases/1.3.0
    state: absent             # rm -rf, idempotent: ok when it is already gone

state: file does not create a file - it fails if the file is missing. To create an empty file use touch (which reports changed every run, because it updates the time), or better copy with content: "".

lineinfile and blockinfile

- name: One setting in a shared file
  ansible.builtin.lineinfile:
    path: /etc/ssh/sshd_config
    regexp: '^#?PasswordAuthentication'
    line: PasswordAuthentication no
    validate: sshd -t -f %s

- name: Our block in /etc/hosts
  ansible.builtin.blockinfile:
    path: /etc/hosts
    block: |
      10.0.5.11 web-1
      10.0.5.12 web-2

lineinfile replaces the last line matching regexp, or adds line at the end when nothing matches. Without regexp it only checks that the exact line exists - so changing the value later adds a second line instead of replacing the first. Always give a regexp that matches every version of the line.

blockinfile wraps its block in # BEGIN ANSIBLE MANAGED BLOCK / # END ... marker lines and replaces whatever is between them on the next run.

What you can now do

Why it helps

Config files are where most production changes live: nginx sites, app settings, sysctl, sudoers. Templating them from one source means a change is one reviewed edit instead of a hand edit on each host, and --diff shows exactly which line changes on which host before it happens.

The validate option is the safety net to remember: nginx -t or visudo -cf runs on the new file before it replaces the old one, so a typo fails the task instead of breaking the service or locking everyone out of sudo.

Commands in this lesson

cd ansible cat ansible-playbook ssh

FAQ

copy or template?

copy puts a file on the host as it is, from files/ or from inline content. template renders a Jinja2 file from templates/ first, so it can contain variables, loops and conditions. If the file is the same everywhere and has no variables, copy is simpler; as soon as one value differs per host or environment, use a template.

When should I use lineinfile?

For one line in a file you do not own as a whole, such as a setting in sshd_config or a line in /etc/hosts. Always give a regexp that matches every form of the line, or a changed value adds a second line instead of replacing the first. If your team owns the whole file, template it instead.

What does ansible_managed do?

It is a text from ansible.cfg, "Ansible managed" by default, that you put in a comment at the top of every templated file. Anyone who opens the file on the host then knows not to edit it by hand, because the next run will overwrite their change. Set a more helpful text in ansible.cfg, for example the repository path.

How do I try an expression without a template?

Use the debug module ad hoc: ansible web-1 -m ansible.builtin.debug -a 'msg={{ users | map(attribute="name") | join(",") }}'. It renders the expression with that host's variables and prints the result, which is much faster than editing a template and running the playbook each time.

Why do some values need quotes in YAML?

A YAML value that starts with {{ looks like the start of a dictionary to the YAML parser, so the playbook fails to load. Quote the whole value: msg: "{{ greeting }} world". Inside template files you never need these quotes, because templates are not YAML.

In an interview Junior

How do you manage a config file that differs slightly per host?

With the ansible.builtin.template module and a Jinja2 template in templates/: the file is written once with expressions such as {{ app_port }} or {{ ansible_facts['processor_vcpus'] }}, Filters like default(30), join or upper, and Statements such as for and if. Ansible renders it per host and writes it only if the result differs, so it stays idempotent; --diff shows the change first. I set mode as a quoted string, put # {{ ansible_managed }} at the top, and add validate: (nginx -t -c %s, visudo -cf %s) so a broken file is refused before it replaces the working one. A handler then reloads the service only when the file changed.

Also asked: What is the difference between copy, template and lineinfile? · How would you list every web server's IP in a load balancer config? · What does the validate parameter do?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.