Think of a recipe card. The top says what the dish is called and what must be ready first ("oven preheated"). The middle says exactly how to cook it. The bottom says "serve this at every Sunday lunch". A cook reading the card knows everything without asking.
A unit file is that card. [Unit] is the title and what comes first, [Service] is how to run the program, and [Install] is the "every Sunday" part, which only matters once you systemctl enable it. The ending of the file name says what kind of card it is: .service for a program, .timer for a schedule, .target for a milestone like "the system is up". systemctl cat ssh shows you the card; systemctl show ssh shows what the cook actually understood.
The problem this chapter solves
Your team's web app runs on a Linux box. At 3am it crashes. Who starts it again? Who notices? Where did its last words go? And after the box reboots, what brings it back? On Ubuntu the answer to all four is systemd, the program running as PID 1 that you met in 1.15 (Who's in charge). This chapter is about telling it what to run, and reading what it tells you back.
What you need to know already: logging in and moving around (1.1-1.6), exit codes and redirection (1.7), sudo and apt (1.11), that PID 1 is systemd (1.15).
A few words first
A process is one running program. Every process has a PID (process ID, a number the kernel gives it).
A daemon is a process that runs in the background with no terminal attached, waiting to do work: sshd (the SSH server that let you log in), cron (runs scheduled jobs), nginx (a web server). Names often end in d.
systemd starts daemons at boot, watches them, restarts them when they die and collects everything they print.
systemd manages "units"
A unit is anything systemd knows how to start, stop and order: one thing it manages. Each unit is described by a unit file, a small text file. The file name's ending (its suffix) tells you the kind of unit:
.service - a program to run and supervise (usually a daemon). 95% of your work. ssh.service is the SSH server.
.timer - a schedule ("every day at 03:00") that starts another unit. It replaces cron, the older Unix job scheduler.
.target - a named group of units, used as a milestone during boot. multi-user.target means "the system is fully up for logins, no graphical desktop".
.socket - a network port (a numbered door on the machine, like 22 for SSH) that systemd holds open itself; it starts the matching service on the first connection. This is called socket activation.
.mount - a mount: attaching a disk (or a slice of one) to a directory, so that /data, say, is really a second disk. Usually generated from /etc/fstab, the file listing what gets mounted at boot.
Others exist (path, slice, scope, swap, device); a couple come back later in this chapter.
Reading a unit file
A unit file is plain text in sections. A section starts with a name in square brackets; under it are directives (also called keys or settings), one Name=value per line. Here is a made-up web app:
[Unit] - what this is and how it relates to other units. Description is the human name. After/Wants say "start after the network is up, and pull it in" (2.14 explains the difference). The StartLimit* settings also live here - remember that, it comes back in the start-limit step.
[Service] - how to run the program. Type says when it counts as started (2.5). User is which Linux user account it runs as. WorkingDirectory is the folder it starts in. ExecStart is the exact command line to run. Restart=on-failure and RestartSec=5 mean "if it crashes, start it again after 5 seconds".
[Install] - what systemctl enable should do: WantedBy=multi-user.target means "start me at boot, as part of reaching multi-user.target". Nothing in [Install] happens until you enable the unit - it describes a link to create, it does not create it.
A unit with no [Install] section can still be started by hand; it just cannot be enabled. Try it and systemd tells you the unit is "not meant to be enabled".
Reading one on a real box
systemctl is the command you use to talk to systemd. Its first word after systemctl is a sub-command (what to do), then the unit name:
systemctl cat ssh # print the unit file(s), with their paths as comments
systemctl status ssh # is it running? its PID, its recent log lines
systemctl show ssh # every setting systemd resolved, one per line
You can leave the .service suffix off: ssh means ssh.service.
cat shows you what a human wrote. show shows you what systemd actually loaded. When those two disagree, someone edited the file and forgot to tell systemd to re-read it (daemon-reload, step 2.5).
What you can now do
say what a unit, a service, a timer and a target are
read the three sections of a unit file
ask systemd about one unit with cat, status and show
Why it helps
Reading a unit file quickly is the first step in almost every "the service is broken" ticket: why does it start before the network, who set this limit, why does it not start at boot. systemctl cat tells you which file and which drop-ins are in effect; systemctl show tells you what systemd actually resolved, and the difference between the two is where the bugs hide.
It also makes you a useful reviewer: a directive in the wrong section, a missing [Install], or a WantedBy= pointing at a target nobody reaches are review comments that save a failed rollout. And the unit types explain services that "start by themselves" - a socket or a timer started them.
What is the difference between systemctl cat and systemctl show?
cat prints the unit file and all its drop-ins exactly as written, with a comment line naming each file's path. show prints the properties systemd actually loaded and worked out, including defaults you never wrote, such as TimeoutStopUSec=1min 30s. If you edited a file and show still has the old value, you have not run daemon-reload. show -p Name --value is the form for scripts.
Can a unit with no [Install] section be started?
Yes. [Install] only tells systemctl enable which links to create for boot. Without it the unit is called static: you can start it by hand, and another unit or a timer can start it, but you cannot enable it on its own. Many system units, and most jobs that a timer runs, look exactly like this.
What is a target, if it runs nothing?
A target is a named milestone that groups other units. multi-user.target means "a normal server is up, ready for logins"; services attach to it with WantedBy=multi-user.target so they start when it is reached. Targets are also handy as ordering points, like network-online.target, which means "the network is actually usable".
Why would I use a .socket unit?
So systemd opens the network port itself and only starts the service on the first connection, handing the connection over. The machine boots faster because services start on demand, and no connection is lost while the service restarts, because systemd keeps the port open. Recent Ubuntu does this for SSH with ssh.socket, which is why ssh.service can look disabled and still run.
Do unit file names matter?
Yes, the name is the unit's identity. demo.service is what you start, and a timer finds its service by having the same name before the dot. The ending decides the type. An @ in the name makes a template (2.28). A typo such as report.sevice means systemd never finds the unit at all, and the error says "Unit not found".
In an interview Junior
What are the main sections of a systemd service unit, and what goes in each?
[Unit]: what it is and how it relates to other units - Description=, ordering and dependencies (After=, Wants=), and the start limit (StartLimitIntervalSec=, StartLimitBurst=).
[Service]: how to run the program - Type=, ExecStart= (the exact command, absolute path), User=, WorkingDirectory=, Restart=, RestartSec=.
[Install]: what systemctl enable should do - WantedBy=multi-user.target means "start me at boot". Nothing in it happens until you enable the unit.
The suffix of the file says the unit type: .service, .timer, .target, .socket, .mount. To look at one: systemctl cat ssh prints the file(s) as written, systemctl show ssh what systemd actually loaded. When they disagree, someone edited the file and forgot daemon-reload.
Also asked: What unit types exist besides services? · What is a target, and what is multi-user.target? · What is the difference between systemctl cat and systemctl show?