Users, groups, and who you are right now

Why identity comes before permissions

"Permission denied" is always a question about who is asking. Before you can fix one, you need to know which user a process runs as, which groups it has right now, and how the box turns names into the numbers it really checks.

What you need to know already: 1.11 (sudo), 2.5 (User= in a unit), 3.14 (/proc/<pid>/status).

The kernel only knows numbers

Every process runs with a UID (user ID, a number that identifies a user) and a set of GIDs (group IDs; a group is a named set of users you can grant access to all at once). Every file stores one UID and one GID. Names are a lookup table in /etc that the tools translate for you - the kernel never sees them.

$ grep -E '^(root|learner|appuser|www-data|nobody):' /etc/passwd
root:x:0:0:root:/root:/bin/bash
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
learner:x:1000:1000:Learner:/home/learner:/bin/bash
appuser:x:1001:1001:App runtime:/opt/app:/usr/sbin/nologin

(grep -E '^(a|b):' keeps lines that start with a: or b:; ^ means "start of line".) /etc/passwd is the user list. Seven colon-separated fields:

learner : x : 1000 : 1000 : Learner : /home/learner : /bin/bash
 name    |   UID    primary  comment   home          login shell
         |          group
         password is in /etc/shadow, not here

The login shell is the program started when that user logs in - normally bash.

The UID ranges on Ubuntu, and what they tell you at a glance:

0            root. The only UID the kernel treats specially.
1-999        system accounts: daemons, created by packages (useradd -r)
1000-59999   humans (UID_MIN / UID_MAX in /etc/login.defs)
65534        nobody - "no user at all", used for sandboxes and for
             network file shares that map unknown users to it

Service accounts

A service account is a user that exists only so a program can run as it - appuser runs orders, www-data runs nginx. Nobody logs in as it. Its login shell is /usr/sbin/nologin, a program that prints a refusal and exits:

$ sudo su - appuser
This account is currently not available.
$ sudo -u appuser id
uid=1001(appuser) gid=1001(appuser) groups=1001(appuser),1002(ops)

• su - appuser ("switch user") starts appuser's login shell, which is nologin, which prints that line and exits 1.
• sudo -u appuser <cmd> runs one command as that UID and never looks at the shell.
• id prints a UID, primary group and group list.

That is why sudo -u appuser <cmd> is the tool for "can the service account read this?" - you will use it constantly in this chapter.

Passwords live somewhere else

$ ls -l /etc/passwd /etc/shadow /etc/group
-rw-r--r-- 1 root root   433 Sep 14 17:43 /etc/group
-rw-r--r-- 1 root root   969 Sep 14 17:43 /etc/passwd
-rw-r----- 1 root shadow 613 Sep 14 17:43 /etc/shadow
$ sudo grep -E '^(learner|appuser):' /etc/shadow
learner:$y$j9T$Qm1vZ2hhbGlh$Xj8Zl2kqf7mV3p0R9sT1uW4bN6cE5dA2hG8iK0oLqPr:20340:0:99999:7:::
appuser:!:20340:0:99999:7:::

/etc/passwd has to be readable by everyone (every ls -l needs it to turn UIDs into names), so the password hashes - one-way scrambles of the password that can be checked but not reversed - moved to /etc/shadow, readable by root and group shadow only. (The -rw-r--r-- strings are permission bits: 4.5 decodes them.) A hash cannot be reversed, but anyone with a copy can try millions of guessed passwords against it on their own machine - so the copy itself has to be kept away from ordinary users.

$y$ marks yescrypt, Ubuntu's hashing method. ! in the password field means "no password will ever match": the account is locked for password login, which is exactly right for a service account.

Groups: one primary, many supplementary

$ grep -E '^(sudo|ops|adm):' /etc/group
adm:x:4:syslog,learner
sudo:x:27:learner
ops:x:1002:learner,appuser
$ id
uid=1000(learner) gid=1000(learner) groups=1000(learner),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),100(users),1002(ops)

/etc/group lines are name:x:GID:members.

• Primary group (gid=, the 4th field of passwd): the group new files get.
• Supplementary groups (the member lists in /etc/group): extra groups the kernel also checks against a file's group.

sudo works for you because you are in group sudo, and /etc/sudoers (the rules file for sudo) says %sudo ALL=(ALL:ALL) ALL - members of group sudo may run anything as anyone. adm is what lets you read most of /var/log without sudo. sudo -l lists what you are allowed:

$ sudo -l
User learner may run the following commands on oncall-lab:
    (ALL : ALL) ALL

Creating accounts

$ sudo groupadd deploy
$ sudo useradd -m -s /bin/bash -G ops alice                 # a human
$ sudo useradd -r -m -d /var/lib/reportgen -s /usr/sbin/nologin reportgen   # a service

groupadd makes a group; useradd makes a user. The flags, because the defaults surprise people:

• -m create the home directory. Ubuntu's useradd does not without it, and the default shell is /bin/sh. (adduser is a friendlier wrapper that asks questions and does both.)
• -d PATH where the home is; -s SHELL the login shell; -G ops extra groups.
• -r a system account: UID below 1000, no password expiry.
• Without -g you get a new group with the same name as the user (a "user private group").

$ getent passwd reportgen
reportgen:x:999:997::/var/lib/reportgen:/usr/sbin/nologin
$ ls -ld /var/lib/reportgen
drwxr-x--- 2 reportgen reportgen 4096 Sep 22 20:00 /var/lib/reportgen

getent passwd NAME asks the system's user lookup - the same one every program uses - rather than reading one file. On a company server users often come from a central directory service (LDAP, Active Directory) and are not in /etc/passwd at all. Use getent, not grep, when the answer matters.

Errors you will meet:

$ useradd alice
useradd: Permission denied.
useradd: cannot lock /etc/passwd; try again later.
$ sudo useradd alice
useradd: user 'alice' already exists
$ sudo usermod -aG nosuchgroup alice
usermod: group 'nosuchgroup' does not exist

The -G trap

usermod changes an existing user:

sudo usermod -aG deploy learner     # APPEND deploy to learner's groups
sudo usermod -G deploy learner      # REPLACE all supplementary groups with deploy

The second one silently removes you from sudo. On a box where you are the only admin, the next sudo says learner is not in the sudoers file and you are locked out of your own server. Always -aG. To add or remove one group, gpasswd -a / gpasswd -d:

$ sudo gpasswd -a appuser adm
Adding user appuser to group adm
$ sudo gpasswd -d appuser adm
Removing user appuser from group adm

Groups are read at login, not live

This one costs everyone an hour once:

$ sudo usermod -aG deploy learner
$ id                      # THIS shell, credentials fixed when you logged in
uid=1000(learner) gid=1000(learner) groups=1000(learner),4(adm),...,1002(ops)
$ id learner               # the DATABASE
uid=1000(learner) gid=1000(learner) groups=1000(learner),4(adm),...,1002(ops),1003(deploy)

A process's groups are copied into it when the login happens, and inherited by every child process. /etc/group changing later does not reach into running processes. Your options: log out and back in, or newgrp deploy for a sub-shell that has it (exit to leave it).

The same is true for services. You add appuser to a group so the service can read a directory - and it still cannot, because the running process was started before the change. systemctl restart is what applies it. Check a live process with grep Groups /proc/<pid>/status (numeric GIDs), never with id appuser.

Files keep numbers, not names

$ ls -ln /srv
drwxr-xr-x 4 0    0    4096 Sep 14 17:43 cache
drwxr-xr-x 2 1000 1002 4096 Sep 14 17:43 ops
$ sudo touch /srv/alice.txt; sudo chown alice /srv/alice.txt
$ sudo userdel -r alice
$ ls -l /srv/alice.txt
-rw-r--r-- 1 1002 root 0 Sep 22 20:00 /srv/alice.txt

ls -n shows the numbers instead of names. chown changes a file's owner (4.5); userdel -r deletes a user and their home. Their files elsewhere now show a bare UID. Create the next user and they may be given the same UID - and silently own every one of those files. After a userdel, sweep for orphans (the find syntax is 4.15):

sudo find / -xdev -nouser -o -xdev -nogroup

Later (Ch 10): a container's processes are ordinary processes with host UIDs, so "the container runs as root" usually means UID 0 on the host too. Everything here - numbers not names, group bits doing the work - is why that matters, and why a volume owned by UID 1001 is unreadable to an app running as UID 1000.

What you can now do

• Read /etc/passwd, /etc/group and id output and say who a process is.
• Create a service account and add a user to a group without losing others.
• Explain why a group change needs a new login or a service restart.