OnCallReady

Lesson 3.1 · Processes & Signals · 16 min read

ps, and the STAT column

In plain words

Imagine a class register. One version lists each pupil with who brought them to school (their parent). Another version lists each pupil with how much of the playground and snacks they are using. Both are the same class, just different columns.

ps -ef is the first register: it shows PPID, the parent. ps aux is the second: it shows %CPU and %MEM. ps -eo pid,ppid,rss,stat,comm --sort=-rss lets you choose your own columns. The STAT column is each pupil's mood: R busy, S waiting happily, D stuck and unable to hear you, Z already gone home but not signed out, T paused. RSS is memory really in RAM; VSZ is everything they could claim, mostly irrelevant.

What is running on this box?

A page says "the server is slow" or "memory is almost gone". Before you can fix anything you need a list of every running program, who started it, and what it is doing right now. That list is the process table, and ps prints it.

What you need to know already: 1.15 (PID 1 and parent processes), 1.7 (pipes, | head).

Reminder: a process

A process is one running copy of a program. The kernel (the core of the operating system, which runs the hardware and every program) gives each one a number, the PID, and remembers which process started it: the parent, whose PID is the PPID.

ps, two old spellings

ps (process status) prints the process table once and exits. It has two option styles from two old Unix families, and you will see both in the wild:

ps aux     BSD style, no dash
ps -ef     UNIX style, with a dash

ps aux: a = processes of all users, u = user-oriented columns, x = also processes with no terminal (services). Its columns:

USER  PID  %CPU  %MEM  VSZ  RSS  TTY  STAT  START  TIME  COMMAND

ps -ef: -e = every process, -f = full format. Its columns:

UID  PID  PPID  C  STIME  TTY  TIME  CMD

The one difference that matters day to day: -ef shows PPID ("who started this?"), aux shows %CPU and %MEM ("what is eating the box?").

Your own columns: -o

ps -eo pid,ppid,user,rss,vsz,stat,etime,comm --sort=-rss | head

You do not have to remember column names: ps L lists every one.

A trailing = on a column removes its header line, which is what makes ps usable inside $(...) and scripts:

ps -p 1 -o comm=          -> systemd          (-p 1 = only PID 1)
ps -o ppid= -p $$         -> the PID that started your shell

$$ is a shell variable holding your own shell's PID.

STAT: what the process is doing

The first letter of STAT is the process state:

R  running, or runnable: on a CPU, or waiting in the queue for one
S  interruptible sleep: waiting for something (a key press, a network
   packet, a timer) and can be woken by a signal. Most processes, most of
   the time. Perfectly healthy.
D  UNINTERRUPTIBLE sleep: waiting inside the kernel, usually for a disk or
   a network file server. Signals are not delivered - you CANNOT kill it,
   not even with kill -9.
Z  zombie: already exited; only a small record is left, waiting for its
   parent to collect the exit status.
T  stopped: paused (by Ctrl+Z or a stop signal).
I  idle kernel thread: the kernel's own helpers, doing nothing.

A signal is a small message the kernel delivers to a process, like "please stop" or "stop now". Lesson 3.6 covers them; for now, kill -9 <pid> is the "stop now" one.

After the letter come modifiers:

s  session leader (the first process of a login, e.g. your shell)
l  multi-threaded (a thread is one line of work inside a process; a
   process can run many at once)
+  in the foreground of its terminal
<  high priority      N  low priority ("niced")

So Ssl is a normal background service with several threads, sleeping until work arrives. D on its own, on twelve processes, is an incident.

RSS vs VSZ, briefly

Two memory columns, and only one of them is useful:

The orders service on this box is a Java program. Java programs run inside the JVM (Java Virtual Machine), a runtime that reserves large address ranges up front, so a JVM using 600 MB of RAM routinely shows 4 GB of VSZ. It means nothing. Sort by RSS. (Chapter 5 shows why even RSS over-counts.)

What you can now do

Why it helps

ps is the command you type most during process incidents, and a good -o line answers the question directly: "what is using the most memory" (--sort=-rss), "who started this" (ppid, then ps -p PPID), "how long has it been running" (etime). The header-less form (ps -o comm= -p 1) is what monitoring and health-check scripts use.

Reading STAT correctly prevents wrong actions: a pile of D processes means stop killing and find the storage problem; Z means fix the parent. Sorting by VSZ instead of RSS makes a Java service look like a monster when it is fine, which is a classic false alarm in capacity discussions.

Commands in this lesson

ps

FAQ

Why are there two option styles, with and without a dash?

History. ps aux comes from BSD Unix, where options had no dash; ps -ef comes from System V. Linux procps supports both and even GNU long options. They mostly overlap: aux gives %CPU, %MEM, VSZ, RSS and STAT; -ef gives PPID and start time. Mixing styles in one command works but can surprise you, so pick one per command.

Why does %CPU in ps differ from top?

ps computes %CPU as CPU time used divided by the process's whole lifetime, so a process that was busy an hour ago and idle now still shows a high number. top computes it over the last refresh interval, so it shows what is happening now. For "what is eating CPU right now", use top; ps is better for totals and for scripts.

Why does my grep show up in ps output?

ps aux | grep nginx lists the grep process itself, because its command line contains "nginx". Common workarounds are grep [n]ginx, whose command line does not match its own pattern, or better, pgrep -a nginx, which never matches itself and prints PIDs with command lines. In scripts, pgrep or pidof is the clean answer.

What does the trailing = do in ps -o comm=?

It sets the column header to empty, so ps prints no header line. ps -p 1 -o comm= outputs just systemd, which is easy to use in scripts and $(...). You can also rename headers: -o pid,rss=RSS_KB. Add --no-headers to suppress all headers at once.

Is a process in S state a problem?

No. S means interruptible sleep: the process is waiting for something (a network packet, a timer, input) and uses no CPU. A web server waiting for requests is S almost all the time. Worry about S only if the process should be working and is not, and then look at what it is waiting for with /proc/PID/wchan, strace or its logs.

In an interview Junior

How do you find the processes using the most memory (or CPU) on a box?

ps -eo pid,ppid,user,rss,vsz,stat,etime,comm --sort=-rss | head: -e every process, -o exactly these columns, --sort=-rss biggest first, head the top ten. Live, open top and press M to sort by memory or P by CPU.

Sort by RSS, not VSZ. RSS is the physical RAM the process really uses right now; VSZ is every address range it has reserved, including memory it never touched. A Java service using 600 MB of RAM routinely shows 4 GB of VSZ - meaningless.

The two old spellings: ps aux shows %CPU and %MEM ("what is eating the box?"), ps -ef shows the PPID ("who started this?"). A trailing = drops the header, for scripts: ps -o ppid= -p $$.

Also asked: What is the difference between ps aux and ps -ef? · What is the difference between RSS and VSZ? · How do you find the parent of a process?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.