SRE Fundamentals: glossary
192 terms, in plain words, with where each one is taught.
- /proc
- A special directory where Linux shows live information about every process: /proc/<pid>/fd lists its open files, /proc/<pid>/limits its limits.
- 499
- nginx's own status code for "the client closed the connection before I answered" - usually someone who gave up waiting. Lesson 0.8
- 4xx
- Any status code from 400 to 499: the client asked for something wrong (404 Not Found, 401 not logged in). Usually not counted as the service failing. Lesson 0.1
- 502 (Bad Gateway)
- Bad Gateway: the proxy in front could not get a valid answer from the service behind it. Lesson 0.1
- 5xx (5xx errors)
- Any status code from 500 to 599: the server failed the request. For reliability, 5xx is what counts as a failure. 500 Internal Server Error, 502 Bad Gateway and 504 Gateway Timeout (a proxy gave up waiting) are the common ones. Lesson 0.1
- access log (access logs)
- A log file where a web server writes one line per request: time, address asked for, status code, how long it took. Lesson 0.1
- ack (acknowledge, acknowledged, acked)
- Pressing "I have got it" on a page, so it stops escalating to others. Lesson 0.21
- action item (action items)
- A follow-up task from a postmortem: specific, owned by a named person, prioritised, ticketed, and about changing the system.
- actionable
- An alert is actionable when a human can and must do something about it now, and it says what and links a runbook. Lesson 0.21
- alert (alerts, alerting)
- An automatic message a monitoring system sends when a number crosses a line for long enough. Lesson 0.1
- alert fatigue
- What too many useless alerts do to people: they stop trusting pages, and miss the real one. Lesson 0.21
- alert grouping (inhibition, inhibits)
- Merging related alerts into one notification (grouping), or muting one alert while another fires (inhibition), so one problem is one page. Lesson 0.22
- API (APIs)
- Application programming interface: the set of addresses a service offers for other programs to call, like everything under /api/. Lesson 0.8
- argument (arguments)
- Each word after the command name, separated by spaces: what you give the program (a file name, a number). Lesson 0.1
- availability
- The share of valid requests answered successfully. The most common SLI.
- awk
- A small language for text in columns: $1 is the first column, $NF the last, NR the line number. Used here for counting and arithmetic. Lesson 0.1
- backend (backends)
- A service that another service calls behind the scenes to do part of the work. Lesson 0.1
- bimodal
- A set of values that falls into two separate groups (fast cache hits and slow database reads). An average of it matches neither group. Lesson 0.1
- blameless
- A postmortem that assumes everyone acted reasonably with what they knew, and looks for what in the system allowed the failure, not who to blame. Lesson 0.30
- blast radius
- How much of the system and how many users one failure can hurt. Lesson 0.30
- burn rate
- How fast the error budget is being spent, compared with spending it exactly evenly: error ratio / (1 - SLO). 1 lasts the whole window; 14.4 spends 2% per hour. Lesson 0.16
- canary (canary stage, canary release)
- The first, small step of a release: the new version gets a few percent of the traffic, so a bad release hurts few users and is caught early.
- cardinality
- How many distinct values something you group by has. Grouping by order number gives thousands of one-request groups: too high. Lesson 0.9
- cause-based alert (cause-based alerts, cause-based)
- An alert on a possible reason for trouble (high CPU, a restart, a full disk) rather than on what users feel. Most causes hurt nobody, so these belong on dashboards. Lesson 0.21
- certificate (certificates, TLS certificate)
- The file that proves a website's identity for HTTPS. It has an end date; an expired one breaks the site. Lesson 0.21
- chaos test (chaos experiment, chaos experiments, chaos testing)
- Breaking something on purpose, in a controlled way, to check the system copes. Lesson 0.16
- command (commands)
- A program name followed by what you give it, typed at the prompt: head -1 access.log. Lesson 0.1
- comms lead
- The incident role that updates the status page, support and stakeholders on a fixed schedule. Lesson 0.29
- connection pool (database connection pool, connection pools)
- A fixed set of open connections to a database that requests borrow and hand back. When all are in use, the next request waits. Lesson 0.1
- contributing factor (contributing factors)
- One of the several things that lined up to cause an incident: the trigger plus the gaps that let it hurt. Used instead of a single root cause. Lesson 0.30
- correctness
- Whether the answer was right, not just fast and successful. A 200 with the wrong price is available but incorrect. Lesson 0.8
- counterfactual (counterfactuals)
- "Should have", "failed to": judging a past decision with hindsight the person did not have. Lesson 0.30
- CPU (CPUs)
- The chip that executes programs. "15% CPU" means it is busy 15% of the time; a box can have several CPUs (cores). Lesson 0.1
- crash loop
- A program that crashes, is restarted, and crashes again, over and over. Lesson 0.35
- dashboard (dashboards)
- A screen of graphs showing a service's metrics, looked at while diagnosing a problem. Lesson 0.1
- decommission (decommissioned)
- Switch a server off for good and remove it. Anything still running on it disappears with it.
- dependency (dependencies)
- Another service yours needs in order to work, like its database. You cannot be more available than what you depend on. Lesson 0.5
- deploy (deployment, rollout)
- Putting a new version of a program into service for real users. Lesson 0.2
- detection time (time to detect, MTTD)
- How long from the start of a problem until an alert or a person notices it. Lesson 0.21
- directory (directories, folder, folders)
- A folder: a named container for files and other directories. Lesson 0.1
- DNS
- The internet's address book: it turns names like shop.example into the numeric addresses of servers. Lesson 0.35
- drop-in (drop-ins, override.conf)
- A small extra file that changes a few settings of a systemd unit without editing the original file. It lives in a folder named after the unit plus
.d.sudo systemctl edit NAMEcreates one;systemctl revert NAMEremoves it (chapter 2). - dry run
- A mode where a script only prints what it would do, without doing it. Lesson 0.35
- EMFILE (Too many open files)
- The error a process gets when it has used all its file descriptors: the next file or connection it tries to open fails.
- engineering
- Work that leaves the service permanently better: automation, design, SLOs, fixing root causes. Lesson 0.2
- EOF
- End Of File. In a heredoc it is just the marker word: the shell feeds the lines that follow to the command until a line that says exactly EOF. Any word works as the marker; EOF is the habit. Quoting it (
<<'EOF') stops the shell from expanding$variablesinside. - error budget (error budgets)
- The amount of failure an SLO allows: 1 - SLO. A 99.9% SLO over 30 days allows 0.1% of requests, about 43 minutes, to fail.
- error budget policy
- A short written agreement, made in advance, on what happens when the budget is spent - typically a feature freeze.
- error rate (error ratio)
- The share of requests that failed (for example, 5xx / all requests) over some stretch of time. Lesson 0.1
- executable
- A file marked as allowed to run as a program. chmod 755 FILE marks it.
- failover (fail over, failovers)
- Switching to a backup system, data centre or copy when the main one fails. Lesson 0.16
- fan-out (fans out)
- One user action calling many services at once; the slowest of them decides how long the user waits. Lesson 0.2
- feature flag (feature flags)
- A switch in a service's settings that turns a feature on or off without a new release. Lesson 0.29
- file (files)
- A named piece of stored data: a log, a settings file, a program. Lesson 0.1
- file descriptor (file descriptors, fd, fds)
- A numbered handle a process holds for each open file or network connection. The system caps how many one process may hold (often 1024).
- five whys
- Asking "why?" again at each answer to walk down a chain of causes. Useful for one branch, misleading as the whole analysis. Lesson 0.31
- flag (flags)
- An argument starting with - that switches a behaviour on: -n in sort -n means "sort as numbers". Lesson 0.1
- flapping
- An alert firing and clearing over and over as a number wobbles around its line. Lesson 0.21
- freeze (feature freeze, release freeze)
- A pause on releasing new features until the service is back inside its SLO. Fixes and reliability work still ship. Lesson 0.16
- function (shell function, functions)
- A named, reusable mini-command you define once in the shell: pct() { ...; } lets you type pct 0.99 later. Lesson 0.2
- game day (game days)
- A planned practice session where the team breaks something on purpose and rehearses the response. Lesson 0.17
- golden signals (four golden signals, golden signal)
- The four things to measure first on any user-facing service: latency, traffic, errors and saturation. Lesson 0.1
- good event (good events, good requests)
- A valid event the service handled well: for availability, one that did not return 5xx. Lesson 0.8
- grep
- A command that prints only the lines containing a piece of text. Lesson 0.1
- handover (handoff, handoffs)
- Passing on-call duty, or an incident role, explicitly to the next person, with the current state. Lesson 0.22
- HEAD request (HEAD)
- A request that asks only "are you there?" without asking for the content itself. Uptime checkers use it.
- health check (health checks, /healthz, probe)
- An automatic "are you alive?" request a monitoring tool sends every few seconds. Not a user, so it is left out of SLIs. A health check that does not touch what can fail (the database) says "healthy" during an outage - checkout's /healthz did. Lesson 0.2
- heap
- The working memory a program uses for its data. Saturation data compares heap used with the maximum allowed.
- heartbeat (heartbeats)
- A small "I am alive" message sent every few seconds; missing several makes the other side assume you are gone.
- heredoc (here-doc, here document)
- A way to give a command several lines of input: cat > FILE <<EOF, the lines, then EOF alone on a line.
- histogram (histograms)
- Counts of how many values fell at or below each of a fixed set of limits (buckets). Counts add up correctly across copies and minutes, unlike percentiles. Lesson 0.2
- home directory
- Your own directory on the server, where your files live. ~ is short for it. Lesson 0.1
- HR (human resources)
- Human resources: the company department for hiring, contracts and people matters. Lesson 0.30
- HTTP
- The protocol (agreed message format) that browsers and web services use for requests and responses. HTTPS is the same thing, encrypted. Lesson 0.1
- idempotent
- Giving the same result however many times you run it. Safe automation is idempotent. Lesson 0.35
- incident (incidents)
- An unplanned event that hurts users, run as a coordinated effort once declared. Each gets a number, like #4471. Lesson 0.1
- incident commander (IC)
- The person who owns an incident: keeps the big picture, assigns roles and decides. Does not debug. Lesson 0.29
- instance (instances)
- One of several identical running copies of a service. The proxy spreads requests across them. Lesson 0.2
- interpolate (interpolation, interpolated)
- Estimate a value between two known points by assuming a straight line between them. How percentiles are guessed from histogram buckets. Lesson 0.2
- ISO 8601 (ISO timestamp, ISO timestamps, ISO time)
- The standard timestamp layout 2026-09-22T18:36:00: date, T, time. Such timestamps sort correctly as plain text.
- ISP (internet provider, internet service provider)
- Internet service provider: the company your home or phone connection goes through before it reaches the internet.
- journal (journalctl)
- The server's central log: programs send their messages there, and journalctl reads them (-t TAG for one program's messages).
- jq
- A command that reads JSON and picks, filters and prints fields from it. Lesson 0.1
- JSON
- A text format for records: {"status": 200, "request_time": 0.042}. Each field has a name and a value. Lesson 0.1
- latency
- How long a request takes, from arriving to its response leaving. Measured with percentiles, not averages. Lesson 0.1
- latent condition (latent conditions)
- A weakness waiting in a system (a stale runbook, a missing check) until something triggers it. Lesson 0.30
- leading indicator
- A number that moves before trouble shows up. Saturation is one: it rises before latency and errors do. Lesson 0.1
- ledger
- The service that records account balances and moves money. It deserves more reliability than, say, search.
- Linux
- An operating system: the base software that runs a computer and every program on it. The server in this course runs Ubuntu Linux. Lesson 0.1
- Little's law (Littles law)
- In-flight items = arrival rate x time each spends inside. For a pool: connections in use = requests per second x how long each holds one. Lesson 0.5
- load average
- Roughly the average number of tasks that wanted a CPU or were stuck waiting for the disk, over the last 1, 5 and 15 minutes. Lesson 0.5
- load balancer (load balancers)
- The first machine that receives users' requests and spreads them across the copies of a service. Lesson 0.8
- log (log file, logs, log files)
- A text file where a program records what it did, one event per line, with a timestamp. Lesson 0.1
- markdown
- A plain-text format where # starts a heading and - starts a bullet. Postmortems and these lessons are written in it. Lesson 0.30
- median
- The middle value of a sorted list (p50): half the values are below it, half above. Lesson 0.1
- memory leak
- A bug where a program keeps taking memory and never gives it back, until it has to be restarted.
- message queue (broker)
- A server that holds work messages until a worker takes them. The broker checks with heartbeats that each worker is still alive.
- metric (metrics)
- A number measured over and over (requests per second, % of errors), so you can watch it change over time. Lesson 0.1
- migration (migrations)
- Moving data or a system to a new setup (a new database, a new server). Risky, so it spends budget. Lesson 0.8
- mitigate (mitigation, mitigated)
- Reduce the harm to users now (roll back, fail over) without yet fixing the underlying bug. Lesson 0.29
- MTTR (MTTA, time to mitigate, time to resolve)
- Mean time to recover (or resolve): the average time from impact start to users being fine again. MTTA is the same for acknowledging a page. Averages across incidents are a weak metric; look at the breakdown per incident. Lesson 0.29
- multi-window burn-rate alert (multi-window, burn-rate alert, burn-rate alerts)
- An SLO alert that fires only when the burn rate is too high over both a long window (significance) and a short one (fast reset). Lesson 0.21
- nano
- A simple text editor inside the terminal. Ctrl+O then Enter saves, Ctrl+X quits.
- near miss (near misses)
- Something that almost went wrong but did not. Worth reporting, because it shows the same holes as a real incident. Lesson 0.29
- nginx
- A popular web server. On this box it receives every request first and passes it on to the right service. Lesson 0.1
- nines (three nines, four nines)
- How availability is spoken about: 99.9% is "three nines". Each extra nine allows ten times less failure. Lesson 0.8
- on-call (on call, on-call shift)
- Taking turns being the engineer whose phone rings when something breaks, day or night. Lesson 0.21
- on-call rotation (rotation)
- The group of engineers who take turns being on-call, usually a week each.
- ops lead
- The one person in an incident who changes production, so changes do not collide. Lesson 0.29
- outage (outages)
- A period when a service does not work for its users, fully or partly. Lesson 0.1
- overhead
- Necessary work that is neither toil nor engineering: meetings, planning, training, HR. Lesson 0.30
- P0 (P1, P2)
- Priority levels for tasks and bugs: P0 is the most urgent, then P1, P2. Lesson 0.16
- page (paged, paging)
- An alert that interrupts a human right now - a call or push notification, even at 3am. Only for urgent, user-visible problems.
- pager export
- The list of every page sent, with when it fired, was acked and resolved. Used to measure alert quality. Lesson 0.22
- path (paths)
- Where a file is, written as directories separated by /: /var/log/nginx/access.log starts at the top (/) and goes down. Lesson 0.1
- pending
- Waiting for a resource. "pending=9" on a pool means nine requests are waiting for a free connection: saturation. Lesson 0.1
- percentile (percentiles, p50, p90)
- A position in the sorted list of values: p99 is the value 99% of requests are at or below. p50 is the median. Nearest rank: sort the values, take the one at position ceil(N x q). Percentiles cannot be averaged. Lesson 0.1
- PID (PIDs, process ID)
- Process ID: the number the system gives each running process. pgrep prints it.
- pipe (pipes)
- A | between two commands: the output of the first becomes the input of the second. Lesson 0.1
- postmortem (postmortems, post-mortem)
- A written review of an incident: what happened, its impact, why the system allowed it, and what will change. Lesson 0.16
- precision
- Of the times an alert fired, the share that were real problems. Low precision = noisy pages. Lesson 0.21
- prober (synthetic prober, synthetic probe, synthetic transaction)
- A small program that makes real requests on a schedule, like a fake user, and checks the answers. Lesson 0.8
- process (processes)
- A running program. The same program started twice is two processes.
- product owner
- The person who decides what a product should do and in what order. Lesson 0.16
- production (prod)
- The live system real users are on, as opposed to test copies. Lesson 0.29
- Prometheus
- A widely used monitoring tool that collects metrics and evaluates alert rules. Lesson 0.2
- promotion
- Giving a canary version to all users once it has proved itself.
- prompt
- The text the shell shows when it is ready for your next command, like learner@oncall-lab:~$. Lesson 0.1
- PromQL
- The query language of the Prometheus monitoring tool; the alert rules in this chapter are written in it. Lesson 0.21
- proxy (edge proxy, reverse proxy)
- A program that sits in front of a service, receives requests for it and passes them on. It sees every request, even ones the service never answered. Lesson 0.1
- recall
- Of the real problems, the share that made the alert fire. Low recall = missed outages. Lesson 0.21
- RED
- Rate, Errors, Duration: the golden signals without saturation, measured per service. Lesson 0.1
- redirection (redirect)
- A > after a command sends its output into a file instead of the screen, replacing what the file held. Lesson 0.1
- reload (systemctl reload)
- Make a running program (here nginx) re-read its settings without stopping it.
- request (requests)
- One message sent to a service asking for one thing ("show my cart"). The service sends back a response. Lesson 0.1
- reset time
- How long an alert keeps firing after the problem is fixed. Lesson 0.21
- resolved (resolves)
- An alert resolves when its condition is no longer true and it stops firing. Lesson 0.21
- resource (resources)
- Anything a service needs a share of to work: CPU, memory, disk, network, a pool of connections, open-file slots. Lesson 0.1
- response (responses)
- What a service sends back for a request: the answer, plus a status code saying how it went. Lesson 0.1
- Restart=on-failure
- A systemd setting that restarts a service automatically when it crashes. It removes restart toil but can hide a crash loop.
- rollback (roll back, rolled back, rolling back)
- Replacing a bad release with the previous version again. The most common first fix in an incident. Lesson 0.16
- root (root user)
- The administrator account: it can read and change everything on the server.
- root cause (root causes)
- The idea that an incident has one single cause. Misleading for complex systems, which fail when several factors line up. Lesson 0.30
- route (routes)
- The address a request asks for inside a service, like /api/cart. Lesson 0.1
- RUM (real user monitoring)
- Real user monitoring: measuring speed and errors in the user's own browser or app, then sending the numbers home. The most honest view, and the noisiest. Lesson 0.9
- runbook (runbooks)
- A written step-by-step guide for handling one kind of problem, linked from the alert. Lesson 0.21
- saturation
- How full a service is: how close its most limited resource (workers, connections, memory, open files) is to running out. It warns before errors start. Lesson 0.1
- scribe
- The incident role that records every decision and action with a timestamp - the raw material of the postmortem. Lesson 0.29
- script (scripts)
- A text file of commands run in one go. Its first line (#!/usr/bin/env bash) says which program runs it. Lesson 0.22
- server (servers)
- A computer that runs services for others, usually in a data centre, with no screen or keyboard of its own. You reach it over the network. Lesson 0.1
- service (services)
- A program that other programs or people send work to, like "checkout", which turns a shopping cart into an order. Lesson 0.1
- severity (SEV, SEV-1, SEV-2)
- How serious an alert or incident is. It decides where an alert goes and who joins an incident (SEV-1 worst). Lesson 0.21
- shadowing
- Being on-call next to the person on duty to watch and learn, without being the one responsible.
- shed load (load shedding)
- Deliberately refusing some requests or switching off expensive extras so the rest keep working. Lesson 0.29
- shell
- The program inside the terminal that reads your commands, runs them and shows the prompt. On this server it is bash. Lesson 0.1
- shift (shifts)
- A block of on-call time, for example 12 hours: 30 days is 60 shifts. Lesson 0.21
- SLA (SLAs, service level agreement)
- Service Level Agreement: a contract promising a level of service to a customer, with a consequence (money back) if it is missed. Set looser than the SLO. Lesson 0.8
- SLI (SLIs, service level indicator)
- Service Level Indicator: a measurement of how well a service is doing, written as good events / valid events (e.g. 96.0% of requests succeeded).
- SLO (SLOs, service level objective)
- Service Level Objective: a target for an SLI over a window, e.g. 99.9% of valid requests succeed over 30 days.
- SLO window (rolling window, calendar window, rolling 30 days)
- The stretch of time an SLI or SLO is measured over. Rolling = always the last 30 days; calendar = this month, reset on the 1st. Lesson 0.8
- sort -n
- Sort lines as numbers, smallest first. Plain sort compares text character by character, which puts 1000 before 120. Lesson 0.1
- sprint
- A fixed block of planned team work, usually two weeks.
- SRE (Site Reliability Engineering)
- Site Reliability Engineering: the job of keeping a software service working for its users, with measurement and automation instead of heroics. The name and most of the practices come from Google, whose "SRE book" (2016) and "SRE workbook" (2018) this chapter quotes. Lesson 0.1
- staging
- A test copy of production where changes are tried before they reach real users.
- status code (HTTP status code, status codes, HTTP status)
- The three-digit number on every HTTP response: 2xx success, 3xx go elsewhere, 4xx the client got it wrong, 5xx the server failed. Lesson 0.1
- status page
- The public web page that tells customers what is broken and when the next update comes. Lesson 0.29
- sudo
- Put before a command to run just that command as root. It asks for your own password.
- swap
- Disk space used as overflow memory. Much slower than real memory, so heavy swapping means memory is saturated. Lesson 0.5
- Swiss cheese model (Swiss cheese)
- Each layer of defence has holes; an incident happens when the holes in every layer line up. Lesson 0.31
- symptom (symptoms, symptom-based)
- What users experience: errors, slowness. Alerts should fire on symptoms. Lesson 0.21
- systemd (systemctl)
- The program that starts and supervises everything on a Linux server. systemctl is the command you give it orders with. Lesson 0.35
- tail latency (long tail)
- The slow end of the sorted list of request times (p99 and beyond). Small in share, large in how much users feel it. Lesson 0.1
- terminal
- A window where you type text commands to a computer and read their text output. Lesson 0.1
- thread (threads, worker threads)
- A worker inside a program that handles one request at a time. A program has a maximum number of them.
- throughput
- How much work gets done per second: requests per second, bytes per second. Latency is how long one piece of work takes; the two often move in opposite directions under load.
- ticket (tickets)
- A task in the team's tracking system, handled in working hours. Where non-urgent alerts should go. Lesson 0.5
- timeout (timeouts, timed out)
- Giving up on an answer after a fixed time. A request that times out is both slow and failed. Lesson 0.1
- timer (.timer, systemd timer)
- A systemd unit that runs a service on a schedule, such as OnCalendar=daily. Lesson 0.35
- toil
- Operational work that is manual, repetitive, automatable, interrupt-driven, leaves nothing better, and grows with the service. Lesson 0.30
- traffic
- How much demand arrives at a service, usually requests per second or per minute. Lesson 0.1
- TSV
- Tab-separated values: a text table with one row per line and columns separated by tab characters.
- uniq -c
- Squash repeated neighbouring lines into one and print how many there were. After sort, it counts each distinct value. Lesson 0.1
- unit file (unit files, systemd unit)
- A small text file describing one job to systemd: a .service says what to run, a .timer says when.
- upstream (upstream_addr)
- The service behind nginx that actually handles a request. upstream_addr in the log says which copy answered.
- USE method (USE)
- Utilisation, Saturation, Errors, checked for every resource (CPU, memory, disk, pool) to find which one is the bottleneck. Lesson 0.1
- user agent
- The name a client sends with every request to say what program it is: a browser, a phone app, a monitoring robot.
- user journey (user journeys)
- One thing a user is trying to do, like "search" or "pay". Important journeys get their own SLO. Lesson 0.9
- UTC
- The world reference time zone. Incident times are always written in UTC so people in different countries agree.
- utilisation (utilization)
- How busy a resource is: % of time busy, or used / capacity. Can be 100% and fine, as long as nothing is waiting. Lesson 0.1
- valid event (valid events, valid requests)
- An event that counts in an SLI: a real user request. Probes and monitors are left out. Lesson 0.2
- variable (variables)
- A name that holds a value in the shell: rpm=12 stores it, $rpm reads it back. rpm=$(cmd) stores a command's output. Lesson 0.2
- YAML
- A text format for settings where indentation (spaces at the start of a line) shows what belongs to what.