OnCallReadyStart learning

Incident Command & Communication: glossary

87 terms, in plain words, with where each one is taught.

3Cs (three Cs)
The SRE Workbook's summary of incident management: coordinate the response, communicate inside and outside, keep control of what is being done. Lesson 46.1
abort condition (abort conditions)
What you agree in advance will make you stop waiting for a decision to work and try the next thing ("if p99 is not under 800 ms 10 minutes after the rollback, we look at the database"). Lesson 46.21
awareness time (became aware, becoming aware)
When the entity became aware of an incident, often the page or the first report. One of the two moments that start the DORA clocks, so it belongs on the timeline. Lesson 46.31
bystander effect
In a group, everyone assumes someone else will act. "Can someone check the database?" in a room of eight means nobody checks it, or three people do. Lesson 46.4
cadence (update cadence)
The fixed rhythm of incident updates, kept whether or not anything changed. The lab's policy: SEV1 and SEV2 every 30 minutes, SEV3 every 60. Lengthening it is fine for a long incident, if an update says so. Lesson 46.8
CAN report (CAN reports)
A short report back in three parts: Condition (what I see), Actions (what I did or am doing), Needs (what I need from you). Lesson 46.4
central counterparty (central counterparties, CCP, CCPs)
A financial market firm that stands between the buyer and the seller of a trade so each side is protected if the other fails. One of the entities that cannot use DORA's weekend rule. Lesson 46.31
close-out checklist
What the IC does before leaving a resolved incident: a final status post, every audience told, a postmortem owner and date, follow-up tickets filed, responders thanked. Lesson 46.23
competent authority (competent authorities)
The national supervisor a financial entity reports major incidents to under DORA; for the biggest banks the report also goes to the European Central Bank. Lesson 46.31
component status (component statuses, partial outage, major outage)
The status a status page shows for each part of the service customers recognise: Operational, Degraded performance, Partial outage, Major outage or Under maintenance. Lesson 46.8
core journey (core journeys)
Something users came to do: log in, search, pay, check out, see their orders. Down means they cannot finish it; degraded means they can, but slowly or with retries. Lesson 46.2
CREATE INDEX (CONCURRENTLY, CREATE INDEX CONCURRENTLY)
The SQL statement that adds an index to a database table. On PostgreSQL without CONCURRENTLY it blocks every write to the table while it runs: a one-way door dressed as a quick fix. Lesson 46.21
credit institution (credit institutions)
The EU legal term for a bank: a firm that takes deposits and grants credit. Lesson 46.31
critical services (critical service)
Under DORA's classification rules: ICT services that support critical or important functions, financial services that need authorisation, or systems hit by a successful malicious access. Lesson 46.31
CTO (CEO, VP, chief technology officer)
Executive job titles: chief technology officer, chief executive officer, vice president. Who noticed an incident changes who you update, not its severity. Lesson 46.2
deputy
In PagerDuty's guide, a second trained incident commander who backs up the first, takes notes and can take over. The first role worth filling after ops on a big incident. Lesson 46.4
DORA (Digital Operational Resilience Act, Regulation (EU) 2022/2554)
The EU's Digital Operational Resilience Act, Regulation (EU) 2022/2554, applied since 17 January 2025. Among other things it makes financial entities manage, classify and report ICT-related incidents. Covers credit institutions, payment and e-money institutions, investment firms, insurers, crypto-asset service providers, trading venues and more (Article 2). Lesson 46.31
DORA metrics (DevOps Research and Assessment)
The software delivery metrics published by Google's DevOps Research and Assessment programme. Nothing to do with the EU regulation of the same name. Lesson 46.28
escalation policy (escalation policies)
A team's rule for who is paged first (level 1, the primary on-call), who next if nobody acknowledges in time (level 2), and so on. incident oncall shows every team's policy and its timeout in the lab. Lesson 46.1
escalation timeout (escalates after)
How long one level of an escalation policy waits for an acknowledgement before the pager moves to the next level by itself. PagerDuty recommends 5 minutes. Lesson 46.15
EUR
The code for the euro, the currency DORA's money thresholds are written in (for example EUR 100,000). Lesson 46.31
expected response time
How quickly an on-call engineer is expected to respond to a page. Google's SRE book gives 5 minutes for user-facing or time-critical services, 30 for less time-sensitive ones. Lesson 46.15
failed deployment recovery time (time to restore service)
The DORA metric that replaced "time to restore service" in the 2023 State of DevOps report, narrowed to failures caused by a deployment. Lesson 46.28
FEMA (Federal Emergency Management Agency)
The US Federal Emergency Management Agency, which trains responders in ICS and NIMS. Lesson 46.1
final report
The last DORA report: no later than one month after the latest intermediate report, once the root-cause analysis is done and real impact figures replace the estimates. Lesson 46.31
FIRESCOPE
Firefighting Resources of California Organized for Potential Emergencies: the group formed in 1972, after the 1970 Southern California wildfires, that built the Incident Command System. Lesson 46.1
fix-time promise (ETA, promised fix time)
Saying when it will be fixed ("fixed in 10 minutes"). A guess that becomes a deadline once posted; promise the next update instead. Lesson 46.8
fixation
Becoming sure of one theory early and reading every graph as proof of it. Keep several hypotheses alive and test the cheapest first. Lesson 46.21
follow the sun (follow-the-sun)
Splitting on-call by time zone so each team covers its own daytime and nobody is woken at night. Lesson 46.15
functional escalation
Bringing in different expertise, such as paging the database on-call. Nobody's authority changes. Lesson 46.15
fundamental attribution error
Explaining other people's mistakes by who they are ("careless") and our own by the situation ("the pipeline let me"). Lesson 46.23
GDPR (General Data Protection Regulation)
The EU's data protection law. A personal data breach must reach the data protection authority within 72 hours of becoming aware of it (Article 33). Lesson 46.31
generic mitigation (generic mitigations)
An action that helps for many causes without knowing which one it is: roll back the last change, drain traffic from a bad region, turn a feature flag off, add capacity. Lesson 46.21
Goodhart's law
"When a measure becomes a target, it ceases to be a good measure" (Marilyn Strathern's phrasing). Target MTTR and incidents get closed early; target fewer incidents and people stop declaring. Lesson 46.28
handoff briefing
The five things the next IC needs: impact now, what we know (and what is ruled out), everything in flight with its owner, when the next update is due, and the open decisions and risks. Lesson 46.15
hero mode
The most experienced person grabs the keyboard and fixes a problem alone, then tells everyone afterwards. It often works, and fails when nobody holds the whole picture, changes collide, nobody outside hears anything, or the hero is away. Lesson 46.1
hierarchical escalation
Going up to a manager or executive for a decision or resources you cannot give yourself: emergency spending, taking a region offline, a public statement. Lesson 46.15
hindsight bias
After the fact, the warning signs look obvious, though they were not at the time. It makes blame feel fair when it is not. Lesson 46.23
How Complex Systems Fail
A short paper by Richard Cook listing how failure works in complex systems, including that attributing an accident to one root cause is fundamentally wrong and that hindsight biases judgements of people. Lesson 46.23
IC loop (size-up, size-up, stabilize, update, verify)
The four steps an incident commander repeats until it is over: size-up (impact, what we know, who is here), stabilize (no uncoordinated changes, the best mitigation owned), update (every audience within the cadence), verify (did the last action work). Lesson 46.4
ICT (ICT-related incident, ICT-related incidents)
Information and communication technology: the systems and networks a financial entity runs on. An ICT-related incident is an outage or attack on them. Lesson 46.31
imminent impact (imminent)
Harm that has not reached users yet but will soon, such as a disk that fills in 40 minutes. You set the severity by the impact it is about to have. Lesson 46.2
impact matrix
A severity table with two questions: how deep the harm is (core journey down, degraded, non-core, no user impact yet) and how wide (most users, a subset or region, a few). incident matrix shows the lab's. Lesson 46.2
INC (INC-4519)
The prefix of an incident number in the lab, as in INC-4519: every incident gets one, so people, tickets and the postmortem can refer to it. Lesson 46.1
incident (command) (incident status, incident log, incident declare)
(simulator) The terminal command that drives the incident room: incident status prints the card, incident declare --sev N "summary" declares, incident wait N lets N minutes of incident time pass. incident help lists the rest. Lesson 46.1
incident card
The summary at the top of an incident: number and title, state and severity, the clock, the alert, the impact now, who holds each role, who is in the channel, the last and next update, decisions and notes. Lesson 46.1
incident check
(simulator) incident check AUDIENCE "draft" runs the lab's reviewer on a draft (an update, a handoff, an action item) without posting it: missing impact or next time, jargon for the reader, blame, a fix-time promise. Lesson 46.8
Incident Command System (ICS)
One standard way to organise any emergency, whoever turns up: common words, roles that grow and shrink, one person in charge. Built by FIRESCOPE for firefighting; software incident command is based on it. Google's Managing Incidents chapter and PagerDuty's Incident Response guide both say they are built on ICS. Lesson 46.1
Incident console
(simulator) The Console view for chapter 46: the incident channel, the incident card, a button for every action and the status page as customers see it. Each button shows the incident command it runs. Lesson 46.1
incident room
(simulator) The lab's stand-in for a chat channel, a paging tool and a status page in one place, with its own incident clock. Driven from the Incident console or the incident command. Lesson 46.1
initial notification
The first DORA report on a major incident: as early as possible, within 4 hours of classifying it as major and no later than 24 hours after becoming aware of it. Lesson 46.31
intermediate report
The second DORA report: within 72 hours of the initial notification, even if nothing changed, and updated whenever the status changes. Lesson 46.31
ITS (implementing technical standards)
Implementing technical standards: binding EU rules on how to do something. For DORA, ITS 2025/302 sets the standard report forms and templates. Lesson 46.31
just culture
Balancing safety and accountability by letting people give a full account of what they did, saw and expected, without fear of punishment, so the organisation can learn. John Allspaw, then at Etsy, 2012. Lesson 46.23
live incident state document
Google's name for the running record of an incident: what was observed, decided and done, with times. Keeping it is the IC's responsibility, usually through a scribe. Lesson 46.4
Under DORA, an incident that affected critical services and either involved a successful malicious unauthorised access that may lose data, or met two or more materiality thresholds. It must be reported to the competent authority. Lesson 46.31
major incident (major incidents)
An incident serious enough for the full incident process. In PagerDuty's scale, anything at SEV-2 or above. Not the same as a DORA "major ICT-related incident" (46.31), which is decided by regulatory thresholds. Lesson 46.2
materiality threshold (materiality thresholds)
One of DORA's tests for a major incident: more than 10% of a service's clients, a critical service down over 2 hours, two or more Member States, data losses, costs over 100,000 euros, reputational impact. Lesson 46.31
MTT metrics (MTT-family, MTT-metric, MTT-metrics)
The family of mean-time-to metrics: MTTD (detect), MTTA (acknowledge), MTTM (mitigate) and MTTR (whose R means different things to different people). Lesson 46.28
MTTM (mean time to mitigate)
Mean time to mitigate: the average time from the start of impact until users stop being hurt. Lesson 46.28
next update time (next update)
The time, with its zone, at which an audience will hear from you again. The one promise in an update you fully control. Lesson 46.8
NIMS (National Incident Management System)
The US National Incident Management System, which made ICS national doctrine. First published in March 2004; the third edition dates from October 2017. Lesson 46.1
NIS2 (NIS2 directive)
An EU directive on the cybersecurity of essential and important entities in many sectors, with its own incident reporting regime. Lesson 46.31
non-action (non-actions)
An action item that changes nothing anyone could check, like "be more careful" or "improve monitoring". Lesson 46.23
on-call load (pages per shift)
How much on-call costs the people on it: pages per shift, how many were actionable, pages out of hours, time on incidents. Google aims for at most 25% of SRE time on call and no more than two incidents per 12-hour shift on average. Lesson 46.28
one-way door (one-way doors)
A decision that is consequential and hard or impossible to reverse, such as restoring a database from backup or deleting data. Stop, think and often escalate. From Amazon's 2015 shareholder letter. Lesson 46.21
OODA loop (OODA, observe, orient, decide, act)
Colonel John Boyd's loop for acting under uncertainty: observe, orient, decide, act, then observe again. Whoever runs it faster than the situation changes stays ahead. Lesson 46.21
operational period (operational periods)
In ICS, the stretch of time (typically 12 to 24 hours) a plan holds before it is reviewed. The software version is "next update at 20:15" and rotating the IC every few hours. Lesson 46.1
polling for strong objections (strong objections)
Deciding without waiting for everyone to agree: state a proposal, ask "Are there any strong objections?", pause, then "hearing none, we proceed". Lesson 46.4
postmortem trigger (postmortem triggers)
A condition that makes a postmortem required. Google lists user-visible downtime beyond a threshold, any data loss, on-call intervention such as a rollback, a long resolution time, and a monitoring failure. Lesson 46.23
priority action (priority actions)
Atlassian's name for the postmortem follow-ups that carry a deadline, an internal target of 4 or 8 weeks depending on the service. Lesson 46.23
proximate cause (proximate causes)
In Atlassian's handbook, a cause that directly led to this incident, as opposed to a root cause: the place in the chain where a change prevents the whole class of incident. Lesson 46.23
recurring incident (recurring incidents)
Under DORA, incidents that are not major alone but happened at least twice within 6 months with the same apparent root cause and together meet the criteria. They are reported as one major incident. Lesson 46.31
resolved (incident)
In Atlassian's words, an incident is resolved when the current or imminent business impact has ended. Later than mitigated: not when the fix is merged, not when the graph first turns green. Lesson 46.23
rollback bias
The default of rolling back a change that went out shortly before the impact began, without first proving it was the cause. Check it is safe and the timing really matches. Lesson 46.21
RTS (regulatory technical standards)
Regulatory technical standards: binding EU detail rules written by the European supervisory authorities. For DORA, RTS 2024/1772 sets the classification thresholds and RTS 2025/301 the report contents and time limits. Lesson 46.31
second story (first story)
The first story is "human error caused the outage"; the second asks what about the system made that error easy, likely or invisible. Lesson 46.23
shallow data
John Allspaw's name for counts like incident duration, frequency and severity: valid, but they generate very little insight compared with what the reviews say. Lesson 46.28
SMART
Specific, measurable, achievable, relevant, time-bound: a common checklist for goals and action items. Lesson 46.23
span of control
How many people one supervisor can direct well. NIMS gives one supervisor to five people as the optimum; above that, add a layer. FEMA's training says three to seven. Lesson 46.1
status page states
Atlassian Statuspage's four incident states: Investigating (something is wrong), Identified (we know why), Monitoring (a fix is in, we are watching), Resolved (it is over). Lesson 46.8
time box (time-boxed, time boxes)
A fixed time by which you will check on a task or a decision ("report back in 5 minutes"). It tells the person how deep to go, and tells you when silence means trouble. Lesson 46.4
transfer of command (establishment and transfer of command)
The ICS rule that someone explicitly takes command, and that a handoff is a briefing plus telling everyone who is in charge now. Lesson 46.1
two-way door (two-way doors)
A decision you can walk back from if it is wrong, such as rolling back a release or turning a flag off. Decide it fast. Lesson 46.21
unity of command
Every person reports to and takes direction from exactly one person. In an incident: one incident commander, and only the ops lead's people change production. Lesson 46.1
VOID (Verica Open Incident Database)
The Verica Open Incident Database, a collection of public incident reports. Its reports found incident durations heavily skewed and not correlated with severity. Lesson 46.28
weekend rule (weekend relief)
Under DORA, a report due on a weekend or bank holiday may go in by noon of the next working day. It does not apply to credit institutions, central counterparties, trading venue operators or NIS2 essential or important entities. Lesson 46.31