OnCallReady

Bash Scripting: interview questions

The question you are most likely to get for each topic, a model answer, and what else comes up. From chapter 6 of the course.

Why doesn't set -e catch a failure in foo | bar? Junior

Because a pipeline's exit status is the status of its last stage only. In ./build.sh | tee build.log, tee succeeds, so the whole pipeline "succeeds" and set -e sees nothing to act on - even though the build failed. This is the hole that ships broken releases.

The fix is set -o pipefail, part of the standard header set -euo pipefail: the pipeline then fails if any stage fails. When you need to know which stage failed, bash keeps every stage's status in the PIPESTATUS array (${PIPESTATUS[0]} is the first) - copy it on the very next line, because the next command overwrites it.

It is one of four places set -e stays quiet; the others are commands in an if/while condition, commands followed by && or ||, and local x=$(cmd).

Also asked: Write a script that always cleans up its temporary directory, even if it is interrupted. · Why does local x=$(cmd) swallow the exit code, and how do you fix it? · Why should you quote variables in bash?

What does set -euo pipefail do, and why is it recommended? Junior

Nobody watches a script, and by default bash just runs the next line after a failure - with a missing file, an empty value, the wrong directory. These three options make it stop at the first problem instead:

The full header is #!/usr/bin/env bash (the shebang: find bash on the PATH - not sh, which is dash on Ubuntu), then set -euo pipefail, then IFS=$'\n\t' so values are not split on spaces. It is a safety net, not a guarantee: -e has blind spots.

Also asked: Why use #!/usr/bin/env bash instead of #!/bin/sh on Ubuntu? · What exit code should a script return when something goes wrong, and why does it matter? · What is IFS, and why do scripts change it?

Learn it: 6.1 The header every script gets

In which cases does set -e not stop a script when a command fails? Junior

Four holes:

  1. Conditions - a command after if or while. A failing condition is the point of if, so bash switches -e off there; a real failure (a missing file) looks like "false".
  2. && and || - important_command || echo warning counts as handled. That is why cmd || true means "may fail", and why a stray || echo hides a failure.
  3. Pipelines - every stage except the last: ./build.sh | tee build.log succeeds because tee did. Fix: set -o pipefail; PIPESTATUS has each stage's status.
  4. local x=$(cmd) - local is itself a command, it succeeds, and its status replaces the failed $(cmd). Same with export and declare. Fix: local x on one line, x=$(cmd) on the next; shellcheck reports it as SC2155.

For anything important, check explicitly: if ! do_the_thing; then echo "failed" >&2; exit 1; fi.

Also asked: What is PIPESTATUS, and when would you use it? · Is set -e worth using, given its holes? · What does cmd || true mean in a script?

Learn it: 6.3 What set -e does NOT catch

How does if work in bash? Is it evaluating a boolean expression? Junior

No. if runs a command and branches on its exit status: 0 is true, anything else is false. [[ ]] is just one more command that happens to compute a status. So the command itself can be the test:

if grep -q "^learner:" /etc/passwd; then echo exists; fi
if systemctl is-active --quiet nginx; then ...
if ! mkdir "$dir"; then echo "cannot create $dir" >&2; exit 1; fi

while, && and || work the same way. Tools have quiet modes made for this - grep -q, cmp -s, diff -q - no output, only the status.

Prefer if ! cmd over running cmd and testing $? afterwards (shellcheck SC2181): any line added in between overwrites $?. For several values of one variable, a case is tidier than a chain of ifs.

Also asked: What runs in a subshell in bash, and why does it matter? · Why use printf instead of echo in scripts? · What is the difference between ( ) and { } around commands?

Learn it: 6.5 Exit status, subshells and case

Why should you quote variables in bash? Give an example of what goes wrong. Junior

Because an unquoted expansion goes through two more steps: word splitting (cut at spaces, tabs and newlines) and globbing (any word with *, ? or [...] is replaced by matching filenames).

file="old report.txt"
rm $file      # rm gets two arguments: "old" and "report.txt"
rm "$file"    # rm gets one: "old report.txt"

The first deletes a file called old if one exists, and complains about report.txt. A value containing * turns into a list of files.

The rule: quote every expansion - "$var", "$(cmd)", "${arr[@]}" - and pass a script's arguments on with "$@" (each argument stays one word; $* glues them together). Never loop over $(ls); use a glob: for f in *. Single quotes are for text you want kept exactly - nothing is expanded inside them.

Also asked: What is the difference between single and double quotes? · What is the difference between "$@" and "$*"? · Why is for f in $(ls) wrong?

Learn it: 6.6 Quoting

What is the correct way to read a file line by line in bash, and why is each part there? Junior

while IFS= read -r line; do
  printf '%s\n' "$line"
done < input.txt

Add || [[ -n $line ]] to the condition so a last line without a newline is not skipped. To split fields: while IFS=: read -r user _ uid _ ; do ...; done < /etc/passwd.

Also asked: A script reads its settings with cat config | while read k v, but afterwards every setting is empty. Why? · How do you split a line into fields in bash? · How do you loop safely over filenames that may contain spaces?

Learn it: 6.8 Reading input: while read, IFS and mapfile

What is the difference between [ ] and [[ ]] in bash? Junior

[ is an ordinary command (there is even /usr/bin/[), so its arguments go through word splitting and globbing first. With name="two words", [ $name = "two words" ] fails with "too many arguments"; with an empty variable, [ -n $x ] becomes [ -n ] and is wrongly true.

[[ ]] is bash syntax: bash reads the expression itself, so nothing is split inside it. It also adds glob matching ([[ $f == *.log ]]), regex matching ([[ $env =~ ^(dev|prod)$ ]], captures in BASH_REMATCH, regex unquoted), and &&, ||, ! inside.

Use [[ ]] in bash scripts, [ ] only in #!/bin/sh scripts. In both, = and < compare text ([[ 10 < 9 ]] is true); for numbers use -lt, -eq or (( )).

Also asked: How would you check that an argument is exactly dev or prod? · Why can (( count++ )) kill a script that uses set -e? · Why must the regex after =~ not be quoted?

Learn it: 6.10 [[ ]] versus [ ]

Explain ${var:-default}, ${var:=default}, ${var:?message} and ${var:+alt}. Junior

Without the colon (${var-default}) only "unset" counts, so an empty value is accepted. Why they matter: a script fails fast with a clear message instead of running with an empty value, and they work under set -u.

Also asked: How do you get a file's name, directory and extension without calling basename or dirname? · How do you store a list in an array and loop over it safely? · What is an associative array, and how do you declare one?

Learn it: 6.12 Parameter expansion and arrays

Write a script that always cleans up its temporary directory, even if it fails or is interrupted. Junior

#!/usr/bin/env bash
set -euo pipefail
tmp=$(mktemp -d)
cleanup() { rm -rf "$tmp"; }
trap cleanup EXIT INT TERM
# ... work in "$tmp" ...

The one thing it cannot handle: kill -9. SIGKILL cannot be caught, so the directory stays - that is what -9 means.

Also asked: Why should variables inside a function be declared local? · What is the difference between return and exit in a function? · How does a function hand data back to its caller?

Learn it: 6.15 Functions, traps and mktemp

What are stdin, stdout and stderr, and how do you redirect each? Junior

Every process starts with three file descriptors: 0 stdin (input), 1 stdout (normal output), 2 stderr (errors and diagnostics).

Why two output streams: only stdout goes down a pipe, so errors still reach you instead of being treated as data. In scripts, send your own messages to stderr: echo "error: ..." >&2.

Two gotchas: order matters - 2>&1 copies wherever fd 1 points at that moment, so cmd 2>&1 > log leaves errors on the screen. And exec >> /var/log/nightly.log 2>&1 at the top of a script logs everything after it.

Also asked: What is the difference between cmd > log 2>&1 and cmd 2>&1 > log? · Why does sort data.txt > data.txt destroy the file? · How would you log everything a nightly script does, errors included?

Learn it: 6.17 Redirection and file descriptors

What is the difference between a quoted and an unquoted here-document delimiter? Junior

With an unquoted delimiter (<<EOF), the body is expanded: $var, $(cmd) and backticks are replaced by the shell writing it. With a quoted one (<<'EOF'), the body is passed literally.

name=world
cat <<EOF      # prints: hello world
hello $name
EOF
cat <<'EOF'    # prints: hello $name
hello $name
EOF

Rule: quote the delimiter whenever the text you are writing contains $ - a script, a systemd unit with $MAINPID, a config file. Forget, and the variables are expanded now, usually to empty strings, and you get a broken file with no error.

To write a root-owned file: sudo tee /etc/systemd/system/demo.service > /dev/null <<'EOF' - tee runs as root and opens the file; sudo cat > file would fail because your own shell does the >.

Also asked: How do you write a root-owned file from a script? · What is process substitution, and what problem does < <(cmd) solve? · What does <<- do?

Learn it: 6.18 Here-documents

How do you parse command-line options in a bash script? Junior

With the getopts builtin in a while loop and a case:

while getopts ":e:t:nh" opt; do
  case "$opt" in
    e) env=$OPTARG ;;
    t) tag=$OPTARG ;;
    n) dry=1 ;;
    h) usage ;;
    :) echo "error: -$OPTARG needs an argument" >&2; usage ;;
    \?) echo "error: unknown option -$OPTARG" >&2; usage ;;
  esac
done
shift $((OPTIND - 1))
[[ -n $env ]] || usage

The optstring lists the letters; a colon after a letter means it takes a value, delivered in $OPTARG; a leading colon means you print the errors yourself. shift $((OPTIND - 1)) drops what getopts consumed, so the remaining arguments start at $1. Conventions: usage prints to stderr and exits 2; required options are checked after the loop (getopts has no "required"); no long options - that is a separate getopt program.

Also asked: Why should a usage message go to stderr and exit 2? · What is $OPTIND, and why shift by OPTIND - 1? · What makes a script pleasant and safe for other engineers to run?

Learn it: 6.20 getopts and a usage function

How do you debug a bash script that is not doing what you expect? Junior

Three tools, three questions:

  1. shellcheck script.sh - "is this correct?" Static analysis: it reads the script without running it and points at known bugs, each with a code - SC2086 unquoted variable, SC2155 local x=$(cmd), SC2164 cd without || exit, SC2162 read without -r. It often finds the bug outright.
  2. bash -n script.sh - "does it parse?" Syntax only, runs nothing, so it is safe on a script you have not read.
  3. bash -x script.sh args - "what did it actually do?" Prints every command after expansion with a + in front, on stderr, so you see the empty variable or the split path. set -x ... set +x around one part; export PS4='+ ${BASH_SOURCE}:${LINENO}: ' adds file and line.

Silence a shellcheck false positive narrowly, with # shellcheck disable=SC2086 directly above the line.

Also asked: What does bash -n catch, and what does it miss? · Which shellcheck warnings do you see most, and what do they mean? · How do you trace only one part of a script?

Learn it: 6.22 shellcheck, bash -n and bash -x

Practise these answers with flashcards and labs Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.