OnCallReady

Lesson 6.6 · Bash Scripting · 12 min read

Quoting

In plain words

Imagine giving a list of names to someone who cuts the list wherever they see a space. "Old Report" becomes two people, "Old" and "Report". And if a name has a star in it, they replace it with everyone in the room whose name fits. Putting the name in quotes is like putting it in an envelope: the cutter leaves envelopes alone.

That is word splitting and globbing. rm $file with file="old report.txt" becomes rm old report.txt, two files. rm "$file" removes one. "$@" passes every argument in its own envelope; $* pours them all into one bag. Single quotes are sealed envelopes where nothing inside changes; double quotes let $var be filled in but keep the result in one piece.

Why this lesson

A backup script is told to delete old report.txt. It deletes a file called old instead, and complains that report.txt does not exist. Nothing is wrong with the logic - one pair of quotes is missing. This is the most common bash bug there is.

What you need to know already: variables and $name (6.1); what a command's arguments are - the words after the command name, e.g. rm a b gives rm two arguments (1.5).

Unquoted expansion does two things you did not ask for

file="old report.txt"
rm $file        # rm gets "old" and "report.txt"  <- TWO arguments
rm "$file"      # rm gets "old report.txt"        <- one

When bash expands an unquoted $var, it then does two more steps to the value:

  1. Word splitting - it cuts the value into separate words wherever it finds a character from IFS (space, tab, newline - 6.1).
  2. Globbing - each word that contains *, ? or [...] is treated as a glob, a filename pattern (*.log = every name ending in .log), and replaced by the matching file names.

So a value containing a space becomes two arguments, and a value containing * becomes a list of filenames. Double quotes "..." switch both steps off: the value stays exactly one argument.

pattern="*.log"
echo $pattern       # expands to every .log file in the directory
echo "$pattern"     # prints *.log

The rule: quote every expansion. "$var", "$(cmd)", "${arr[@]}" (an array, 6.12). The exceptions are rare enough to think about one at a time when they come up.

A script's own arguments: $1, $@ and $*

When you run ./deploy.sh prod "my app", the script receives its arguments in positional parameters: $1 is prod, $2 is my app, $# is how many there are (2), and $0 is the script's own name. Two special forms mean "all of them":

"$@"    each argument stays a separate word, spaces preserved.  ALWAYS this.
"$*"    all arguments glued into ONE string, joined by a space.
$@      unquoted: word-splits every argument again. Never useful.

A wrapper is a small script or function that adds something and then calls another command with the same arguments:

wrapper() {
  echo "running with $# arguments"
  real_command "$@"     # passes the arguments through exactly as given
}

With $* or unquoted $@, wrapper deploy "my app" silently becomes three arguments: deploy, my, app.

Where it bites

for f in $(ls); do ...            # breaks on ANY filename with a space
for f in *; do ...                # correct: the glob gives one word per file

for VAR in WORDS; do ...; done runs the body once per word, with $VAR set to each in turn. $(ls) produces text that is then word-split - so old report.txt turns into two loop rounds. A glob like * hands bash each filename whole. Never loop over the output of ls.

Single vs double quotes

'single: nothing is expanded - $var and $(cmd) stay as typed'
"double: $var and $(cmd) are expanded, but no splitting or globbing"

Single quotes for anything you want kept exactly - a pattern, a password, a program passed to another tool. Double quotes when you need the value in.

What you can now do

Why it helps

Unquoted variables are the most common real bug in shell scripts and the most common shellcheck finding (SC2086). They cause the wrong files to be deleted, "too many arguments" errors, commands receiving arguments in the wrong positions when a value is empty, and security issues when user-controlled values are expanded as globs.

In platform work, file names with spaces, variables that are empty because a timer runs the script with a different environment, and wrapper scripts that pass arguments on to another tool are everyday situations. A wrapper that uses $* instead of "$@" breaks the first time someone passes an argument like "my app". Quoting every expansion is a simple rule that removes a whole class of incidents.

Commands in this lesson

echo

FAQ

When is it safe to leave a variable unquoted?

Rarely, and only deliberately: inside [[ ]] and (( )), on the right side of a plain assignment (x=$y), in case words, and when you intentionally want splitting or globbing, such as a regex variable on the right of =~. Even then, quoting is harmless in most of those places. The simple habit "quote every expansion unless you can say why not" avoids nearly all quoting bugs.

What is the difference between "$@" and "$*"?

"$@" expands to each positional parameter as a separate word, preserving spaces inside them: exactly the arguments you received. "$*" joins all parameters into one string separated by the first character of IFS, a space by default. Unquoted $@ and $* both re-split everything. For passing arguments on, always use "$@"; use "$*" only to build a single message string.

Why is for f in $(ls) wrong?

$(ls) produces text, which is then split on whitespace and globbed, so any filename with a space, tab, newline or glob character breaks into several words. Also, ls output formatting is meant for humans and may change. Use a glob directly, for f in * or for f in /path/*.log, which gives each name as one word, and find ... -print0 with read -d '' for recursion.

How do I put a single quote inside single quotes?

You cannot escape inside single quotes. End the quote, add an escaped or double-quoted quote, and reopen: 'it'\''s' or 'it'"'"'s'. Bash also supports ANSI-C quoting, $'it\'s', where backslash escapes work. For complex strings, a quoted here-document or a variable is often clearer than nested quoting.

Does quoting matter for command substitution too?

Yes. "$(cmd)" keeps the output as one word, including internal spaces and newlines; unquoted $(cmd) is split and globbed like a variable (SC2046). rm $(find . -name '*.tmp') breaks on spaces and can hit ARG_MAX; find ... -delete or -exec rm {} + is better. Inside double quotes, $(...) can contain its own quotes safely.

In an interview Junior

Why should you quote variables in bash? Give an example of what goes wrong.

Because an unquoted expansion goes through two more steps: word splitting (cut at spaces, tabs and newlines) and globbing (any word with *, ? or [...] is replaced by matching filenames).

file="old report.txt"
rm $file      # rm gets two arguments: "old" and "report.txt"
rm "$file"    # rm gets one: "old report.txt"

The first deletes a file called old if one exists, and complains about report.txt. A value containing * turns into a list of files.

The rule: quote every expansion - "$var", "$(cmd)", "${arr[@]}" - and pass a script's arguments on with "$@" (each argument stays one word; $* glues them together). Never loop over $(ls); use a glob: for f in *. Single quotes are for text you want kept exactly - nothing is expanded inside them.

Also asked: What is the difference between single and double quotes? · What is the difference between "$@" and "$*"? · Why is for f in $(ls) wrong?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.