OnCallReady

Lesson 6.10 · Bash Scripting · 15 min read

[[ ]] versus [ ]

In plain words

Imagine two ways of asking a question. The first is shouting it across a noisy room, where your words may get split up or swapped for other words on the way, so "two words" arrives as two separate questions. The second is writing it on a card and handing it over, so it arrives exactly as written.

[ ] is the shouting: it is an ordinary command, so unquoted values are split and globbed before it sees them, and [ $name = "two words" ] complains about too many arguments. [[ ]] is the card: bash reads the whole question itself, so no splitting, and it can do more, like glob matching (== foo*) and regular expressions (=~ ^(dev|prod)$). (( )) is a separate card for arithmetic.

Why this lesson

Scripts constantly ask questions: does this file exist? Is this variable empty? Did the user type prod or something else? In bash, a question is a command that exits 0 for "yes" and non-zero for "no" - so if (6.3) can use it. There are two such commands, [ ] and [[ ]], and the older one breaks on exactly the values quoting (6.6) warned you about.

What you need to know already: if runs a command and checks its exit status (6.3, 6.5); word splitting and globbing (6.6).

Use [[ ]]

[ is an ordinary command (there is even a program at /usr/bin/[), whose last argument must be ]. Because it is a command, its arguments go through word splitting and globbing first:

name="two words"
[ $name = "two words" ]      # bash: [: too many arguments
[[ $name = "two words" ]]    # fine - no splitting inside [[ ]]

The first line became [ two words = "two words" ] - too many words for [. [[ ]] is bash syntax (bash reads the whole expression itself), so no splitting happens inside it. That gives you:

[[ -n $var ]]                  true if var is not empty (quote anyway, from habit)
[[ $a == foo* ]]               glob matching: does $a start with "foo"?
[[ $env =~ ^(dev|prod)$ ]]     REGEX matching (below)
[[ -f $f && -r $f ]]           && and || inside the test
[[ ! -d $dir ]]                ! = not

Use [ ] only when the script must run under /bin/sh (dash, 6.1).

The tests worth knowing

-e PATH   exists              -z STR    empty string
-f PATH   regular file        -n STR    non-empty string
-d PATH   directory           STR = STR / != string comparison
-r -w -x  readable/writable/executable
-s PATH   exists and is non-empty
-L PATH   is a symlink

N -eq -ne -lt -le -gt -ge N    integer comparison

(-eq = equal, -ne not equal, -lt less than, -le less or equal, -gt greater than, -ge greater or equal.)

Note = compares strings (text), -eq compares integers. [[ "10" > "9" ]] is false: as text, "10" comes before "9" (it compares the first character, 1 vs 9). Use -gt for numbers, or (( 10 > 9 )) (below).

=~ and regular expressions

A regular expression (regex) is a pattern that describes text - more powerful than a glob. The few pieces you need now:

^        start of the text          $        end of the text
( | )    one of the alternatives    [0-9]    any one digit
+        one or more of the thing before it
\.       a real dot (a bare . means "any character")

So ^(dev|prod)$ means "the whole text is exactly dev or prod". Chapter 7 (7.3) covers regex fully.

[[ $x =~ REGEX ]] is true when $x matches. Parts in ( ) are capture groups: bash saves what each one matched in the array BASH_REMATCH:

version=2.14.1
if [[ $version =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
  major=${BASH_REMATCH[1]}     # 2
  minor=${BASH_REMATCH[2]}     # 14
fi

Do not quote the regex - a quoted right-hand side is matched literally:

[[ $env =~ "^(dev|prod)$" ]]    # literal string match. Always false.
[[ $env =~ ^(dev|prod)$ ]]      # regex. Correct.

If the pattern is in a variable, that variable must be unquoted too - which is one of the few places you deliberately leave off the quotes.

(( )) for arithmetic

count=0
(( count++ ))
(( count > 5 )) && echo "plenty"
if (( $# < 1 )); then usage; fi

(( )) does arithmetic (integer maths) and comparisons; count++ adds 1. $(( )) does the same but gives you the result as text: n=$((n + 1)). No $ is needed on variables inside. $# is the number of arguments (6.6).

Careful with set -e: (( )) exits non-zero when its result is 0. So (( count++ )) when count is 0 counts as a failure, and kills the script. Use count=$((count + 1)) instead.

What you can now do

Why it helps

Validating input is a large part of every deploy or ops script: is the environment one of dev or prod, is the version in the form 1.2.3, is the file there and readable, is the count within limits. [[ $env =~ ^(dev|prod)$ ]] is the pattern from this chapter's mission, and it stops a typo like "prd" from deploying somewhere unexpected.

The traps cost real time: a quoted regex that never matches, [[ "10" > "9" ]] being false because it compares strings, and (( count++ )) killing a set -e script when count is zero. Knowing when [ ] is required (POSIX sh scripts that must run under dash) and when [[ ]] is better is also a common interview question.

FAQ

Do I still need quotes inside [[ ]]?

Not for word splitting or globbing on the left side: [[ -n $var ]] is safe even if var contains spaces or is empty. On the right side of == and !=, quoting matters: unquoted means glob pattern, quoted means literal string. On the right of =~, unquoted means regex and quoted means literal. Quoting variables on the left anyway is a harmless habit.

Why is my =~ regex never matching?

Most likely it is quoted. Since bash 3.2, a quoted right-hand side of =~ is matched as a literal string, so [[ $env =~ "^(dev|prod)$" ]] looks for those exact characters. Leave the regex unquoted, or put it in a variable and use the variable unquoted: re='^(dev|prod)$'; [[ $env =~ $re ]]. The variable form also avoids escaping trouble with spaces and special characters.

How do I compare numbers correctly?

Use arithmetic operators: [[ $a -gt $b ]], [ "$a" -lt 10 ], or arithmetic context (( a > b )). Inside [[ ]], < and > compare strings lexically, so [[ 10 > 9 ]] is false because "1" sorts before "9". Bash integers are whole numbers only; for decimals, use a calculator tool such as bc. Validate that input is numeric first, for example [[ $n =~ ^[0-9]+$ ]].

What is BASH_REMATCH?

An array bash fills after a successful =~ match: ${BASH_REMATCH[0]} is the whole match and ${BASH_REMATCH[1]} onward are the capture groups. For [[ $v =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]], elements 1 to 3 are major, minor and patch. It is overwritten by the next match, so copy the values into named variables right away.

Why did (( count++ )) exit my script?

(( expr )) returns status 1 when the expression evaluates to 0. count++ evaluates to the old value, so with count at 0 the result is 0, the status is 1, and set -e exits. Use count=$((count + 1)) or (( ++count )) (pre-increment returns the new value, which is non-zero), or (( count++ )) || true. The same applies to any arithmetic test that is legitimately zero.

In an interview Junior

What is the difference between [ ] and [[ ]] in bash?

[ is an ordinary command (there is even /usr/bin/[), so its arguments go through word splitting and globbing first. With name="two words", [ $name = "two words" ] fails with "too many arguments"; with an empty variable, [ -n $x ] becomes [ -n ] and is wrongly true.

[[ ]] is bash syntax: bash reads the expression itself, so nothing is split inside it. It also adds glob matching ([[ $f == *.log ]]), regex matching ([[ $env =~ ^(dev|prod)$ ]], captures in BASH_REMATCH, regex unquoted), and &&, ||, ! inside.

Use [[ ]] in bash scripts, [ ] only in #!/bin/sh scripts. In both, = and < compare text ([[ 10 < 9 ]] is true); for numbers use -lt, -eq or (( )).

Also asked: How would you check that an argument is exactly dev or prod? · Why can (( count++ )) kill a script that uses set -e? · Why must the regex after =~ not be quoted?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.