Back to bash: Quoting

As the default interactive shell on many Linux based systems, bash is used by everyone and their grandmother - which may not be the most apt expression in this case - on a daily basis. I suspect most people, like myself, gradually roll into it while focusing on other tasks. While this gave me a workable knowledge, i think it can still be improved in some areas.

Back to bash is a series where i revisit bash principles from A to Z, using the official bash manual as a reference. Each post will focus on one specific element in detail, supported by examples. I will try to include pitfalls as much as possible.

To start off lightly, let's begin with quoting rules in bash. The purpose of quoting is to remove the meaning of characters that are interpreted in special ways. There are 3 supported ways of quoting, and each have their use-cases.

Scenario: we want to print that a is larger than b.

Backslash

A backslash can be used to quote the subsequent character.

$ echo a\>b
a>b

When not escaped, > has the special meaning of redirection to files. The following would redirect the command output a to a file named b, which we print using cat.

$ echo a>b
$ cat b
a

Single quotes

Single quotes escape all enclosed characters without exception.

$ echo 'a>b'
a>b

Note that single quotes can never be nested, they cannot be escaped within single quotes. Because of this it is generally not great to use in combination with natural language strings.

#Wrong!
$ echo 'Isn\'t it true that a>b?'

We can use the upcoming double quotes or escape single characters with special meaning.

$ echo Isn\'t it true that a\>b?
Isn't it true that a>b? 

It is ideal for encoding some things like simple json strings, which have double quotes in them, and rarely feature single quotes (even though that is not invalid).

$ echo '{"option_1":42}' > config.json

Double quotes

Double quotes escape enclosed characters with some notable exceptions for these symbols: $, `, \, and, if history expansion is enabled and the shell is not in POSIX mode, !.

$ echo "a>b"
a>b

Double quotes can be nested by escaping them.

$ echo "He said \"Isn't a>b?\""
He said "Isn't a>b?"

One notable thing to point out here is the irregular behavior of escaping !. When escaped, the backslash is maintained! As an example, consider creating a git commit that follows the default formatting guidelines of gitchangelog

• Attempt 1: Doesn't work, bash will try history expansion and fail.
  

  $ git commit -m "fix: dev: Login doesn't set correct access rights !wip"
  bash: !wip: event not found
  

• Attempt 2: The commit will now include the backslash!
  

  $ git commit -m "fix: dev: Login doesn't set correct access rights \!wip"
  [master (root-commit) f71201c] fix: dev: Login doesn't set correct access rights \!wip
  ...
  

• Attempt 3: Works but ugly construct
  

  $ git commit -m "fix: dev: Login doesn't set correct access rights "'!'"wip"
  

The bash manual doesn't hint at the reasoning for maintaining the backslash. I'd be curious to know that. Note that, if you don't use it you can also disable history expansion using set +H in your interactive shell. It is disabled by default in a non-interactive environment (i.e. running scripts).

ANSI-C quoting

There is also a special form of quoting $'string' which will follow escape rules as in ANSI-C, and can be used to interpret things like newlines, tabs and Unicode characters.

$ echo $'a\tb'
a   b
$ echo $'a\nb'
a
b
$ echo $'\u00AE'
®

Note that echo also has its own support for this

$ echo -e "a\tb"
a   b

Some final thoughts

Quoting is often necessary around variables. Especially when creating scripts that take outside input, we need to be mindful of this. Consider the following script to search for text in a given string. It will take two arguments and print out if the second was found in the first. We will name it search

#!/bin/bash
input=$1
search_str=$2
echo $input | grep -qi $search_str && echo "Found a match!" \
    || echo "No match found!"

If we execute this, it only works with very specific input.

$ ./search trains i
Found a match!

If the input contains spaces the script will break

$ ./search "I like trains" "like trains"
grep: trains: No such file or directory
No match found!

Consider the modified script, which we will name search_v2. Note that echo doesn't really need the extra quotes as it can work with any number of arguments.

#!/bin/bash
input=$1
search_str=$2
echo "$input" | grep -qi "$search_str" && echo "Found a match!" \
    || echo "No match found!"

Which will now give the expected output

$ ./search_v2 "I like trains" "like trains"
Found a match!

That's it for quoting. While it is not the most exciting topic, a solid of understanding of the rules can save you a lot of time!