Bash Execution Tips for Shell Jockeys and Script Fabricators

Have you ever wanted to execute one command if another failed? Or one command only if the first one succeeded? What about background execution?

Bash and other popular shells such as Zsh and Ash/Dash have some useful but sometimes confusing operators for command execution.

What is the difference between &&, &, || and ;?

A very brief cheatsheet:

• Use && to execute one command only when the previous one succeeds.
• Use || to execute one command only when the previous one fails.
• Combine the above for conditional branching.
• Use ; to join two commands when you want the second to execute no matter the result of the first one.
• Use & to run the first job in the background while the next executes. Follow all with wait for a clean return to the command prompt

Execute if previous command succeeds: &&

When running one command, sometimes it is desirable for another command to execute if and only if the first one was successful.

Let's say we want to create a user's home directory if and only if that user already exists. We can test for the user's existence with id, then, if successful, create the home directory, then, if that is successful, write a .bashrc file:

id fredflintstone && mkdir /home/fredflintstone && echo 'echo "Welcome, $USER"' > /home/fredflintstone/.bashrc

Note the use of the &&, a logical "and", for conditional execution. If this, then that.

The && operator should not be confused with the background execution operator & noted below.

Execute if previous command fails: ||

Sometimes, the opposite of the above is desired: execute the next command if the previous one failed.

For instance, perhaps we want to add a DNS server to /etc/resolv.conf only if it doesn't already exist. We can use grep to test if the server name already exists, then write back to the file if it wasn't already there:

grep 1.1.1.1 /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee -a /etc/resolv.conf

Note the use of the ||, a logical "or", for conditional execution. If not this, then that.

Also note the pipe | operator. It should not be confused with the || operator. The pipe | means "send the output of this command to the input of the next." In this case, the echo command pipes output to the tee command, which appends (due to the -a option) the input to /etc/resolv.conf.

Branching logic with both && and ||

A pleasant combination of the above is indeed possible. Imagine that you want to execute a command, then, if it succeeds, execute one command, but if it fails, execute a different command. Some creative chaining is possible, as in this example:

id fredflintstone && mkdir -p /home/fredflintstone || id bettyrubble && mkdir /home/bettyrubble

In the above, if one user does not exist, another will be tried. However, it should be pointed out that this is not equivalent to an if/then/else statement. If the 2nd command mkdir -p /home/fredflintsone would fail (not likely, with the -p flag), id bettyrubble would still run. In other words, when executing true && false || true all three commands run, because the failure of the 2nd command triggers the 3rd.

Sometimes, I use this method to explicitly set human-readable variables in a script, so that later readers (i.e. me) will easily understand what is going on:

sudo passwd -S $USER && PWD_IS_SET=true || PWD_IS_SET=false

In effect, the above "caches" a test so that the script can later test for the value of $PWD_IS_SET numerous times, in a memorable and readable way.

Unconditional execution with ;

To mash a couple commands together in one line, use the semicolon. In human language, a semicolon says, "Do this, wait until it completes, then do that." For instance:

echo Hello ; echo World

The execution is unconditional; no matter what the first command does, the second will also execute afterward:

cat filename_that_does_not_exist ; echo Continue anyway

Background execution with &

Try this:

echo Hello & echo World

Yes, both commands executed, but the behavior may well be quite different than the use of ; above. The above two commands were executed in parallel. In other words, at the same time. There is no guarantee regarding which will complete first. In fact using just & at the end of a long running command will allow it to run in the background indefinitely.

The main point in the context of this article: & is not && nor is it ; (the order and conditions for execution are different for each).

Testing teaser

A thorough exploration of the builtin shell command test is better left for a future article. That said, let's at least dabble a bit, as the concept is quite applicable to conditional execution.

Note: in POSIX-derived shells such as Bash, Zsh, and Ash/Dash, the test command and the [ command are the same command, with the exception that when [ is used, it should end with a ]. While [ works well in a pattern like "if [ -d some_directory ]; then" for multi-line readability (the last line should be "fi" to end the if statement), for succinct one-liners I prefer "test -d some_directory" and the like.

A quick cheatsheet with some commonly used tests using the test command:

• test -d some_directory will be true if a directory exists
• test -r some_file will be true if a regular file is readable
• test -n "$SOME_STRING" will be true if a string (such as a variable) is non-empty
• test -z "$SOME_NONEXISTENT_STRING" will be true if a string is empty

See the POSIX spec for test for many more options. You might also browse Bash Conditional Expressions or Zsh Conditional Expressions for shell-specific docs. When possible, I try to write shell scripts in POSIX-compliant ways for portability (scripts that work across a variety of shells). That said, sometimes you may prefer to use the full power of your shell's specific features. Browse the relevant docs to consider your options.

The above can be very useful for conditional execution. Something like this works well:

test -r /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee /etc/resolv.conf

In other words, if /etc/resolv.conf does not exist, create it with the appropriate contents. But if it already exists, do nothing.

The possibility of idempotence (alternate title: put a little Dev in your Ops)

Perhaps you can see a clear use case here for system configuration. If you have ever used Ansible and similar configuration tools, you may note that it is desirable for every task to be idempotent; in other words, even when run more than once, the desired result is the same.

I believe it is good for shell scripts to be as idempotent as possible, when they are intended to configure a system in a certain state. The above examples that reference /etc/resolv.conf do this well, so I will reference them again here:

test -f /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee /etc/resolv.conf
grep 1.1.1.1 /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee -a /etc/resolv.conf

These two lines are very similar, and only one is needed, depending on the desired outcome. The first is a great example of creating a new file with the desired initial state. It uses test -f to first determine if the file exists. Sure, you could always overwrite the file and ensure state that way, but that would update the file stats, such as timestamp, needlessly. In addition, this example may be useful on configuration files with a desired initial state, when future modifications are expected and should not be altered.

The second is instead an example of ensuring that a certain line exists in a file. It uses grep to test for a word or line in the file, then adds the line if and only if it isn't already there.

These methods achieve idempotency with the combination of testing and conditional execution. A worthy goal.

One-liners and robust scripts

The examples and concepts in this article are equally at home in a terminal window or in a substantial configuration script. Shell execution logic should be your friend. Feel free to post examples and tips in the comments below.