DEV Community

spO0q 🐒🎃
spO0q 🐒🎃

Posted on • Updated on

Another cheat sheet for Bash

Basics

Starting somewhere... Be careful, some command might require bash 4 (the vast majority of snippets are for bash 3, though).

Bash is an acronym

Bash stands for "Bourne Again Shell", which refers to "born again shell". Indeed, Bash has replaced the old Bourne shell initially released in 1979.

Bash is a "Unix-versal" scripting language

Bash is a crazy good language you can learn to automate various tasks whether it's for your personal use, for server maintenance, or in devOps actions. Bash scripting may consist of writing a series of command lines in a plain text file.

Note that you can type these commands on your terminal as well. You can use existing commands such as grep, sed, find but also additional commands from third-party libraries you might have installed.

However, it's often more convenient to group all instructions in files you can git and transfer from one machine to another or run in multiple servers. Bash scripts are usually extension-less files you can execute this way:

bash myscript
Enter fullscreen mode Exit fullscreen mode

Alternatively, you can make your script executable and run it:

chmod +x myscript && ./myscript
Enter fullscreen mode Exit fullscreen mode

Bash is not sh!

While Bash can run most sh scripts (the syntaxes are quite similar), sh cannot handle all Bash commands.
The following command lines will probably work but they are a bit misleading:

bash mysh.sh
sh myscript
Enter fullscreen mode Exit fullscreen mode

Don't forget the Shebang

It's the #! at the top of bash files. "Bang" is for the exclamation point "!":

#!/usr/bin/env bash
Enter fullscreen mode Exit fullscreen mode

The above line is not decorative at all! It links the script to the absolute path of the Bash interpreter. /usr/bin/env is usually a smart choice as it grabs the executable set in the user's $PATH variable.

Writing comments

Use a #:

# comment
Enter fullscreen mode Exit fullscreen mode

Variables

Variables are easy to declare, use, and manipulate in Bash.

Built-in variables (not an exhaustive list)

echo "$1 $2" # positional arguments passed one by one, e.g., bash myscript arg1 arg2
echo "$#" # number of arguments passed
echo "$@" # all arguments passed
echo "You are here: $PWD" # $PWD is the current path
Enter fullscreen mode Exit fullscreen mode

Assigning new variables

You assign variables with the = sign and no space around:

MyVar="My string"
MyArray=(all in one)
Enter fullscreen mode Exit fullscreen mode

Using variables

You can use declared variables with the $ sign:

echo $Variable
echo "This is my var: $Variable" # double quotes are required for interpolation
echo "${MyArray[@]:0:3}" # "all in one", because it prints 3 elements starting from the first (0)

# Be careful the following does not display a value
echo ${#MyArray[2]} # "3" as there are 3 chars in the third element of the array, which is the word "one"
Enter fullscreen mode Exit fullscreen mode

Modifying variables

You can modify declared variables with various signs enclosed with {}:

MyVar="My variable"
# Length
echo ${#MyVar}

# Substitution
echo ${MyVar//a/O} # "My vOriOble"

# Expansion
echo ${!BASH@} # all variable names than begin with "BASH"
echo ${!BASH*} # all variable names than begin with "BASH"

# Removals
MyFile="list.txt"
echo ${MyFile%.*} # the filename without extension
echo ${MyFile##*.} # the file's extension

# Default value
echo ${AnotherOne:-"Another one"} # displays "Another one" even if AnotherOne is not defined
Enter fullscreen mode Exit fullscreen mode

Dictionaries

declare -A Movies

Movies[title]="The Dark Knight Rises"
Movies[bestActress]="Anne Hathaway"
Movies[director]="Christopher Nolan"
Enter fullscreen mode Exit fullscreen mode

printf

printf is the C-like way to print preformatted text. It provides way more advanced features than a simple echo. It takes formats and parameters as arguments and prints the formatted text.

For example, you can specify the type and some formatting option in one line:

printf "%s\n" "Learn it" "Zip it" "Bash it" # it prints 3 lines
printf "%.*f\n" 2 3.1415926 # prints 3.14
Enter fullscreen mode Exit fullscreen mode

Loop, loop, loop

For

for i in "${MyArray[@]}"; do
    echo "$i"
done
Enter fullscreen mode Exit fullscreen mode

It works with strings too:

MyString="First Second Third"
for n in $MyString
do
   echo "$n line"
done
Enter fullscreen mode Exit fullscreen mode

While

while [ true ]
do
    echo "We loop"
    break
done
Enter fullscreen mode Exit fullscreen mode

Handy ranges

echo {1..7} # 1 2 3 4 5 6 7
echo {a..g} # a b c d e f g
Enter fullscreen mode Exit fullscreen mode

Until

until [ $counter == 111 ]
do
    echo "Number is: $((counter++))"
done
Enter fullscreen mode Exit fullscreen mode

Get user inputs

echo "6x7?"
read answer
echo answer # hopefully 42 but can be anything the user wants
Enter fullscreen mode Exit fullscreen mode

Conditional statements

if [[ CONDITION1 || CONDITION2 ]]
then
    # code
elif [[ CONDITION3 && CONDITION4 ]]
then
    # code
else
    # code
fi
Enter fullscreen mode Exit fullscreen mode

Alternatively, you might use a case statement:

case $COLOR in

  Red)
    echo -n "red"
    ;;

  Blue | Green)
    echo -n "blue or green"
    ;;

  Yellow | "Sunny" | "Gold" | "Fire")
    echo -n "Yellow"
    ;;

  *)
    echo -n "What?"
    ;;
esac
Enter fullscreen mode Exit fullscreen mode

Errors & exit strategies

It's best if you can raise errors to prevent any misuse of your scripts.

Exit immediately if a command failed

#!/usr/bin/env bash
set -e
Enter fullscreen mode Exit fullscreen mode

Exit N

You can exit the script and specify an exit code:

#!/usr/bin/env bash
if [ CONDITION ]; then
    exit 0 # 0 1 2 3 ... N
fi
Enter fullscreen mode Exit fullscreen mode

0 indicates a success whereas {1,2,3, N} are errors.

Test errors

# $? is the exit status of last command
if [[ $? -ne 0 ]] ; then
    echo "You failed!"
    exit 1
else
    echo "You are the best!"
fi
Enter fullscreen mode Exit fullscreen mode

Debug bash scripts

#!/usr/bin/env bash
set -x # set -n can also be used to check syntax errors
ARGS=("$@") # store all script args in a var
Enter fullscreen mode Exit fullscreen mode

You can also execute the script with the -x option in the terminal:

bash -x myscript
Enter fullscreen mode Exit fullscreen mode

Filesystem & directories

You can use all basic filesystem commands such as cp, rm, ls, mv or mkdir in your bash scripts.

To check if a file is executable before anything:

if [ ! -x "$PWD/myscript" ]; then
    echo "File is not executable!"
    exit 1
fi
Enter fullscreen mode Exit fullscreen mode

Use -f for files, -d for directories, -e for existence, etc. There are tons of options, you can even test symbolic links with -L and compare files by date with the -nt (newer than) and -ot (older than) options.

A bit more advanced usages

Here are some tips I consider a little more complicated than the basic ones.

Leverage the benefits of the bash syntax

Bash is pretty convenient to build output fast, especially using brace expansions:

echo {1..10}{0,5}h # bash 3
echo {10..120..10}km # requires bash 4
Enter fullscreen mode Exit fullscreen mode

Define functions

function test ()
{
    echo "$1 $2" # displays positional arguments passed to the function ("All" "In")
    return 0 # if you return a value, it !must be numerical!
}
test "All" "In"
Enter fullscreen mode Exit fullscreen mode

Scopes

You don't have to declare variables in Bash. If you call a variable that does not exist, you don't get any error but an empty value.

Environment variables

Env vars are defined at the system level. They exist everywhere, in all processes.
To list them, use printenv:

printenv USER # if you don't specify a variable, all variables will be displayed
Enter fullscreen mode Exit fullscreen mode

Shell variables

The following assignment defines a shell variable:

TEST="My test"
Enter fullscreen mode Exit fullscreen mode

You can use it directly everywhere within the script, including in functions. However, they're not passed to child processes unless you use the export command to transmit such information.

The local keyword

The local keyword can only be used within functions:

function test ()
{
    local MyVar="My string" # local variable
}
test
Enter fullscreen mode Exit fullscreen mode

You cannot export a local variable.

Best practices

As a general rule, it's better to use local variables and functions. Global variables should be used meaningfully. It prevents unwanted overriding and confusions.

Loop through the results of a command

for r in $(ls $PWD)
do
    # task using the result of the command => r
    echo "$r"
done
Enter fullscreen mode Exit fullscreen mode

Use the output of a function

function hello() {
    echo "Hello Ethan"
}

echo "This is the result of the function : $(hello)"
Enter fullscreen mode Exit fullscreen mode

Store the result of a command in a variable

Users=$(cat users.txt)
Enter fullscreen mode Exit fullscreen mode

Capture yes/no inputs

read -p "Continue? [y/n]: " -n 1 -r
echo # extra line
if [[ $REPLY =~ ^[Yy]$ ]]
then
    echo "ok"
elif [[ $REPLY =~ ^[Nn]$ ]]
then
    echo "Too bad!"
else
    echo "Sorry, I did not understand your answer :("
fi
Enter fullscreen mode Exit fullscreen mode

Capture user's selection

The following bash code prompts a selection to the user and capture the anwser:

select w in "zsh" "bash" "powershell" 
do
  echo "You prefer $w"
  break  
done
Enter fullscreen mode Exit fullscreen mode

Aliasing and overriding commands

You can create special aliases:

alias lz='ls -alh'
Enter fullscreen mode Exit fullscreen mode

You can also override existing commands simply by redefining them.

Chaining commands

wget -O /tmp/logo.jpg 'https://dev-to-uploads.s3.amazonaws.com/uploads/logos/resized_logo_UQww2soKuUsjaOGNB38o.png' && echo "echo only if the first hand is true"
wget -O /tmp/logo2.jpg 'https//example.com/logo.jpg' || echo "echo only if the first hand is wrong"
wget -O tmp/logo3.jpg 'https//example.com/logo.jpg' ; echo "echo whatever happens"
Enter fullscreen mode Exit fullscreen mode

The hard mode

Here some advanced concepts in Bash you might find useful.

Execute last executed command

sudo !!
Enter fullscreen mode Exit fullscreen mode

Redirections, stderr, stdout

COMMAND > out.txt   # write in out.txt 
COMMAND >> out.txt  # append in out.txt 
COMMAND 2> test.log # stderr to test.log
COMMAND 2>&1        # stderr to stdout
COMMAND &>/dev/null # stdout and stderr to (null)
Enter fullscreen mode Exit fullscreen mode

Replace COMMAND with your command.

It's a Trap!

The trap command is an advanced approach of errors. The key to understand it is to forget about exceptions and similar concepts you might know in other languages.

The idea is to execute something only in specific conditions:

#!/usr/bin/env bash

trap 'catch' EXIT

catch() {
  echo "Exit game!"
}

echo "ok"
echo "---"

exit 0
Enter fullscreen mode Exit fullscreen mode

The message "Exit game!" displays only because of the exit 0 at the end.

More pragmatically, it's not uncommon to see that kind of usage:

#!/usr/bin/env bash

trap "rm /tmp/scan.txt" EXIT
Enter fullscreen mode Exit fullscreen mode

Execute scripts inside a Bash script

Use the source command:

#!/usr/bin/env bash

source ./otherScript
Enter fullscreen mode Exit fullscreen mode

Subshells

A subshell is a separate instance of the command processor. A shell script can itself launch subprocesses. These subshells let the script do parallel processing, in effect executing multiple subtasks simultaneously.

Source: tldp.org

In other words, commands enclosed inside parenthesis execute in a subshell. It partly explains the use of the export command that passes environment variables on to sub-processes.

You sometimes read the term "forks" instead, but it's the same idea.

pipefail

You might use the following as a more advanced error handling:

#!/usr/bin/env bash

set -eu # u is to exit if using undefined variables
set -o pipefail
Enter fullscreen mode Exit fullscreen mode

The above set commands not only enable "error mode" but also prevent any error in pipeline from occurring silently, which is the default behavior if you don't set the -o pipefail option.

My favorite tricks in the terminal

The following commands might be more Unix-based than pure bash scripting, but it's not completely off topic 🤞🏻.

Use bash

You might have install third-party solutions for terminal prompt or use zsh. Just type:

bash
Enter fullscreen mode Exit fullscreen mode

If you press enter, you will now use the bash prompt. Then you might type man bash to display the help.

Use \

You can use \ at the end of long command lines to improve readability and not lose your focus.

Use &

nohup bash script &
Enter fullscreen mode Exit fullscreen mode

The above allows executing bash scripts in the background and catch the hangup signal.

Misc

Some tricks I use frequently:

List sub-directories only in the current folder

SUBDIRS=$(ls -d */)
for sub in $SUBDIRS
do
    echo $sub
done
Enter fullscreen mode Exit fullscreen mode

Quick rename a file

mv /project/file.txt{,.bak} # rename file.txt.bak
Enter fullscreen mode Exit fullscreen mode

Execute local bash script on a remote server

ssh REMOTE_HOST 'bash -s' < myScript
Enter fullscreen mode Exit fullscreen mode

Find heavy files (larger than 10MB)

find . -size +10M -print
Enter fullscreen mode Exit fullscreen mode

Create multiple files quickly

You can type:

touch {script1,script2,script3,script4,script5,script6,script7}
Enter fullscreen mode Exit fullscreen mode

But that's still a bit tedious, so use this:

touch script{1..7}
Enter fullscreen mode Exit fullscreen mode

Bash for hackers

Hackers would bash for sure. While Python is amazingly convenient for pen-testing, bash can also help automate many tasks. It's not uncommon for professionals to create custom bash scripts to speed up analysis.

Hello world: scanning script

Instead of repeating the same command lines over and over, you can group them into specific files. Most security binaries are built like that.

You might use the read command to make your script a bit more friendly to use or just pass arguments. Let's name our script scanning

#!/usr/bin/env bash

echo "Welcome aboard"
echo "What's the targeted IP?"
read TargetIp

if [[ ! $TargetIp ]]
    then echo "Missing targeted IP!"
    exit 1
fi

echo "What port do you want to scan?"
read PortNumber

if [[ ! $PortNumber ]]
    then echo "Missing port number!"
    exit 1
fi

nmap $TargetIp -p $PortNumber >> scan.txt # append results in scan.txt
Enter fullscreen mode Exit fullscreen mode

Then we can run bash scanning. Of course, there are existing tools to ease the use of nmap and it's a very incomplete example, but you see the idea.

Aliasing

Attackers would likely use aliases when using advanced commands in the terminal:

alias whatsMyIp="echo $(ifconfig -a | grep broadcast | awk '{print $2}')"
whatsMyIp # IP local
Enter fullscreen mode Exit fullscreen mode

Pen-testing distribution may have better autocompletion features, though, but you don't always operate in such optimized environment, so aliases can save a lot of time and energy.

Hacking bash

Hackers can hack bash like any other language. Just read this post by hacktricks.

Investigate

Start here

Top comments (1)

Collapse
 
dsculptor profile image
Mayank Raj

Note: In zsh arrays are 1-indexed

echo ${#MyArray[2]} # This will display 2 = size("in") in zsh.
Enter fullscreen mode Exit fullscreen mode