Getting information
Return your user name
whoami
Return your user and group id
id
Return operating system name, username, and other info
uname -a
Display reference manual for a command
man top
Get help on a command
curl --help
Return the current date and time
date
Monitoring performance and status
List selection of or all running processes and their PIDs
ps
ps -e
Display resource usage
top
List mounted file systems and usage
df
Working with files
Copy a file
cp file.txt new_path/new_name.txt
Change file name or path
mv this_file.txt that_path/that_file.txt
Remove a file verbosely
rm this_old_file.txt -v
Create an empty file, or update existing file's timestamp
touch a_new_file.txt
Change/modify file permissions to 'execute' for all users
chmod +x my_script.sh
Get count of lines, words, or characters in file
wc -l table_of_data.csv
wc -w my_essay.txt
wc -m some_document.txt
Return lines matching a pattern from files matching a filename pattern - case insensitive and whole words only
grep -iw hello \*.txt
Return file names with lines matching the pattern 'hello' from files matching a filename pattern
grep -l hello \*.txt
Navigating and working with directories
List files and directories by date, newest last
ls -lrt
Find files in directory tree with suffix 'sh'
find -name '\*.sh'
Return present working directory
pwd
Make a new directory
mkdir new_folder
Change the current directory: up one level, home, or some other path
cd ../
cd ~ or cd
cd another_directory
remove directory, verbosely
Printing file and string contents
Print file contents
cat my_shell_script.sh
Print file contents page-by-page
more ReadMe.txt
Print first N lines of file
head -10 data_table.csv
Print last N lines of file
tail -10 data_table.csv
Print string or variable value
echo "I am not a robot"
echo "I am $USERNAME"
Compression and archiving
Archive a set of files
tar -cvf my_archive.tar.gz file1 file2 file3
Compress a set of files
zip my_zipped_files.zip file1 file2
zip my_zipped_folders.zip directory1 directory2
Extract files from a compressed zip archive
unzip my_zipped_file.zip
unzip my_zipped_file.zip -d extract_to_this_direcory
Performing network operations
Print hostname
hostname
Send packets to URL and print response
ping www.google.com
Display or configure system network interfaces
ifconfig
ip
Display contents of file at a URL
curl <url>
Download file from a URL
wget <url>
Bash shebang
!/bin/bash
Pipes and Filters
Chain filter commands using the pipe operator
ls | sort -r
Pipe the output of manual page for ls to head to display the first 20 lines
man ls | head -20
Shell and Environment Variables
List all shell variables
set
Define a shell variable called my_planet and assign value Earth to it
my_planet=Earth
Display shell variable
echo $my_planet
List all environment variables
env
Environment vars: define/extend variable scope to child processes
export my_planet
export my_galaxy='Milky Way'
Metacharacters
Comments
# The shell will not respond to this message
Command separator
echo 'here are some files and folders'; ls
File name expansion wildcard
ls *.json
Single character wildcard
ls file_2021-06-??.json
Quoting
Single quotes - interpret literally
echo 'My home directory can be accessed by entering: echo $HOME'
Double quotes - interpret literally, but evaluate metacharacters
echo "My home directory is $HOME"
Backslash - escape metacharacter interpretation
echo "This dollar sign should render: \$"
I/O Redirection
Redirect output to file
echo 'Write this text to file x' > x
Append output to file
echo 'Add this line to file x' >> x
Redirect standard error to file
bad_command_1 2> error.log
Append standard error to file
bad_command_2 2>> error.log
Redirect file contents to standard input
$ tr “[a-z]” “[A-Z]” < a_text_file.txt
The input redirection above is equivalent to
$cat a_text_file.txt | tr “[a-z]” “[A-Z]”
Command Substitution
Capture output of a command and echo its value
THE_PRESENT=$(date)
echo "There is no time like $THE_PRESENT"
Command line arguments
./My_Bash_Script.sh arg1 arg2 arg3
Batch vs. concurrent modes
Run commands sequentially
start=$(date); ./MyBigScript.sh ; end=$(date)
Run commands in parallel
./ETL_chunk_one_on_these_nodes.sh & ./ETL_chunk_two_on_those_nodes.sh
Scheduling jobs with Cron
Open crontab editor
crontab -e
Job scheduling syntax
m h dom mon dow command
minute, hour, day of month, month, day of week
'*' means any
Append the date/time to file every Sunday at 6:15 pm
15 18 * * 0 date >> sundays.txt
Run a shell script on the first minute of the first day of each month
1 0 1 * * ./My_Shell_Script.sh
Back up your home directory every Monday at 3 am
0 3 * * 1 tar -cvf my_backup_path\my_archive.tar.gz $HOME\
Deploy your cron job
Close the crontab editor and save the file
List all cron jobs
crontab -l
Top comments (0)