A Senior Dev's List of Key Unix Utilities, Explained by a Junior Dev

If you are a developer working on a Mac, chances are you've used a Unix utility or two. At least to cd around to different directories or touch a new file. But chances are you haven't used the vast majority of Unix programs you have on your machine - I have 1325 on mine! Luckily, there's no need to sit down and memorize them all. What is important is to recognize the power of the Unix command line, and to know a few commands that will make your life in the terminal much easier. Below is a list of those commands. But first, a couple notes about using Unix commands in general.

General Rules of Unix Utilities

• Commands are lowercase
• Options modify the command. They are often single letters added in short form with a single dash -. Some have long form options added with a double dash --.
• Options can be combined. For example, adding the -a and -l options to ls can be written as ls -a -l or ls -al
• Commands can take arguments, often filenames
• Options come before filenames
• There must be spaces between commands, options, and filenames

Unix Utilities to Know by Heart

ls

Lists all your files in the current directory. There are a few important option flags to know as well.

• -a will also show all the hidden files in a directory (those that start with a .).
• -l will list the files in long format, showing permissions, size, and modified date
• -lh lists in long format with readable file size (eg 1K, 23M)
• -s lists the allocated size of each file in blocks

cp source destination

Copies the source to the destination. Source can be one or more files or directories. Destination is the name of the directory to copy to or the name of the new file. When copying directories, the -r (or -R) flag must be used. This copies recursively, and will ensure that the contents of a folder is also copied.

cd

Change directory. cd with no arguments will bring you to your home directory, which is represented by ~. You can also pass a pathname or make smaller "steps" with cd .., which will move to your parent directory. cd will take relative or absolute pathnames, which can be built with ~ (home directory), .. (parent directory), . (current directory) and / within directories. Note that if you are in a directory with a subfolder test, cd test is equivalent to cd ./test and both will move you into the test folder.

mv source destination

Moves (or renames) source to destination. If the destination is an existing directory, source(s) will be moved there.

rm source

Removes files or directories. Similarly to cp, to remove directories the -r or -R flag must be used which will recursively removed the contents of the directory.

mkdir directory

Create a new directory. The -p flag is used to create parent directories if they don't already exist. For example, if I am in a directory with a subdirectory test and want to create a director example within test, I could run mkdir -p test/example. Since test already exists, a new folder example would be created in test. However, if test didn't exist, the command would simply create test and then create example within test. Either way there are no errors.

grep patterns file

Searches for patterns in each file, and prints each line that matches. This is a really useful command for finding other commands you've used if they are in your logs. The -r flag reads files in subdirectories recursively. -n adds to the output the line number in the file where the match occurred.

cat filename

Used to view the contents of a file. With an argument of a filename, cat will simply print the content of that file. Other options can be added to view only parts of the file or to write contents to a new file.

echo

Displays a line of text. Most often used in scripts and other commands where you need to insert text.

head filename

Output the first part of the file. By default, head prints the first 10 lines. Useful when you only need to see the beginning of a file with lots of content.

tail filename

Output the last part of the file. Like head but for the end of the file.

less filename

Opens a file in the terminal and allows forward and backward navigation within the file. In this mode, patterns can be searched for with /pattern and using n and N to move backwards and forwards between matches. There are a bunch more navigation options to use as well. q will exit navigation.

Unix Utilities to Know About

find

Searches for files based on some user specified criteria, and returns a list of the found files. The search criteria could be pattern matching, a file type, or file name to name a few. For example, find . -name 'test*' would search the current directory for filenames that started with 'test', with * being a placeholder for anything that might follow.

curl

A tool to transfer data to or from a server, using any of its supported protocols (FTP or HTTP for example). There are whole bunch of options for using curl but know that it exists so you can search for the specifics when you need it! wget is a similar command with a couple differences, mainly its ability to download recursively.

vim filename

Opens a file with the vim text editor. Using vim is a whole other blog post, but it is a powerful tool that many developers swear by.

sed

Used to perform text transformations on an input stream (a file or input from a pipeline). It can be used to search for strings in a file, update content, and replace content.

awk

Another big topic, awk can run programs written in the AWK language that are used to transform text in a file. awk can pattern match and perform operations on matched lines, useful for file formatting.

pipes

Piping is a process of sending the output of one command as the input to another. The syntax for a pipe is | and information flows from left to right. So, to pipe the output of ls into less for example, you could write ls | less, and enter the less navigation with all the filenames in the directory - the output of ls.

xargs

Converts standard input to arguments. Useful with commands that only accept arguments and do not read standard input such as echo and rm. With piping, input can be piped to xargs which will execute a given command for each input. A common use of this is with the find command. Results of find can be piped to xargs to run an operation on each result, such as rm. For example, to remove any file that begin with test, I could write find . -name 'test*' | xargs rm.

And there's so much more

The internet is your friend when it comes to learning the details of Unix utilities! Don't be afraid to play around in your terminal either and see what you can do. Hopefully this is a good start for some useful commands to know about and the basics of how to use them. Thanks for reading!