Standard out

From your command line you are able to create a new file and insert text into it.

$ echo Hello from the command line > test.txt

If you try this command in your terminal, you'll notice a new file named test.txt was created and if you open this file, you'll see the text Hello from the command line inside of it.

echo prints the text Hello from the command line to the screen.
By default the echo takes the input (standard input or stdin) from the keyboard and returns the output (standard output or stdout) to the screen.

> is a redirection operator that allows us to send the standard output to go into a file.

if you want to add more text to the file test.txt you can use the >> to append text to it instead of overwriting the file.

Standard input

Just like the standard output redirection operator > we can also use the is operator as well < and what is it called? yes you guessed it! Standard input redirection operator!

Let's look at an example

$ cat < test.txt > readme.txt

if you don't know what the cat command does Check out my article 17 commands to master the Linux Command Line

Here we redirected test.txt file to be our standard input and the output of this file Hello from the command line will be redirecte to the readme.txt

so if we tried to read the readme.txt file it will have Hello from the command line as output

$ cat readme.txt

Standard Error

So far we've seen 2 different streams standard input (stdin) that receives data as input < and standard output which is what you see on the terminal and can be redirected to a file using >.

Standard Error or stderr is specifically for error messages produced by program on the terminal even if stdout is redirected. Here's an example:

$ ls /nonexistent_directory/ > test.txt

The error will be No such file or directory

You can redirect an error message to the file by using file descriptors.
What are file descriptors you may ask? Well a file descriptor is a positive number that is used to access a file. The file descriptors for stdin stdout and stderr is 0 1 and 2 respectively.

To redirect the error message to a file:

$ ls /nonexistent_directory/ 2> test.txt

The error message should be inside test.txt

We can also send the error message to the file and the result of ls with it as well. (even if the directory doesn't exist there might be a minimal output which is not common). Common output will be an empty string or a new line.

$ ls /nonexistent_directory/ > test.txt 2 > &1

if we want to get rid of sderr messages you can use this command

$ ls /nonexistent_directory/ 2 > /dev/null

Pipe and Tee

The pipe operator is represented by |, it acts as a connector by taking the output of a command and sending it as the input to another program.

let's list the details about the dev directory.

$ ls -la /dev/

Now you can use the pipe and the 'less' command to navigate through the list using the up and down arrow keys.

$ ls -la /dev/ | less

You can send the output of of the command to a file. You can write of this command to two streams which will allow you to see the result of the command on the terminal and then in the file.

$ ls -la /dev/ | tee my_output.txt

Environment

An environment variable (PATH) exists on your system. When you write a command name this variable informs the computer where to look for the programs.
If you download a program and place it in a directory where the variable PATH does not check and you try to use the tool it by typing its name in the terminal, what typically happens is thet the computer searches for it in the listed directory in PATH and since where you placed it is not checked by path you will get command not found

$ echo $HOME
$ echo $USER
$ echo $PATH

The home directory path, your username and a list of path separated by colons that your system searches when it runs a command will be displayed on the terminal respectively.

cut

The command cut is used for extracting pieces of text from files. To grab the pieces of data the cut command does it based on fields which is a descrete pieve of data within a line such as a comma, tab, semicolon etc...
We use the flag -f which stands for fields to tell cut what to extract. By default cut uses the tab as delimiters to seperate fields.

For example, we created a file named text.txt with some data in it

$ echo 'The bear jumped over the fence; The   snake followed it to the forest' > text.txt

$ cut -f 2 text.txt

The command using cut will extract the word snake. Let's break it down further.
We told cut to use the default delimiter which is tab by using -f and not specifying a delimiter and 2 stands for the second column which is what comes after the ; semicolon. since the word snake is preceeded with tab then it will be extracted.

Now let's specify a delimiter other than tab.

$ cut -f 1 -d ";" text.txt

Here we're telling cut to select the first column 1 and assigned a new delimiter instead of tab which is the ; by using the -d flag. The output will be The bear jumped over the fence.

Paste

The paste command appends lines in a file for example we have a file named testing.txt

Hello

 I am

 Learning

 How to use

 Linux commands

Let's say we want all the content of testing.txt merged into one line

$ paste -d ' ' -s testing.txt

Here we specified the the delimiter to be a space. There is a space from the second line down to the end in testing.txt. paste will make the testing.txt a oneliner by bringing each sentence preceeded by a space to be appended to the first sentence.

Head

The head command will show you the first 10 lines in a file by default. You can also modify how many lines you want to see. For example you can choose to see 20 lines instead of 10 by using the -n flag (number of lines).

$ head path_of_file
$ head -n 20 path_of_file

Tail

The tail command is the same as the head command but it shows you the last 10 lines of a file instead of the first 10.

Expand and unexpand

Let's say you want to convert tabs in a file to spaces

$ expand name_of_file.txt

If you want to convert the spaces back to tabs you can use this command

$ unexpand -a name_of_file.txt

Joining and splitting files

If you have two text files you want to combine you can use this command

$ join text_file_1.txxt text_file_2.txt

They will be joined by fields meaning the first line of both files will be on the first line, the second line of both files will be joined on the second line etc..

This command will split a file into different files the default naming of the files will start with x followed by alphabetical suffixes aa, ab etc...

• -a #: Sets the number of characters used in the suffix (default is 2).

$ join -t ',' accounts.txt logged.txt

• -t ',': defines the delimiter which is separating the fields

The purpose of join is to merge lines from multiple files based on a matching filed. For example both of your files could have numbered lines and will be merged based on that.

You can split by size (bytes -b) or by lines -l

$ split -b 1000 name_of_file.txt
$ split -l 20 name_of_file.txt

Sort

The sort command sorts lines in a file by alphabelitcal order

$ sort file_name.txt

You can also sort in reverse alphabetical order

$ sort -r filename.txt

We can also numerically sort by using the -n flag. If you've got a file with words in it, it will be sorted based on the ASCII value of the first character in each word.

Here's an example fruits.txt

Mango
Kiwi
Date
Pear
Banana

The ASCII value for the first character of each fruit is 77, 75, 68, 80, 66 respectively.

Now when we use it to numerically sort

$ sort -n fruits.txt

The output

Banana
Date
Kiwi
Mango
Pear

This command is also used to sort numbers inside a file. Example:

25
9
peach
5

The output

5
9
25
peach

Translate

The tr command is used to translate a set of characters into another.
For example let's convert from lower to uppercase.

$ tr a-z A-Z

hello

HELLO

Unique

The uniq command allows you to remove duplicates within a file

$ uniq file.txt

Using the -c flag you can get how many occurances each word has in the file

$ uniq -c file.txt

You can also find the unique value inside a file.

$ uniq -u file.txt

You can also know what words are the duplicates inside a file.

$ uniq -d text.txt

Word count

You can know the total word count in a file by using the wc coommand

$ wc file.txt

This displays the lines, number of words and number of bytes respectively. You can select which one you want to see by using these flags

• -l: number of lines
• -w: number of words
• -c: number of bytes

nl is another command you can use to check the number of lines in a file

$ nl file.txt

grep

It allows you to search files for characters that match a certain pattern.

You want to know if a word exists in a file?

$ grep hello testing.txt

This will find the word hello in the file testing.txt

you can use regular expression in your pattern so you can add the -i flag directly before the pattern you're looking for if it is case sensitive.

$ grep -i [a-zA-Z0-9]{6,10} file.txt

In this example we're trying to find a string that has a length between 6 and 10 characters and is alphanumeric and is case sensitive.