There are a lot of command line utilities available in bash that can help you process millions of data in seconds. Some of them are
Head
Head is a command line utility to print first part of files given in standard input
head -1 data.txt - prints the first line of data.txt file
Tail
Tail is a command line utility to print the last part of files given in standard input. ie It works just the opposite of head
tail -1 data.txt - prints the last line of data.txt file
Cat
Cat is a command line utility for concatenating files and printing to standard output.
cat data.txt - prints the contents of data.txt file
Grep
Grep is a is a command line utility for printing lines that match a pattern.
cat data.txt | grep "some-data" - Search for some-data in the data.txt file and print the lines which contains the text
Shuf
Shuf is a command line utility which generates random permutations from input lines to standard output.
cat data.txt | shuf - Prints the content of data.txt shuffled
Uniq
Uniq is a command line utility which can be used for filtering repeated lines in a file.
cat data.txt | uniq - Filtering repeated lines in data.txt file and prints it.
Cut
Cut is a command line utility for cutting sections from each line of files and writing the result to standard output.
echo 'baz' | cut -b 2 - Prints a that is the secord character as specified in the cut.
Join
Join is a command line utility for joining lines of two files on a common field.
join file1.txt file2.txt - Joins the data of 2 files based on some common field.
Split
Split is a command line utility that can be used to split a file into pieces.
split -l 200 split_me.txt - Split split_me.txt to n files having 200 lines
jq
jq is a command line utility used for processing JSON.
jq '.' data.json - Pretty prints data.json file
awk
awk is a command line utility that searches for certain patterns and specified actions on that line.
Top comments (0)