DEV Community

Discussion on: What are your UNIX pipeline commands that saved you from lot of coding/time?

Collapse
 
val_baca profile image
Valentin Baca • Edited

The most common pipes I use are:

grep # ALL HAIL. Should know these:
grep -F # fixed/literal mode, use to search exactly what you type, no regex
grep -i # case insensitive
grep -v # reverse, shows only lines that do NOT match
grep -w # matches on "words" only, grep -w "lag" won't match "flag"

sort # usually combine with uniq

uniq # usually combined with sort

wc -l # count lines

head -n 10 # top 10! you won't believe number 7!

tail -f # tail with follow, useful for log that are being updated, such as:

tail -f | grep -i error

cut # to only get certain columns, for example:
cut -d' ' -f12-15 # split on spaces, only show columns 12 through 15
# or use gawk when dealing with buffers
gawk '{$12,$13,$14,$15}'

Here's a real command I'm using in a demo in 5 minutes to show something that's hidden deep in our logs:

tail -f our.log | grep -w "WARNING" | grep --color=always -F "something=" | cut -d' ' -f12-15

EDIT: Unfortunately that doesn't work because of file buffering. Here's an actual working command for a log that is getting updated in live, but it does effectively the same thing:

tail -f our.log| grep --line-buffered -w "WARNING" | grep --color=always -F --line-buffered "something=" | gawk '{print $12,$13,$14,$15}'

Bonus: Here's a super evil one >:)

$ yes > /dev/null & # does nothing but waste CPU
Collapse
 
gregorybodnar profile image
Greg Bodnar

Cheers for the tip about grep options. I do a lot of grepping, but I think those options will save me heaps of time in the future.

Collapse
 
tuomotanskanen profile image
Tuomo Tanskanen

sort -u to optimize sort | uniq ;)

Collapse
 
lilorox profile image
Pierre

Unless you need to count the unique results: blah | sort -n | uniq -c