DEV Community

István Lantos
István Lantos

Posted on

What are your UNIX pipeline commands that saved you from lot of coding/time?

This question came to my mind last day when I wanted to free up some space on my HDDs and when it comes to what to delete (feel the pain), I wanted to know without lot of right-clicks which folders are the fattiest. As a good Windows user, I installed Total Commander, because some random Google result told me to do so.

Then I realised, heck, I have an entire UNIX environment on my PC (MSYS2), so maybe there is an utterly simple one liner command for achieve this. And guess what, it has: :)

$ du -sh * | sort -h
Enter fullscreen mode Exit fullscreen mode

Life hack, place this in your .bashrc file:

# Within the current directory, list all folders and files
# and print their size, then sort it from smallest to largest:
alias du='du -sh * | sort -h'
Enter fullscreen mode Exit fullscreen mode

What are your UNIX pipeline commands, where you can combine some program's standard output and creating something wicked simple time saver?

Latest comments (42)

Collapse
 
voyeg3r profile image
Sérgio Araújo • Edited

Not exactly pipes but some very useful aliases that sometimes save me a lot of keystrokes

# open the last edited file with neovim
alias lnvim='nvim -c'\'':e#<1'\'
alias pbpaste='xclip -i -selection clipboard -o'
alias pbcopy='xclip -selection clipboard'

zsh global aliases:

alias -g ND='./**/*(/om[1])' # newest directory --> ls -d ND
alias -g NF='./**/*(.om[1])' # newest file

cd ~/music/recordings && audacity NF
# using fasd cd I just type
z recor<Enter> auda<Tab> NF

Also using zsh with:

1 - fasd cd
2 - zsh-autosuggestions
3 - zsh-autopar

But xargs combined with find like:

 find -type f -print0 | xargs -0 cp -t ~/backup

print0 and xargs -0 makes us avoid some erros comming from files with spaces on its names

Collapse
 
clawfire profile image
Thibault Milan

I know it's about pipe commande but since I saw some other interesting elements without pipe in it, I want to share some of them that help me daily. Hope you don't mind.

I have multiple aliases in my .aliases but some also in my .gitconfig more specific to git.
My favorite one is :

[alias]
        # Remove branches that have already been merged with master
    # a.k.a. ‘delete merged’
    dm = "!git branch --merged | grep -v '\\*' | xargs -n 1 git branch -d"
        # List contributors with number of commits
    contributors = shortlog --summary --numbered
        # pull repository, rebasing, by auto stashing before and restore after and prune"
        up = "pull --rebase --autostash --prune"

You can find more of them into my .dotfiles repo on github

Collapse
 
speedmeup_net profile image
SpeedMeUp • Edited

Top client IP in apache logs.

# cut -d" " -f1 apache.access.log | sort | uniq -c | sort -rn | head
 349963 22.208.1.241
  16434 15.99.2.62
   8685 7.8.27.98
   2047 52.14.4.76
    265 83.12.37.3
    149 3.71.24.250
     78 14.213.14.6
     13 182.37.3.88

Work also for any field : top URL, top browser etc. Damn fast !

Collapse
 
bertvv profile image
Bert Van Vreckem
topcmd ()
{ 
    history | awk '{a[$4]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head
}

This prints the top 10 most used commands in the history. I then created one-letter aliases for these (e.g. alias s='git status'), saving numerous keystrokes.

Collapse
 
motilevy profile image
motilevy • Edited
hgrep () {
        grep --color=auto --color -E "$1|$" "${@:2}"
}

this will just highlight the word you are greping for but still print out the entire file.

Collapse
 
cptstroparo profile image
Cristian Stroparo • Edited

I have some personal libraries that are easily installable in other hosts, by following the README:

github.com/stroparo/ds

I also have a personal setup script which installs my selection of APT (debian/ubuntu) and Yum (fedora/redhat) packages, plus Oh-My-Zsh, plus the Daily Shells ('ds') repo linked to above. Just follow the installation instructions for setup-dev.sh at the bottom of this README:

github.com/stroparo/cmds#setup-a-l...

Best,

Collapse
 
cptstroparo profile image
Cristian Stroparo

Exploded with files + dirs:

du -ma | sort -rn | head # summary (10 items)
du -ma | sort -rn | less # scroll with the 'less' command

Collapse
 
antonrich profile image
Anton

I've used this exact line a couple days ago!

Collapse
 
oscherler profile image
Olivier “Ölbaum” Scherler

I have a lot of data in JSON format, and gron has been a lifesaver. Combined with the usual suspects grep, sed, cut, awk, sort, etc.

Collapse
 
nafhn profile image
Nathan Macey

Cool! This looks a lot simpler than jq (which I use regularly).

Collapse
 
lucianoq profile image
Luciano Quercia • Edited
alias ctrlc='xclip -selection clipboard -i'
alias ctrlv='xclip -selection clipboard -o'

Examples:

# copy something without opening editors or showing it in terminal
cat ~/.ssh/mykey.pub | ctrlc

# paste clipboard somewhere
ctrlv >> ~/.ssh/known_hosts


ctrlv | tr '[:upper:]' '[:lower:]'
ctrlv | tr -d "\n"
ctrlv | base64 -d 
ctrlv | base64
ctrlv | md5sum

cat myfile | base64 | ctrlc
cat myfile | md5sum | ctrlc



ctrlv | base64 | ctrlc
ctrlv | base64 -d | ctrlc
Collapse
 
barjo profile image
Jonathan Bardin

I often use this one get the pid of the parents of zombie process.

ps -A -ostat,ppid | awk '/[zZ]/{print $2}'

Apart from that I often use grep, sort -u, sed.

A funny one is to pipe fortune with cowsay

fortune -a | cowsay -f vader
Collapse
 
erikdstock profile image
Erik

Some dumb and possibly dangerous variations I use:

# A blunt tool to fix changed ruby files before a commit
alias precommitfix="git status -s | awk '{print $2}' | grep .rb$ | xargs bundle exec rubocop -a"

# Delete merged remote branches (remove --dry-run after verifying)
git branch -r --merged | \
grep origin | \
grep -v '>' | \
grep -v master | \
xargs -L1 | \
awk '{sub(/origin\//,"");print}'| \
xargs git push origin --delete --dry-run

Collapse
 
jonasbn profile image
Jonas Brømsø

I have used perl in pipes a lot for when you need to change something easily using a regular expression.

$ find . -name "*Relevant*.pm" | xargs perl -pi -e 's/something/somethingelse/'

The find name part can be tweaked to suit your needs and the regular expression in the Perl part also.

Collapse
 
heartfelttech profile image
HeartfeltTech: Sarah

On a mac:

‘pbpaste | sort | pbcopy’

To sort (or otherwise process) what’s in the clipboard.

Collapse
 
robertodessi profile image
Roberto

This is a link to my bash_profile where I added a bunch of functions and aliases that helped me save a lot of time github.com/robertodessi/custom_con...