This question came to my mind last day when I wanted to free up some space on my HDDs and when it comes to what to delete (feel the pain), I wanted...
For further actions, you may consider blocking this person and/or reporting abuse
The most common pipes I use are:
Here's a
realcommand I'm using in a demo in 5 minutes to show something that's hidden deep in our logs:EDIT: Unfortunately that doesn't work because of file buffering. Here's an actual working command for a log that is getting updated in live, but it does effectively the same thing:
Bonus: Here's a super evil one >:)
sort -u to optimize sort | uniq ;)
Unless you need to count the unique results: blah | sort -n | uniq -c
Cheers for the tip about grep options. I do a lot of grepping, but I think those options will save me heaps of time in the future.
I think process substitution is incredible helpful. Took me a few years to find out about it.
A super powerful command is
xargs
. Also make sure to have a look at the-I
and-p
flags, both really powerful.I just used these two features as part of an article I wrote,
Automate Your Mac Setup and Keep It Up to Date
Compare the output of two commands, get the result list and pass in one line to another command:
And some aliases I like:
If you are curious which commands you use a lot, you can find out like this:
What Are Your Most Used Shell Commands?
Oooh I feel like this is a weakness of mine. Looking forward to the responses
Examples:
The
Just Commit Everything
command:Having a shortcut to add and commit is priceless. I use this function with a useful commit message and commit often.
This might be very dangerous if you don't
git pull
after your commit. So I suggest:Thank you! I have updated my function.
I have something similar, but I'm inserting a timestamp as a commit message. Still totally useless. :)
I also replaced
origin
word in the git repo creation atgit remote add gitlab ...
. So if I pushing something to gitlab, I use thegitlab
command, which usinggit push -u gitlab --all
. I have similar to Github.Why do you use git? Just use dropbox
xargs
is amazing for just about anything that doesn't like piped results. Want to pull a bunch of Docker images in one command?So if I understand correctly, this is equivalent to the following?
I never thought of using
xargs
this way, good to know!I guess you could parallellize the loop by adding
-P 0
to thexargs
invocation.I didn't consider using parallelization, that's a great idea! Though, what happens with Docker if it pulls in two copies of the same image at once? If two images have the same dependency, will Docker deal with this parallel pull fine, or will it bork?
EDIT: Docker seems to handle this rather well; It just queues up the layer pulls in some arbitrary order. Still, I don't know if I entirely trust this, but I can't see a reason for this to not work (And I can see a reason that one might want this to work!).
This is cool! I never thought about that.
Some dumb and possibly dangerous variations I use:
I often use this one get the pid of the parents of zombie process.
Apart from that I often use
grep
,sort -u
,sed
.A funny one is to pipe
fortune
withcowsay
I have used perl in pipes a lot for when you need to change something easily using a regular expression.
$ find . -name "*Relevant*.pm" | xargs perl -pi -e 's/something/somethingelse/'
The find name part can be tweaked to suit your needs and the regular expression in the Perl part also.
I know it's about pipe commande but since I saw some other interesting elements without pipe in it, I want to share some of them that help me daily. Hope you don't mind.
I have multiple aliases in my
.aliases
but some also in my.gitconfig
more specific to git.My favorite one is :
You can find more of them into my .dotfiles repo on github
Top client IP in apache logs.
Work also for any field : top URL, top browser etc. Damn fast !
I use this pattern to find out the last commit that for which a certain condition was true, e.g. in which something wasn't yet broken.
In this example,
test.sh
just returns a non-zero code to mimic the condition you want to check for, but this could be anything that can be tested in a shell conditional.If you have a binary test such as tesh.sh in your example, use git bisect to find faulty commit faster.
This prints the top 10 most used commands in the history. I then created one-letter aliases for these (e.g.
alias s='git status'
), saving numerous keystrokes.I can provide a counter to your question. Working on an embedded system with multiple processors, we needed to generate a system binary that included microcontroller and DSP images, where the micro booted first and loaded the DSP after verifying a CRC. I built the CRC out of a pipeline featuring awk. Don't do that. It certainly didn't save me any time.
I use this one a lot:
find * -type f | xargs grep "the-string-im-looking-for"
Why not use grep -R to avoid find? Might also want to try git grep and/or ag (silversearcher).
grep -R
might not be available on older systems.Honestly, I didn't know about that flag.
Thanks for the tip!
:)
It is also handful:
I don't have too many size in my hard disk so I usually use:
Also df -h . does the same (if you're in a dir on that volume)
Not exactly pipes but some very useful aliases that sometimes save me a lot of keystrokes
zsh global aliases:
Also using zsh with:
1 - fasd cd
2 - zsh-autosuggestions
3 - zsh-autopar
But xargs combined with find like:
print0 and xargs -0 makes us avoid some erros comming from files with spaces on its names
I have some personal libraries that are easily installable in other hosts, by following the README:
github.com/stroparo/ds
I also have a personal setup script which installs my selection of APT (debian/ubuntu) and Yum (fedora/redhat) packages, plus Oh-My-Zsh, plus the Daily Shells ('ds') repo linked to above. Just follow the installation instructions for setup-dev.sh at the bottom of this README:
github.com/stroparo/cmds#setup-a-l...
Best,
On a mac:
‘pbpaste | sort | pbcopy’
To sort (or otherwise process) what’s in the clipboard.
I have a lot of data in JSON format, and gron has been a lifesaver. Combined with the usual suspects grep, sed, cut, awk, sort, etc.
Cool! This looks a lot simpler than jq (which I use regularly).
This is a link to my bash_profile where I added a bunch of functions and aliases that helped me save a lot of time github.com/robertodessi/custom_con...
this will just highlight the word you are greping for but still print out the entire file.
I've used this exact line a couple days ago!
Exploded with files + dirs:
du -ma | sort -rn | head # summary (10 items)
du -ma | sort -rn | less # scroll with the 'less' command