My favourite small tools, scripts and aliases

I enjoyed reading Julia Evans’ post on Tiny Personal Programs so I thought I would share some of my own. I’m hoping that they can be useful for others to copy or at least consider if they can steal some useful parts of my workflow.

Warning: These tools are use-at-your-own-risk. They work well for me but aren’t “production” ready or necessarily hardened in any way.

Also note that all the shell scripts are written in ZSH. I’ve also summarized some of them, so they may require small adjustments to run on your machine.

w (watch)

This program is a cornerstone of my everyday life. So important that I gave it a one-letter name shadowing a classic UNIX utility.

The current source is simply this:

f=${XDG_RUNTIME_DIR:-/tmp}/save
[ -e $f ] || touch $f
exec entr -r $@ <<<$f

It is just a tiny wrapper around entr, a brilliantly simple program to watch files and run programs.

They key innovation is that I realized that I don’t need to bother telling entr which files to watch 99% of the time. I can save myself brainpower and avoid requiring a restart when files are added or renamed by just telling it to watch a specific file, then I just touch that file whenever I want a rerun.

Almost all of my updates come from when I save a file. I have a simple keybind in Vim to save and touch this file.

let g:touchfile = expand('$XDG_RUNTIME_DIR/save')
nnoremap <silent> <Leader>w :echo 'No changed files.'<CR>:wa<CR>:call writefile([], g:touchfile)<CR>

Instead of doing :w or :wa I just do <Leader>w and it saves everything, then touches the magic file.

My dev workflow almost always consists of some program running under w on the left side of my screen and it re-runs every time I save my editor on the right side of the screen.

A random selection of commands from my shell history:

w cargo test
w d python3 ./test.py

# Using a command rather than an alias works great with "nested" commands.
nix-shell -A ecl --run 'w cargo run'

# Build my site, then do a word-diff against a reference copy.
w zsh -c 'nix-build -A www && git dw /nix/store/0nf7l88dpbjfzp64lfmiyl8c52xgip85-kevincox.ca result/ | cat'

# Build and upload a docker image on save.
w zsh -c 'nix-build -A docker && podman load <result && podman push nix-ci:latest registry.gitlab.com/kevincox/nix-ci:test'

You get the idea. If you can think of a command you can run it on save. Don’t bother worrying about what files and directories you want to watch and don’t worry about the time to set up watches over large directories. Just call w and save.

d

Mostly a companion on w, it adds a number of useful behaviours to commands.

1. Sets up completion notification. So that if a command completes while I am not focused on the terminal I get a desktop notification.
2. Prints starting and stopping messages. Some commands are a bit too quiet and it is hard to tell if they are running or have already finished and you are waiting for nothing. This message also includes the expanded command and some resource utilization stats.

if [[ $VTE_VERSION -ge 3405 ]]; then
    echo -n $'\e]777;preexec\e\\'
fi

color=$'\e[2m'
nocolor=$'\e[0m'

formatted="${(@q-)@}"

echo "${color}Starting $formatted$nocolor" 1>&2
TIMEFMT="${color}Done. Total: %*E (%P), User: %*U, System: %*S - $formatted$nocolor"
if [[ $VTE_VERSION -ge 3405 ]]; then
    TIMEFMT=$'\e]777;precmd\e\\'$TIMEFMT
fi
time (exec "$@")

Another “feature” is that this serves as an easy way to launch a shell. Running d eval "foo && bar" will run the argument in the shell which is just a bit less typing than zsh -c "foo & bar".

c (clipboard)

Another one important enough for a one-letter name.

xsel --clipboard

Ok, this could be an alias at this point. But over the years this has been updated to support different operating systems and use different tools to implement the command. It is also nice to have a “real” executable on your $PATH.

It comes in handy all the time. Especially since it is the easiest way to get data from programs like a web browser into a command or file without worrying about escaping. In ZSH $(c) is even binary-safe!

# base64 decode your clipboard.
c | base64 -d

# Generate a UUID and put it in the clipboard.
uuidgen | c

# Process the clipboard and put it back into the clipboard afterwards.
c | tr A-Z a-z | c

I also use this frequently in my reliance on shell history. I will commonly call up commands like the following where a common variable part is pulled from the clipboard so that I can quickly reuse the command with no editing. (I should probably make these into scripts or aliases one day but shell history is just such low overhead and easy to tweak over time.)

# Show an HTTP response with headers in my editor. (expects the URL in the clipboard)
curl -i $(c) | v

# Test building a nixpkgs patch. (expects PR ID or URL in the clipboard)
nixpkgs-review pr $(c) --no-shell --post-result -c commit

watchlog

This one is actually a full program. It is useful to throw onto the end of something that is streaming logs or otherwise slowly making progress. It gives you an idea of when progress was last made as well as injects some timestamps in to the logs during pauses.

watchlog

crypt

From time to time I need to insert a password hash somewhere. The UNIX crypt format is a common standard for upgradable password hashes. I have a small wrapper that can generate or check password hashes.

import crypt
import getpass
import sys

p = getpass.getpass()

argc = len(sys.argv)
if argc == 1:
    salt = None
    assert p == getpass.getpass(), "Passwords not the same"
elif argc == 2:
    salt = sys.argv[1]
else:
    raise ArgumentError("Too many arguments. Expected 0 or 1")

hash = crypt.crypt(p, salt=salt)
print(hash)

if salt:
    if hash == salt:
        print("Match");
    else:
        print("No Match")
        sys.exit(1)

Just call crypt to generate a hash or crypt $hash to check it.

u (UNIX timestamp)

A simple tool to take a number and guess what time it represents. (Is that timestamp in ns, ms or s???)

It can take an argument or pull a number from your clipboard, but the core code looks like this.

t=$1
t="$(perl -e '$i=$ARGV[1];while($i>1e10){$i/=1000}print($i)' tos "${t//[^0-9.]/}")"
date "-d@$t" --rfc-3339=ns "$@"

% u 1655150354 # UNIX seconds
2022-06-13 15:59:14.000000000-04:00
% u 1653150192354 # UNIX ms
2022-05-21 12:23:12.354000000-04:00

r (retry)

A simple tool to retry commands. It takes a command as an argument and retries it until it succeeds. It supports options for target exit code, delay and max attempts.

Example: Keep retrying a download on a flakey endpoint.

r --delay=10 wget --continue https://example.com/somefile.tar.gz

Example: keep tailing kube logs even if the pod restarts. (-u=-1 means until the process returns -1, AKA forever)

r -u=-1 kubectl logs -f deployment/something

Git Tools

I use git and here is a quick subsection for aliases and commands (for when git x is 4 characters too long).

For context here is a convention for my git checkouts. I have origin set to a repo that I have write access to. For example, my personal fork. I set u to the upstream repo. For my projects this may be the same as origin but for other projects this is likely a repo that I don’t have write-access to.

ga (git amend)

Before publishing, history-rewriting is great. I have a quick alias for quickly merging a fixup into an existing commit.

git commit --amend --no-edit "$@"

gf (git force)

If you don’t know about git push --force-with-lease you should really read up! It is similar to plain-old --force but provides an important safety feature. Instead of blindly overwriting whatever is at the target it checks that what you thought was there is actually there. For example, it may avoid you from overwriting a colleague’s work if they pushed to your branch with a fix. It definitely isn’t perfect (at least not with explicitly specifying which commit you expect) but at least catches some common mistakes.

But that is a mouthful to spell out and I use this frequently with development branches. So I like the short name. This allows me to simply run gf and is frequently paired like ga && gf to merge local changes into an existing commit and push the result.

git push --force-with-lease "$@"

git-clean-merged

This deletes any branches that are subsets of the HEAD ref on the u remote. For my work this means that they have been fully integrated into the upstream project.

Note that I don’t make any effort to avoid some errors like deleting the current branch. Instead, I just let git emit a nice warning in the log.

branch=${1-refs/remotes/u/HEAD}
echo "Deleting branches merged into $branch."

git branch --merged $branch --format='%(refname:lstrip=2)' \
    | grep -vFx $branch \
    | grep -v '\(detached at [0-9a-f]*\)' \
    | xargs -r -- git branch -D --

gn (git new)

A fast way to start a new branch. This is based off of whatever the HEAD ref of the u remote is. I have this set to be the branch that I want to start new changes off of.

branch=$1

git fetch u
git checkout u

[[ -n $branch ]] && git branch $branch

This is similar to git checkout master && git pull but has a key advantage; this doesn’t leave you on the main branch, so you can’t accidentally push it, and you won’t create an accidental merge if your local master is dirty.

I typically don’t pass the branch name argument. This leaves me in a “detached HEAD state”. The downside is that it is easier to lose your work because it doesn’t have an easy to find name. However, git reflog is my friend, so it isn’t a major concern. I prefer to defer giving branches names because I can often think of something more meaningful once I have played around with the code a bit.

gs (git switch)

This is a simple command for switching branches. The main reason I use this command is that it is the same to create or switch to a branch. This means that I can type gs <Up> to search my shell history for recent branches and just run the command again to switch to it. I find this much more convenient than git switch -c <Up> then switching the command to git branch once I find it.

It also has a small convenience feature where you can run gs with no arguments to switch to the “main” branch.

branch=${1-$(git symbolic-ref refs/remotes/origin/HEAD | sed 's#^refs/remotes/origin/##')}
git switch $branch || git switch -c $branch

git b (git branches)

Show “active” branches in my checkout

[alias]
b = branch -v --no-merged HEAD

Gives some tidy output like:

% git b


  renovate/tonic-0.x 670b383 Update Rust crate tonic to 0.7.2


  secure-websub f8e3737 Don't use HTTP WebSub hubs for HTTPS feeds.