Bash Command Completion - Finding all the cats in your $PATH!

Today, I was curious to find out how many programs ending with cat I had on my system. I remember using cat, zcat and wanted to know if there are similar programs. Pressing TAB after cat only gives programs which start with cat.

1. apropos

In my last post, I had mentioned that apropos is a way to search for what programs are available on your system. The search string for apropos can be any regex. So, apropos 'cat$' should solve the problem. cat$ means all words ending with 'cat'

The output has

STAILQ_CONCAT (3)     - implementations of singly-linked lists, singly-linked tail queues, lists and tail queues
OPENSSL_strlcat (3ssl) - Memory allocation functions
..
bzcat (1)            - decompresses files to stdout
cat (1)              - concatenate files and print on the standard output
fc-cat (1)           - read font information cache files
gencat (1)           - Generate message catalog

Clearly, the top 2 do not look like programs. Why is apropos then returning them?
Let's have a look at the apropos manual

man apropos
# apropos - search the manual page names and descriptions

So apropos searches the man pages. And looks like there are man pages for other things and not just programs...

Digging deeper, let's try manual for the man pages!

man man
# The table below shows the section numbers of the manual followed by the types of pages they contain.
#       1   Executable programs or shell commands
#       2   System calls (functions provided by the kernel)
#       3   Library calls (functions within program libraries)
#       4   Special files (usually found in /dev)
#       5   File formats and conventions eg /etc/passwd
#       6   Games
#       7   Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
#       8   System administration commands (usually only for root)
#       9   Kernel routines [Non standard]

Ok. We are interested in are executable programs i.e section 1 of the man pages. apropos has a way to limit which sections we search using-s flag.
apropos -s 1 'cat$' gives us all programs ending with name cat which have an entry in the man pages but it does not show us any programs which do not have a man page.

2. List all programs on your path

The way Bash knows which programs can be called directly by their name (e.g ls) and not by their full path (e.g /usr/bin/ls) is by looking at the $PATH environment variable.

** Listing all executable files on path **

Here's a small bash snippet which lists the executable files in PATH (let's call it paths.sh)

#!/bin/bash
# The directories in $PATH are separated by ":", so we split by it to get individual directories
for pdir in $(echo "$PATH" | tr ":" "\n")
do
    # We `find` all files in the directory which are executable and print the filename
    find "$pdir" -maxdepth 1 -executable -type f -printf "%f\n"
done

If you prefer Python, here's a small Python program for the same (let's call it paths.py)

from itertools import chain
import os
path_dirs = os.environ['PATH'].split(':') # Split PATH by ':'
print(path_dirs)
all_files = chain(*(os.walk(p) for p in path_dirs)) # Iterable of all files in the directories contained in PATH
is_exec = lambda x : os.access(x, os.X_OK) # Function to check if a filename is executable
execs = chain(*([f for f in fs if is_exec(os.path.join(r,f))] for r,_,fs in all_files))
for x in execs:
    print(x)

Running either our Bash or Python scripts will give us the correct output!

sh paths.sh | grep 'cat$'
python3 paths.py | grep 'cat$'

3. Power of Bash Completion!

When I press TAB TAB after typing a letter, I get a list of suggestions. How does Bash do that? The Bash manual says that it uses complete and compgen built-ins for suggesting completions.

compgen generates completions using a list of words (-W) or list of commands (-c). The latter is of particular interest to us. compgen -c prints every executable on our path and all shell built-ins and shell-functions.

compgen --help prints following message:
compgen: compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [word]
Display possible completions depending on the options.

The options stand for:

• a : aliases
• b : shell builtins
• c : executable commands
• d : directories in current directory
• e : export variables
• f : files in current directory
• g : groups in system
• j : pending jobs (in background / stopped)
• k : Bash keywords
• s : System services
• u : users
• v : All shell variables

So compgen -c | grep 'cat$' should give us every single executable ending with cat.

Epilogue

Diving into this rabbit-hole has given me a better understanding of how Bash completion works, how apropos finds relevant programs and why man pages are organized into various sections.