Installing software, brief guide for when stuff don’t work.

In this short post we are going to understand how to install software when stuff don’t work out of the box. We will understand how a *NIX shell search for software and how to make sure that our binaries are always found.

When dealing with software, installation is a classical issues. Hopefully the software you wan to install is available as a package from your favorite package manager (deb, rpm, something else) and usually those packages are well done and everything works out of the box.

However, you may need to install software that is not available as a package, or the package is broken, or something that yesterday use to work, today is not working anymore.

In those cases there are usually two options:

1. Start everything from scratch again (delete the virtual machine or stop the docker containers)
2. Understand the inner working of the system so that you can fix it, and make sure that similar problems don’t happens again.

This is a brief guide for the second option. I assume a basic knowledge of *NIX systems and some familiarity with the command line.

What it means to install software

The more proficient you become with the topic, the less “install” is simple to define.

In this post, with “install” we mean to set up the system in such a way that is possible to invoke a binary. A complete installation make sure that all the necessary environmental variables are set up correctly.

It can be as simple as apt-get install or it can be more complex.
Binaries

To install a binary is sufficient to place it in the system PATH. The system path is an environment variable that stores an ordered list of PATHs. At the moment, in my system it looks like this:

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The paths are ordered and separated by a colon : so in this case the paths are:

/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin

The different directories are there for convention, for instance the sbin directories are for system-recovery software.

When we start a binary, the shell check if it find the binary in the path, the check is done by name.

To visualize let’s try to log all the system calls when we invoke the tree binary. (tree print the directory structure of a given folder, and you can install it from system packages.)

To visualize the system calls we can use strace (again available from system packages).

However, we cannot call just strace tree since strace will start to log after we have already found tree, but we can strace bash that in turn will invoke tree like so.

$ strace bash -c "tree -D 1"
... a lot of stuff ...
stat("/usr/local/sbin/tree", 0x7ffcb15ffea0) = -1 ENOENT (No such file or directory)
stat("/usr/local/bin/tree", 0x7ffcb15ffea0) = -1 ENOENT (No such file or directory)
stat("/usr/sbin/tree", 0x7ffcb15ffea0)  = -1 ENOENT (No such file or directory)
stat("/usr/bin/tree", {st_mode=S_IFREG|0755, st_size=77384, ...}) = 0
... yet more stuff ...

As expected the system is checking all the directories in the path, in order. It start checking if a file called tree is present in the first directory stat /usr/local/sbin/tree but it returns an error, -1 the file is not there. Similarly for /usr/local/bin/tree and /usr/sbin/tree. Finally it find the file in /usr/bin/tree and it can finally invoke it.

So there are two way to install software, the first one is to add the binaries to one of the path in $PATH, the other is to add the path that contains our binaries to $PATH.

Tricking the shell into invoking the wrong command

This system is quite fragile, the checks happens only at level of strings, without doing anything more than a plain string comparison.

What happens if we install a new software, called tree?

$ mkdir -p /fake/bin
$ cat /fake/bin/tree
#! /bin/bash
echo "fake tree"
$ chmod +x /fake/bin/tree
$ /fake/bin/tree
fake tree

Here we have created a new binaries directory (/fake/bin), and we put inside it an executable (chmod +x) called tree. The fake tree just print out a string.

Now if we invoke tree the regular process will happen, all the directories in $PATH are checked until a tree executable is found, and if found it is executed.

$ tree -D 1 
1 [error opening dir]

0 directories, 0 files

Indeed the regular tree software is invoked.

Let’s change the $PATH variable:

$ export PATH="/fake/bin:$PATH"
$ echo $PATH
/fake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Now the first directory checked is /fake/bin, and the system will find an executable called tree in there.

And if we invoke tree again:

$ tree -D 1
fake tree

As expected the fake tree is invoked.

This is source of great flexibility but also of many frustrations.

It is flexible because it allow us to install new software without being administrators (sudo access). Moreover it allow to have in the system system different version of the same software. But of course it is easy to make mistake and invoke by mistake the wrong executable.

which to the rescue

$ which tree 
/fake/bin/tree

The which utility let us discover what path is followed when looking for a binary.

Let’s fix this:

$ export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
$  which tree
/usr/bin/tree
$ tree --version
tree v1.7.0 (c) 1996 - 2014 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro

Subscribe to the mail list, or follow me on twitter.