What the #! shebang really does

What exactly happens when we run a file starting with #! (aka shebang), and why some people use #!/usr/bin/env bash.

How the #! works

The #! shebang is used to tell the kernel which interpreter should be used to run the commands present in the file.

When we run a file starting with #!, the kernel opens the file and takes the contents written right after the #! until the end of the line. For didactic purposes, let's consider it saves in a variable called command the string starting after the shebang and ending in the end of line.

After this the kernel tries to run a command with the contents of the command and giving as the first argument the filename of the file we're trying to execute.

Therefore, if you have an executable file called myscript.sh with some shell commands and starting with #!/bin/bash, when you run it, the kernel will execute /bin/bash myscript.sh.

In the examples below you're going to see it very clearly.

Starting with the classic hello.sh:

#!/bin/bash
echo "Hello World!"

Assuming this file has the executable permission, when you type this in the command line:

$ ./hello.sh

The kernel will notice the #! in the very first line and then will get what's after it, in this case /bin/bash. And then what is executed has the very same effect of this:

$ /bin/bash hello.sh

Let's use another example using #!/bin/cat. The name of the file is shebangcat:

#!/bin/cat
All the contents of this file will be
printed in the screen when it's executed
(including the '#!/bin/cat' in the first line).

Let's remember:

• What's after the shebang: /bin/cat
• Name of the file: ./shebangcat

Therefore this is what's executed: /bin/cat ./shebangcat

See it by yourself:

$ ./shebangcat
#!/bin/cat
All the contents of this file will be
printed in the screen when it's executed
(including the '#!/bin/cat' in the first line).

Let's take another example to make it very clear that things are like I'm saying. The following file is called shebangecho:

#!/usr/bin/echo
The contents of this file will *NOT* be
printed when it's executed.

Let's check:

$ ./shebangecho
./shebangecho

The output was the name of the file because this is what was executed by the kernel /usr/bin/echo ./shebangecho.

Another interesting thing, is that if we pass arguments when calling our script, such arguments will also be passed to the command executed by the kernel. As we can see in the following example called shebangls.sh:

#!/bin/ls
The contents here doesn't matter.

Now, when we run it:

$ ./shebangls.sh
./shebangls.sh

$ ./shebangls.sh -l
-rwxr-xr-x 1 meleu meleu 41 Nov 28 14:42 ./shebangls.sh

$ ./shebangls.sh notfound
/bin/ls: cannot access 'notfound': No such file or directory
./shebangls.sh

Why some people use #!/usr/bin/env?

You probably saw some scripts starting with #!/usr/bin/env bash where you're used to see just #!/bin/bash. The reason of this is to increase the portability of the script (even thought it's a debatable matter, as we're going to see below).

The env command, if used with no arguments, prints a (big) list with all the environment's variables. But if env is used followed by a command, it runs that command in another instance of the shell.

🤔 - OK, but how does that influence portability?!

When you use #!/bin/bash you're clearly saying that bash is in the /bin/ directory. This seems to be the default in all Linux distributions, but there are other Unix flavors where it can possibly not happen (for example the bash can be placed in the /usr/bin/). In systems like that your script starting with #!/bin/bash would cause a bad interpreter: No such file or directory.

When you run env bash, the env will search for bash in your $PATH variable, and then run the first one it finds. Usually bash is in /bin/, but a user running your script on some other system can have it in /usr/bin/ or even testing an alternative version in /home/user/bin/bash.

So, in order to make the script have a greater reach and be used in environments other than Linux, some people recommend the use of the env technique.

🤔 - But wait! What guarantees that the env will always be in the /usr/bin/?

There are no guarantees... 😇

The recomendation is based in what is commonly seen in the Unix systems. I see /usr/bin/env being used in some modern projects (like RetroPie), but where it's specially useful is when you need to run a python or even a NodeJS script.

Let's take this NodeJS usage as an example. I want to call a NodeJS script just by calling the script's filename. Then I could do something like this:

#!/usr/bin/node
console.log('Hello World from NodeJS');

The problem is that I usually install node via Node Version Manager, instead of using the the distribution's package manager. So, my node is like this:

$ which node
/home/meleu/.nvm/versions/node/v14.15.1/bin/node

By any means I want to put #!/home/meleu/.nvm/versions/node/v14.15.1/bin/node in my script!

So, the solution here is to use #!/usr/bin/env node.

And if I don't want to use #! at all?

I strongly recommend you to never write neither run a shell script without a #! shebang!

As we said, the shebang tells to the kernel which interpreter is to be used to run the commands present in the file. If you run a script without specifying the interpreter, the shell will spawn another instance of itself and try to run the commands in the script. Which means that it will execute whatever commands found in the file, even if it was written for zsh, ksh, dash, fish, node, python, or whatever.

Summing up: Always start your scripts with a #! shebang. Preferably with #!/usr/bin/env.

Links

• http://mywiki.wooledge.org/BashProgramming#Shebang
• https://wiki.bash-hackers.org/scripting/basics#the_shebang
• Here's an email from Dennis Ritchie in 1980, talking about these "magic characters".
• man env
• Node.js shebang