David J Eddy

Posted on Nov 27, 2018

Using `shellcheck` to lint your bash/sh scripts.

#bash #devops #linting #bestpractice

Opening

Being involved in technology, specifically web, it does not take long we have to write a bash (or shell) script. If you work with server side technology this event happens very soon after logging into a server. As a Dev(Sec)Ops practitioner linting code during the a CI/CD process is a basic requirement for me personally. Until recently linting Bash/sh code for me was a painful process. However, like so much else it was just a matter of time before a solution was found to automate the process. Enter 'shellcheck'.

Installation

For this quick demo I am running Ubuntu 18.04. The installation process is very straight forward, APT work for this.

sudo apt-get install -y shellcheck

Once executed we should see the standard install output. Just to make sure everything completed as expected I execute a quick version check.

shellcheck --version

And has hoped shellcheck is indeed installed.

Options

Getting to the options of shellcheck is as easy as the installation. A quick '--help' argument provides us with the list of execution options.

shellcheck --help

Usage

So far so good. Now lets look at how it executes. I has a BASH file in my home directory that provides the SVN history of a file so I used that as a demo.

shellcheck ./svn_file_history.sh

Hey, that's pretty neat. Give it is standard output it would be easy to pipe these messages to a reporting system or quality gate process. Nice.

Closing

shellcheck is one of those tools that makes life much easier, as long as you know about it before trying to write your own monster of a syntax checker. Easy to install, easy to use, easy to integrate with it becomes yet another quality and security insurance step along the development pipeline.

Share your favorite linter in the comments so we can all learn.

Additional Resources

Top comments (15)

Alexey Melezhik • Nov 28 '18 • Edited

Hi David. From the post it is not clear why and what you want to check in Bash scripts? Code style, code policies, code smell? Security checks? Cycle dependencies? All that kind of checks make sense for high level languages aiming to apply strict patterns for team development. I am kinda sceptic it is all useful for Bash scripting. Sorry but it seems overkill or irrelevant for me.

David J Eddy • Nov 28 '18

Very valid points, I apologize for not being clear about the goal during the post. Above anything else I try out and share tools to assist fellow developers produce better quality code; that is code that requires less effort to maintenance, accepts changes with lower chance of unwanted side effects, is consistent in code syntax style, and conforms the best practices. (That last one is very subjective I know, but we can try.)

To those ends I found shellcheck an interesting tool to help normalize bash script logic in a predictable manner. As stated in the tools readme:

The goals of ShellCheck are

To point out and clarify typical beginner's syntax issues that cause a shell to give cryptic error messages.
To point out and clarify typical intermediate level semantic problems that cause a shell to behave strangely and counter-intuitively.
To point out subtle caveats, corner cases and pitfalls that may cause an advanced user's otherwise working script to fail under future circumstances.

The readme even contains a gallery of examples, some of which I am guitly of on a number of occasions. github.com/koalaman/shellcheck/blo...

Thank you again for the input. I strive to become a better writer and your input if a big help! Keep up the good work on your posts as well!

Alexey Melezhik • Nov 28 '18

Thank you, David

Boris Jamot ✊ / • Nov 27 '18

Great tool. I use it in a pre-commit git hook to automatically check all my shell scripts.

David J Eddy • Nov 27 '18

That is an excellent usage example Boris! Would you mind sharing your pre_commit logic for others to see how you do this.

Boris Jamot ✊ / • Nov 27 '18

Hi @david_j_eddy , I have no access to my git repo at work now, but the idea is really simple. I use the following command to get all the staged files in my git index:

STAGED_FILES_CMD=`git diff --cached --name-only --diff-filter=ACMR HEAD | grep \\\\.sh`

Then, I loop on these files and run shellcheck against each of them:

for FILE in $STAGED_FILES_CMD
do
    shellcheck $FILE
    if [ $? != 0 ]
    then
        echo "Fix the errors before commit."
        exit 1
    fi
done

The idea being to break the commit in case of any error raised by shellcheck and let the developer correct before committing again.

It's nothing really extraordinary, but tools like this make your dev life easier every day.

For those who want to check everything at pre-commit stage, have a look at swagger-cli for your swagger spec, dockerlint for your Dockerfiles and composer validate for your composer.json.

kip • Nov 27 '18

Why did you use grep \\\\.sh 4 backslashes ?

Could be grep .sh$, no ?

Boris Jamot ✊ / • Nov 28 '18

I don't remember why, but I know I have to.
Maybe there's a simpler way.

Daniel Levy • Nov 30 '18

Can you add install instructions (after the apt install bit) for the OSX peeps:

brew install shellcheck

Great article!!!

Bash doesn't get enough love.

Nick Cinger • Nov 27 '18

Ooo this is excellent, thanks for sharing! Can't believe I never thought of looking for a linter for my bash scripts.

Thomas H Jones II • Nov 30 '18 • Edited

It can be an annoying linter. Fortunately, it's easy enough to disable some of the annoyances. E.g., I'm a frequent user of the <TARGET_ACTION> && <SUCCESS_ACTION> || <ERROR_ACTION> construct. Unless you put # shellcheck disable=<LINT_ID> in either your script-header or peppered with your violating-construct, you're going to be annoyed. Especially fun when you've got git server-side commit-validation going on and you see that little red x come up next to a commit that is doing exactly what you want it to do and how you want it done.

Nick Cinger • Dec 2 '18

Well, I guess it's kind of the point for linters to be somewhat annoying :D
Thanks for the tips, will keep them in mind!

Thomas H Jones II • Dec 3 '18

Yeah, it's just that some of the linters' determinations are a matter of philosophical disagreement. I mean, sure, warn me that a given thing exposes me to corner-cases, but don't make my damned commit fail.

Presumably, some of these linters will end up being enhanced with ML so that they have enough "intelligence" to know "in this case, we don't need to fail this".