Your bash scripts are rubbish, use another language

(Headline photo from nixcraft's post to which I was reacting below)

TLDR:

• Shell scripting is real programming
• If you want to write shell script learn it properly
• Use programming best practices like you would with other languages
• Source-control it
• It's weirdly idiosyncratic and needs extra attention
• If the above doesn't suit you, you'll be better served by a less idiosyncratic language

So the below was a rant I posted in response to some pushback - someone suggested using Python instead of bash, and a few people complained about how it's overkill, how there are two versions of Python you need to get right, or you have to get it onto the machines in the first place, or suchlike.

I love shell scripting, and I still use it a lot, but I'm no fool. There are so many issues with it that blindly defending it for all use cases is foolhardy.

Not least because in company settings, most other people you will work with haven't the slightest idea what they're doing with shell scripts. They already get brownie points for thinking of putting the scripts in source control, kind a like getting grades for writing your name on your test.

SHELL SCRIPTING IS REAL PROGRAMMING. It should be source controlled, code reviewed and written to clean, maintainable standards. Because that code is meant for production.

So this is what I retorted:

If it's a personal machine, install Python - or other language of choice.

If it's the enterprise machines, have a policy to ensure Python - or other language of choice.

Shell is good and great, I'm a command line junkie myself, and I still turn to shell scripts for a lot of my work, but a shell script that's for more than wrapping a long pipe or two quickly becomes madness unless you actually put in the time and effort to learn the language PROPERLY. (The article feature pic is an example of utterly shitty code and their problem is not the filename spaces YET, but their RUBBISH handling of variables, and the lack of any effort to write cleanly)

I am an extensive bash scripter, and became so historically only for the reasons listed around here, which I once saw as valid reasons. They can all be worked around - and the overhead of solving "language X not on our machine farm" is often better return on effort than the years of unmaintainable brittle shell scripts you've been writing dozens of and never maintaining, or your colleagues cannot fathom and won't touch for love nor money. Unfoathomable shell scripts that run the backbones of deployments, builds, farm management and more - the backbones of many a company.

I've seen bash scripts by seasoned developers, and those are also utter trash. Your code may be cleaner than the average, but that's an extremely low bar.

Shell scripting is good and powerful in its own right, it is true, and I have advocated that people give it a proper try and actually learn it, but the sad reality is that nobody does. If it's any "proper" language, they learn the ins and outs gladly, by peer expectation or inherited bias; but shell, even if you learn it properly, someone else will come f/ck up your clean code because they can't be arsed.

I am endlessly pushing for developers and admins to actually learn to use bash/shell properly, make use of functions, encapsulate steps of logic, write clean code. "It's just a shell script," "that's overkill," "it's fine like this," "I don't want to sink time into learning this, it's not a real language anyway."

The other truth is that, in the sysadmins space, most can't even write clean Python/Ruby/Perl/PHP/JavaScript/chosen-lang either, and given the number of gotchas and things shell lets you get away with until you hit a catastrophic bug (of the coder's carelessness, the misunderstood shell behaviour is documented) (Steam bug anyone?), they'd be better off in a safer environment than shell scripting.

Shell scripting perennial issues that are not the fault of the coder:

• will gladly let you get away by default with undefined variables (unless you explicitly set -ue)
• comparison vs assignment is the only place where space matters (a=b and a = b are COMPLETELY different statements, wtf)
• you cannot return arrays from functions, only a stream of text (the power and the Achille's Heel) (this one issue compounds many of the others, by preventing workaround functions from being written)
• arrays cannot be passed down to functions as distinct items alongside other arguments (you can use references as a way around, but how many bash scripters know those?) (easier to use global variables right? yuck)
• variables are global by default. unless you make your iteration counter local, you stand to see some weeeeird bugs
• string splitting is done around an inherent part of the string, not as a a function operating on it (do we all know about IFS, does everybody know how to use it? didn't think so)
• Is it really the shell you thought it you were running? Ever deployed bash scripts only to find that the only interpreter on the machine is sh? Or the environment forces you into sh by default? Or that in fact you're not running bash but ash? Or maybe the system default is dash. Anyhow, you have to write everything now in plain sh and lose any improvements that bash ever brought that make the task more bearable.
• Inconsistent environments for common commands are rife. Your script uses the "mail" function? GNU or BSD, which options to use? You use netcat? Which variant, which options? You use tar, grep, rsync? You using GNU, BSD or Busybox implementations? (these variations happen endlessly when mixing Ubuntu, CentOS and Alpine deployments, and that's just the surface)
• (I scoff at any pushback of "ensuring the right version of Python on the company systems")
• attempting anything remotely event-driven yields a nasty pile of workarounds (I've tried, with muted success) Granted, this is a space which shell is definitely not designed for, but that's to say how far I tried to do everything in shell at one point. It's possible, but it's damn hard work where another language would have been better.

Most fundamentally, the view of shell as "not a proper language" hampers any impetus at large to learn it correctly and extensively, and understand its own idiosyncracies. At least with one of the other languages, developers have an inherited mindset that their skill in that language needs continual improvement, and will work towards this.

I still write tons of bash scripting. I love it. But recommending other people use another language is much more sane. Personally, I chose Python too. But in the end, I wouldn't recommend it unless you are going to do your darndest to learn. It. Properly.