Discussion on: Your bash scripts are rubbish, use another language

View post

I saw once a Go library designed to mimic a UNIX shell and some of UNIX utilities. The idea was to use Go for what would traditionally be shell scripts.

The main two strengths of a UNIX shell are effortless pipes (try that with Python!) and external command execution as a first-class citizen.

Tai Kedzierski • Sep 7 '20

Yeah, doing pipes is a nightmare in anything other than shells, and the reason why I still love using shell scripts - chaining tools.

But there's a lot of logic that can often be sub-moduled out to other languages to make a cohesive whole. Some systems people don't like the idea of a program not being self-contained in a single file and you end up with 2000+ lines of sub-optimal code at best, more often than not downright horrendous though... take a peek at the install scripts of some of your favorite software to see what I mean...

xtofl • Sep 7 '20

Most modern languages allow function composition (pipes between functions). And they have extensive, stable repositories of libraries.

This boils down to shipping the right packages with the distribution - something Linux distro's already have, of course.

Cliff • Sep 10 '20 • Edited

Python's implementation support for pipes is reasonable and not that hard to use, but it's painful compared to actually using pipes directly in bash/tcsh/etc. Even using FIFOs on the command line feels more natural than the way you have to compose them in Python. The Unix way of composing operations is probably the best implementation of functional/stream programming ever.

And I am a Python lover, so don't think I'm knocking Python!

xtofl • Sep 10 '20

Absolutely right if you're going in and out of Python to create a pipeline of full fledged processes (I suppose you refer to the subprocess module and alike).

What I meant is: you can stay in the environment that supports 'tacit/point-free programming', and make sure you have everything you need:

# 'function' composition is a dash
# functions are processes
tac logs.txt | grep "http://" | xargs wget

Could be just as easy:

# function composition not built-in
# functions are native
compose( read_file("logs.txt"), filter_lines("http://"), wget )

provided you have these functions lying around somewhere.

Granted: Python has these FP concepts built-in, but not as nicely as the unix way. There are better languages for that: Haskell, F#, erlang...

-- my haskell is rusty - but function composition is a dot:
-- functions are native
pipeline = read_file . (filter_lines "http://") . web_get
pipeline "logs.txt"

Interestingly enough, someone already thought up a Haskell shell: Turtle

Cliff • Sep 10 '20

Yes, I was talking about composing processes like you would at the shell.

I see what you're saying about point-free programming, though. I still think the Unix style is the cleanest, most natural implementation of point-free programming, and I think the fact that it is a genuine stream of processing is a big point in its camp. However, if your Haskell example is accurate, I like it. The examples the Wikipedia article give seem less intuitive and a lot more LISP-y.

I think most programmers would probably find the use of compose in Python a lot less intuitive than nested generator functions, and it's certainly an inelegant implementation of point-free programming. I also wonder if it can eliminate some of the advantages of the generators? It probably doesn't based on the sample implementation, but I'd have to think carefully about if applying partial like that would have unintended consequences, at least in some cases.

xtofl • Sep 10 '20 • Edited

I think so, too. You can make a nice 'fluent' DSL out of it, though.


class Chain:
        def __init__(self, *fns):
                self.fns = fns
        def __or__(self, fn):
                return Chain(*self.fns, fn)
        def __call__(self, arg):
                return reduce(lambda ret, f: f(ret), self.fns, arg)

Chain() | read_file | create_filter("https://") | web_get

def double(x):
        return 2*x

def fromstr(s):
        return int(s)

def inc(x):
        return x+1

def repeat(n):
        return lambda s: s * n

c = Chain() | fromstr | inc | double | str | repeat(3)

assert c("1") == "444"
assert c("20") == "424242"

Tai Kedzierski • Sep 10 '20

..... ingenious.

I'm still trying to brain this, its possibilities and its limitations but... wow.

A bit of commentary would be very welcome :-)

xtofl • Sep 10 '20

I'll expand it in a full fledged post :) Or 'leave it as an exercise'?

Cliff • Sep 10 '20

I either love this or hate it, I can't decide. Bravo, sir!

xtofl • Sep 16 '20

Somehow, I have hit the 'publish' button on dev.to/xtofl/i-want-my-bash-pipe-34i2.

Cliff • Sep 16 '20

I've only skimmed the article so far, but it looks like a good one. I like the title! 😁

Tai Kedzierski • Dec 9 '20 • Edited

@xtofl , @Cliff , I have finally gotten round to this, I think you will be gleefully dismayed.

dev.to/taikedz/shellpipe-shellpipe...