5 JavaScript Pipelining Techniques

Photo by Quinten de Graaf on Unsplash

Pipelining using 5 different techniques, current and future.

We'll refactor two chunks of code lifted from the TC39 pipeline proposal:

i) "Side-effect" chunk

const envarString = Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
const consoleText = `$ ${envarString}`
const coloredConsoleText = chalk.dim(consoleText, 'node', args.join(' '))
console.log(coloredConsoleText)

ii) "Pure" chunk

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList

Each has a "chain" of operations used one after the other against the previous value.

The first chunk logs the final value, the second returns it:

1. envars > envarString > consoleText > coloredConsoleText > log
2. values > keys > uniqueKeys > items > unorderedList > return

In both cases, the final value is the only one we're really interested in, so this makes them candidates for pipelining!

Let's start with...

i) The "Side-effect" chunk

1. Using let tmp

The simplest way to drop those temporary variables is to declare a mutable let tmp and continuously reassign it:

let tmp = envars
tmp = Object.keys(tmp)
tmp = tmp.map(envar => `${envar}=${envars[envar]}`)
tmp = tmp.join(' ')
tmp = `$ ${tmp}`
tmp = chalk.dim(tmp, 'node', args.join(' '))
console.log(tmp)

It'll work, but maybe there are less error-prone ways of achieving the same thing. Also, mutable variables aren't exactly en vogue these days. 🤔

2. Using Promise

We can use Promise.resolve and a sequence of then's to keep the scope of each temporary variable under control:

Promise.resolve(envars)
  .then(_ => Object.keys(_))
  .then(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .then(_ => _.join(' '))
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(_ => console.log(_))

No polluting the enclosing scope with tmp here! A Promise carries the idea of "piping" from envars all the way to logging the final colourised output without overwriting a temporary variable.

Not quite how we'd typically use Promise perhaps, but since many of us are familiar with how they chain together it's a useful jumping-off point for understanding pipelining for those not already familiar.

By the way, we could have used Object.keys and console.log first-class:

Promise.resolve(envars)
  .then(Object.keys) // instead of: _ => Object.keys(_)
  .then(console.log) // instead of: _ => console.log(_)

But I'll avoid using this "tacit" style here.

I'm also intentionally avoiding:

Promise.resolve(
  Object.keys(envars)
    .map(envar => `${envar}=${envars[envar]}`)
    .join(' ')
)
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(console.log)

Instead I'll try to keep the first level of indentation equal, as I think it helps convey the full pipelined-operation a bit better.

Anyway, using a Promise isn't ideal if we want a synchronous side-effect.

Popping an await before the whole chain is possible of course, but only if the pipeline sits inside an async function itself, which might not be what we want.

So let's try some synchronous pipelining techniques!

3. Using pipe()

With this magic spell:

function pipe(x, ...fns) {
  return fns.reduce((g, f) => f(g), x)
}

...we can have:

pipe(
  envars,
  _ => Object.keys(_),
  _ => _.map(envar => `${envar}=${envars[envar]}`),
  _ => _.join(' '),
  _ => `$ ${_}`,
  _ => chalk.dim(_, 'node', args.join(' ')),
  _ => console.log(_)
)

We dropped all those .then()'s and left the lambdas (arrow-functions) behind as arguments to pipe that will run in sequence, with the first argument providing the starting value to the first lambda.

Handy!

4. Using Hack-pipes

If you're using Babel or living in a future where the TC39 pipeline proposal has landed, you can use Hack-pipes:

envars
  |> Object.keys(^)
  |> ^.map(envar => `${envar}=${envars[envar]}`)
  |> ^.join(' ')
  |> `$ ${^}`
  |> chalk.dim(^, 'node', args.join(' '))
  |> console.log(^)

Terse! And starting to look like an actual pipe on the left there, no?

Notice that a token ^ acts as our "previous value" variable when we use |>, just like when we used _ or tmp previously.

5. Using the Identity Functor

Let's cast another magic spell:

const Box = x => ({
  map: f => Box(f(x))
})

...and make a pipeline with it:

Box(envars)
  .map(_ => Object.keys(_))
  .map(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .map(_ => _.join(' '))
  .map(_ => `$ ${_}`)
  .map(_ => chalk.dim(_, 'node', args.join(' ')))
  .map(_ => console.log(_))

Looks suspiciously like the Promise pipeline, except then is replaced with map. 🤔

So that's 5 different pipelining techniques! We'll apply them now in reverse-order for...

ii) The "Pure" chunk

Here's the reference code again as a reminder:

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList

To start, we'll first make Box a monad:

const Box = x => ({
  map: f => Box(f(x)),
  chain: f => f(x) // there we go
})

By adding chain we can return the JSX at the end of a pipeline without transforming it into yet another Box (which didn't really matter in the side-effect chunk since we weren't returning anything):

return Box(values)
  .map(_ => Object.keys(_))
  .map(_ => Array.from(new Set(_)))
  .map(_ => _.map(item => <li>{item}</li>))
  .chain(_ => <ul>{_}</ul>)

Kinda feels like the Promise.resolve pipeline if it had an await at the beginning, eh? Instead it's a Box with a chain at the end. 🤔

And synchronous too, like pipe()!

Speaking of which, let's go back and use it now:

Using pipe()

return pipe(
  values,
  _ => Object.keys(_),
  _ => Array.from(new Set(_)),
  _ => _.map(item => <li>{item}</li>),
  _ => <ul>{_}</ul>
)

Fairly similar to the side-effect chunk, except to reveal that yes, pipe will indeed give us back the value returned by the last lambda in the chain. (That lovely <ul /> in this case.)

Using Promise

Back in the land of async, does it make sense to return JSX from a Promise? I'll leave the morals of it up to you, but here it is anyway:

return await Promise.resolve(values)
  .then(_ => Object.keys(_))
  .then(_ => Array.from(new Set(_)))
  .then(_ => _.map(item => <li>{item}</li>))
  .then(_ => <ul>{_}</ul>)

(await thrown-in just to communicate intention, but it's not required.)

Lastly, let's bring it right back to let tmp:

Using let tmp

let tmp = values
tmp = Object.keys(tmp)
tmp = Array.from(new Set(tmp))
tmp = tmp.map(item => <li>{item}</li>)
tmp = <ul>{tmp}</ul>
return tmp

And that's where we came in!

Conclusion

All in all we covered 5 different ways of pipelining: A way of transforming one value into another in a sequence of steps without worrying about what to call the bits in between.

1. let tmp
2. Promise#then
3. pipe(startingValue, ...throughTheseFunctions)
4. Hack |> pipes(^)
5. Identity Functor/Monad (Box#map/chain)

If you learned something new or have something to follow-up with, please drop a comment below. In any case, thanks for reading!