Composable Callbacks

A Promise implementation in under sixty characters

You’ve heard it before: nested callbacks don't scale, and leave you in callback hell. So in this article, we will see how far we can push the callbacks approach, and see if we can reach a point where they're just as scalable/composable as Promises.

Let’s start by thinking about how we define functions for a moment. A regular addition function might be defined as such:

//    add :: (Number, Number) -> Number
const add = (a, b) => a + b

But we can also define it slightly differently, as a function that takes a single argument, and returns a function that takes another argument, which in turn returns the result of adding the two arguments together:

//    add :: Number -> Number -> Number
const add = a => b => a + b

Many of you will recognise the latter as being the "curried" variant of the first. You can read up on currying in Chapter 4 of the Mostly Adequate Guide.

Defining the function this way unlocks some new ways of using the function. For example, we can easily define a new add5 function by applying add to 5, for mapping over an Array, for example:

[1, 2, 3, 4, 5] .map (add (5))
//> [6, 7, 8, 9, 10]

We are going to define all of our functions in the curried way, which is the first step to making callbacks more managable.

Let’s take a basic example of an asynchronous program using callbacks:

fs.readFile ('input.txt', 'utf8', (e, input) => {
  if (e) console.error (e)
  else fs.readFile (`${input}-file.txt`, 'utf8', (e, result) => {
    if (e) console.error (e)
    else console.log (result)
  })
})

When we do it like this, it sends us straight to callback hell. Let’s see what we can do after creating a curried version of readFile. (I'll also simplify the callback a little bit by taking away the error argument. We’ll get back to this near the end of this article.)

//    readFile :: String -> String -> (String -> Undefined) -> Undefined
const readFile = encoding => filename => callback => {
  fs.readFile (filename, encoding, (e, contents) => {
    if (e) console.error (e)
    else callback (contents)
  })
}

By now you might be wondering what those ::-comments are doing above every function. They are type definitions in a neat type language called Hindley Milner. The "HM" language is very succinct when describing curried functions in particular. If you take a short moment to understand how it works, it’ll help you to see more clearly what’s happening with our functions. You can read more about it in Chapter 7 of the Mostly Adequate Guide.

You may also have noticed that I’ve shuffled the argument order a little bit. This is to be more optimised for partial application. This new definition of readFile allows us to partially apply it, and not pass the callback yet.

//    readText :: String -> (String -> Undefined) -> Undefined
const readText = readFile ('utf8')

//    step1 :: (String -> Undefined) -> Undefined
const step1 = readText ('input.txt')

//    step2 :: String -> (String -> Undefined) -> Undefined
const step2 = input => readText (`${input}-file.txt`)

//    step3 :: String -> Undefined
const step3 = console.log

Let’s look at what we’ve created here:

1. readText: A partial application of readFile, with the encoding. We can just reuse it without having to pass 'utf8' everywhere.
2. step1: A partial application of readText. The only argument left now is the actual callback. So step1 becomes a function that takes a callback to which the contents of input.txt will be passed.
3. step2: A function that takes some input and uses it to read a file with a name containing said input. It doesn’t actually read any files though, it just partially applies readText again and returns the function waiting for a callback.
4. step3: Just an alias to console.log for illustrative purposes. It used to be nested inside the callback to step2.

Now if we study the signatures of each of these functions, we’ll find that they all plug into each other quite nicely. step3 could be used as a callback for step2, and the entirety of step2 could be used as the argument to step1. Doing that would require a lot of nesting, but we can define a helper function which "flattens" the nesting. Let’s call it then ;)

//    then :: (a -> (b -> Undefined) -> Undefined)
//         -> (     (a -> Undefined) -> Undefined)
//         ->       (b -> Undefined) -> Undefined
const then = transform => run => callback => run (value => transform (value) (callback))

Our then function takes three arguments:

1. A transform function, which receives a value and produces a function waiting for its callback. Our step2 actually fits this description.
2. A function still waiting for its callback. Our step1 fits this.
3. A callback. Our step3 fits this one.

What’s cool about this function, is that when we partially apply it with its first two arguments, we get back a function that fits precisely into the second argument of then again. This is what will allow us to stick multiple "steps" next to one another, rather then nested within each other.

You might have noticed from the signature that there are three instances of (a -> Undefined) -> Undefined. It would become a lot more clear if we gave this particular type a special name, and use that in our types instead. Let’s create a simple alias (Future) for the callback-taking function. The constructor for this type has no implementation: it just returns the input (because it’s an alias). But it will help to make our code clearer. Let’s redefine our then function with more clearly named types.

//    Future :: ((a -> Undefined) -> Undefined) -> Future a
const Future = x => x

//    then :: (a -> Future b) -> Future a -> Future b
const then = transform => future => Future (callback => {
  future (value => transform (value) (callback))
})

This new then function is exactly the same as the previous one, but it suddenly becomes a lot clearer what it’s doing: It takes a function that creates a Future, and it takes a Future and finally returns a new Future. Speaking in these terms, step1 is a Future of a String, and step2 returns a Future of a String, after taking a String.

Equipped with our then function and type alias, we can rewrite our callback hell program.

//    Future :: ((a -> Undefined) -> Undefined) -> Future a
const Future = x => x

//    then :: (a -> Future b) -> Future a -> Future b
const then = transform => future => Future (callback => {
  future (value => transform (value) (callback))
})

//    readFile :: String -> String -> Future String
const readFile = encoding => filename => Future (callback => {
  fs.readFile (filename, encoding, (e, contents) => {
    if (e) console.error (e)
    else callback (contents)
  })
})

//    readText :: String -> Future String
const readText = readFile ('utf8')

//    step1 :: Future String
const step1 = readText ('input.txt')

//    step2 :: String -> Future String
const step2 = input => readText (`${input}-file.txt`)

//    program :: Future String
const program = then (step2) (step1)


program (console.log)

Our then function is actually doing mathematically accurate flat-mapping. Just see what happens if we replace Future by Array in the type signature. The abstract interface behind flat-map-able types is called "Monad" (because the mathematicians beat us to it).

The fact that we could use program as an argument to then in order to compose a bigger program means we have achieved our goal of making callbacks composable just like Promises.

Let’s get back to this console.error-bit though, because we’ve lost the ability to manually handle errors. We can add that back, simply by having our function take two callbacks instead of one.

//    Future :: (((a -> Undefined) -> Undefined)
//           -> ((b -> Undefined) -> Undefined))
//           -> Future a b
const Future = x => x

//    then :: (b -> Future a c) -> Future a b -> Future a c
const then = transform => future => Future (reject => resolve => {
  future (reject) (value => transform (value) (reject) (resolve))
})

//    readFile :: String -> String -> Future Error String
const readFile = encoding => filename => Future (reject => resolve => {
  fs.readFile (filename, encoding, (e, contents) => {
    if (e) reject (e)
    else resolve (contents)
  })
})

//    readText :: String -> Future Error String
const readText = readFile ('utf8')

//    step1 :: Future Error String
const step1 = readText ('input.txt')

//    step2 :: String -> Future Error String
const step2 = input => readText (`${input}-file.txt`)

//    program :: Future Error String
const program = then (step2) (step1)


program (console.error) (console.log)

The then function in our last example gives us similar asynchronous function composition and error handling benefits to those that Promises give us, in a function that can be written in under sixty characters:

const then = f => m => l => r => m (l) (x => f (x) (l) (r))

It even does away with many of the problems that Promises have. But it does leave some things to be desired, such as good performance and stack safety. For our purpose though, it will do just fine: to solve the Async Problem and demonstrate that callbacks are just as composable as synchronous code.

The original version of Fluture was pretty much implemented like this, except that then is called chain.

Solving the Async Problem

The Async Problem is a little challenge set to identify how well an abstraction allows the user to break an asynchronous algorithm into small, manageable sub-problems. To conclude this post, let’s dive into the depths and solve it with callbacks.

//    pipe :: Array (Any -> Any) -> Any -> Any
const pipe = fs => x => fs.reduce ((y, f) => f (y), x)

//    lmap :: (a -> b) -> Array a -> Array b
const lmap = f => xs => xs.map (f)

//    append :: a -> Array a -> Array a
const append = x => xs => [...xs, x]



//    pure :: b -> Future a b
const pure = x => l => r => r (x)

//    then :: (b -> Future a c) -> Future a b -> Future a c
const then = f => m => l => r => m (l) (x => f (x) (l) (r))

//    fmap :: (b -> c) -> Future a b -> Future a c
const fmap = f => then (x => pure (f (x)))

//    all :: Array (Future a b) -> Future a (Array b)
//        -- Note: This implementation resolves things in sequence for brevity.
const all = ms => ms.reduce
  ((mxs, mx) => then (x => fmap (append (x)) (mxs)) (mx), pure ([]))



const filesystem = require ('fs')
const path = require ('path')

//    readFile :: String -> String -> Future Error String
const readFile = encoding => filename => l => r => {
  filesystem.readFile (filename, encoding, (e, contents) => {
    if (e) l (e)
    else r (contents)
  })
}

//    readText :: String -> Future Error String
const readText = readFile ('utf8')

//    lines :: String -> Array String
const lines = s => s.split ('\n')

//    unlines :: Array String -> String
const unlines = ss => ss.join ('\n')

//concatFiles :: (String -> String) -> Future Error String
const concatFiles = path =>
  pipe ([ path
        , readText
        , fmap (lines)
        , fmap (lmap (path))
        , fmap (lmap (readText))
        , then (all)
        , fmap (unlines) ])
       ('index.txt')


const main = () => {
  concatFiles (x => path.resolve (process.argv[2], x))
              (e => { process.stderr.write (e.message); process.exit (1) })
              (x => { process.stdout.write (x); process.exit (0) })
}

main()