Composability: from Callbacks to Categories in ES6

Promises are a well known solution to the Callback hell problem that arises in asynchronous JavaScript programs.

Borrowing some ideas from Functional languages, I am exploring a different approach to address callback hell in this post. This solution will be more general than Promises, in fact we will take advantage of these ideas to make Promises even more composable.

I use a notation similar to Haskell’s. But in many ways I will divert from rigid Haskell notation everywhere that I think it helps.

You only need to be familiar with Callbacks, Promises, and ES6 anonymous function (lambda) syntax to follow this post. We will be playing with some ideas from Functional Programming (FP).

TOC:

• Callbacks
• Composable Callback class
• Composable Callback class and Promise class are Monads
• Monads
• Categories
• Func Category
• Kleisli Category

Callbacks

Many programming languages utilize callbacks for continuation. When we encounter:

    db.getSomething(callback)

We know that db.getSomething is a void function, it executes some code (potentially asynchronously) and passes the result of operation to the callback function to handle it.

Callbacks in JavaScript are more powerful than just continuation. We can model a function that returns more than one result using callbacks:

function next2(x, callback) {
  callback(x + 1, x + 2)
}

next2(10, (eleven, twelve) => …)

In fact this is how callbacks are used for propagating errors. By convention the first argument to a callback is the error (if any) that was produced by the operation:

function sqrt(x, callback) { 
  if(x < 0) 
    callback(Error('Sqrt of negative value', null))
  else 
    callback(null, Math.sqrt(x))
}

If the operation produces any error, we always ignore the second argument (whatever result it might had produced).

Callback hell happens when we want to pass the result of the first async operation to the second async function and to the third and so on:

function myLongOperation(userId, callback) {
  db.getUser(userId, (error, user) => {
    if(!!error)
      return callback(error, null)
    else
      api.generateMessage(user, (error, message) => { 
          if(!!error)
            return callback(error, null) 
          else
            client.sendMessage(message, callback)
      })
  })
}

Here we are passing userId to getUser in order to get the user asynchronously then we’re passing the user to generateMessage to … You know instead of narrating it in words let’s use some notation to describe this process:

userId → (getUser ⋙ generateMessage ⋙ sendMessage)

The above notation perfectly describes what our myLongOperation function does. Error handling at every step is clearly redundant. Promise fans know that this notation is very similar to (but not exactly the same as) what we do with Promises:

    getUser(userId).then(generateMessage).then(sendMessage)

Promise.then takes care of error handling and chaining.

But our goal is to come up with a construct that is more general than Promises.

In our notation ⋙ is a way of composing (piping async functions). We will discuss it later.

x → y denote a function from x to y. For example:

const plus1 = x => x + 1
//        Number → Number

myLongOperation is a function from userId to a series of async operations, hence:

    userId → ( … ⋙ … ⋙ … )

Haskellers know that this is not a proper type definition. But for our purpose this notation perfectly describes myLongOperation function.

Composable Callback

Promises are not the only solution to the callback hell problem. Promises provide more features than composability (for example they have an internal state which remembers if they’ve been resolved or not plus some other kinks).

Let’s define a bare minimum solution to the callback hell problem by implementing a “composable Callback” class:

class Callback {
  constructor(f) {

    // this.run = f
    this.run = callback => {
      try {
        f(callback)
      } catch (ex) {
        callback(ex, null)
      }
    }

    // this.map = ...
    // this.bind = ...

    // this :: Callback x
    // x -> (y || Callback y) -> Callback y
    this.then = g => new Callback(callback => {
      this.run((error, ...result) => {
        if(!!error) {
          callback(error, null)
        } else {
          try {
            const y = g(...result)
            if (y instanceof Callback) {
              y.run(callback)
            } else {
              callback(null, y)
            }
          } catch(ex) {
            callback(ex, null) 
          }
        }
      })
    })

    this.bindTo = g => this.bind(Callback.from(g))
  }
}

// x -> Callback x
Callback.pure = x => new Callback(cb => cb(null, x))

Callback.resolve = Callback.pure

// Callback.from casts f into a Callback instance, where
// f is a function that takes x and a callback function
Callback.from = f => (...x) => new Callback(cb => f(...x, cb))

Check out the full code here.

Callback class provides this interface:

• constructor takes an async function (f which will produce either an error or a value x)

• run instance function: receives a callback function and feed it to the f

• map instance function analogous to Array.map, transforms the x (the result of f)

• bind instance function is similar to Promise.then, it is used for chaining Callback instances

• then instance function corresponds to Promise.then; it’s a combination of map and bind functions.

• bindTo instance function is a utility for chaining Callback instances to normal async functions

• pure (alias resolve) static function is similar to Promise.resolve, it creates an instance of Callback.

• from static function casts an async function to an instance of Callback.

It’s not an accident that Callback interface resembles the interface of Promise. pure is an alias for resolve. If you’ve ever used Promise.resolve() you know what Callback.pure does. I think pure is a better name for our Callback class. Similarly Callback.then is analogous to Promise.then. I consciously avoid Callback.map and Callback. bind.functions in this post, because *Callback.then *is sufficient as it both maps and binds.

We start with Callback.pure. It puts a value into a new Callback instance:

    Callback.pure(64).run((error, result) => console.log(result))

Will log 64 in the Console.

This is how we can compose Callback.pure(64).with our sqrt function:

  Callback.pure(64)
    .bindTo(sqrt)
  .run((error, result) => console.log(error || result))

Under the hood, bindTo casts sqrt to an instance of Callback. The above snippet is equivalent to the followings:

Callback.pure(64)
  .then(Callback.from(sqrt))
.run((error, result) => console.log(error || result))

Callback.pure(64)
  .then(x => new Callback(cb => sqrt(x, cb)))
.run((error, result) => console.log(error || result))

Using Callback class our myLongOperation function can be written more concisely as:

    // userId → (getUser ⋙ genMessage ⋙ sendMessage)

    const myLongOperation = (userId, callback) => 
      Callback.pure(userId)
        .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage)
      .run(callback)

Notice how closely this implementation matches the notation.

• .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage).is denoted by (getUser ⋙ genMessage ⋙ sendMessage)

• But Callback.pure(userId) seems unnecessary. (userId → (…) is the denotation of the whole myLongOperation function.) We will back to this point later.

Our changes to myLongOperation function are not visible to the user of this function. myLongOperation is still an async function that takes a userId and a callback.

We can always use bindTo utility to chain Callback instances to async functions. For example let’s assume we have another async function like getUserId(userName, callback) which we want to pipe its result into myLongOperation:

const messageUser = (userName, callback) =>
  Callback.pure(userName)
  .bindTo(getUserId)
  .bindTo(myLongOperation)
  .run(callback)

Notice that now run() is being called twice: once inside myLongOperation and the second time inside messageUser. There’s a catch here. Nothing really happens unless we call run().

const proc = Callback.pure(5)
  .then(x => new Callback(cb => {
    console.log(`binding ${x} to x + 1`)
    setTimeout(() => cb(null, x + 1), 100)
  }))

console.log() in the third line only gets called after proc.run(). Try it here:

proc (as an instance of Callback class) represents the instructions to an async operation that JavaScript only executes after run() is called. This is very different from Promises:

const prom = new Promise(resolve => {
  console.log('Promise executes immediately')
  resolve()
})

When you run this snippet, ‘Promise executes immediately’ is logged immediately, even if you never use the prom or prom.then(x => …).

So let’s change our myLongOperation function to return an instance of Callback (we can save one call to run() this way):

// userId → (getUser ⋙ genMessage ⋙ sendMessage)

const myLongOperation = userId => 
  Callback.pure(userId)
  .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage)

Now this definition matches the notation even better since we eliminated the callback function completely.

In the same spirit, we update our messageUser function:

// userName → (getUserId ⋙ myLongOperation)

const messageUser = userName =>
  Callback.pure(userName).bindTo(getUserId).then(myLongOperation)

We changed the last bindTo().to then(), because now our updated myLongOperation is a function that returns an instance of Callback (remember originally before the change, it was a void function that was taking a callback in its second argument).

This is how we can use messageUser :

messageUser(userName).run((error, result) => ...)

We call run() only at the end of the operation. run() executes the operation and returns the result in its callback argument.

We achieved composability and avoided callback hell without resorting to Promises. Check out the full example here:

Functional programmers know that there must be some eta reduction to convert

myLongOperation(userId) = userId → (getUser ⋙ genMessage ⋙ sendMessage) to
myLongOperation = getUser ⋙ genMessage ⋙ sendMessage

In the rest of this post we build some constructs that ultimately enable us to eliminate this redundant parameter.

Callback and Promise are Monads

Our Callback class and the standard Promise class have a lot in common. We call these constructs monad, by which I mean they have a bind (then) function that chains an instance of Callback (or Promise) to a function that returns another instance of Callback (or Promise).

    const proc = Callback.pure(10)
    proc.bind(x => new Callback(…))

We use this notation to describe proc as an instance of Callback monad:

proc :: Callback x

proc.bind :: (x → Callback y) → Callback y

We might read the notation like this:

• proc is a Callback of x

• proc.bind is a (higher order) function that takes a function from x to Callback of y and produces a Callback of y.

For example Callback.pure(10) can be bound to a function that takes a Number and returns a new Callback:

Callback.pure(10)
  .bind(x => new Callback(cb => cb(null, x + 1)))

(remember that resolve() is an alias for pure() and then() has a similar functionality to bind())

Promise class also forms a monad:

Promise.resolve(10)
  .then(x => new Promise(resolve => resolve(x + 1)))

These two expressions look vary similar and that’s indeed the power of monads. Monads provide an abstraction that is useful in many different programs. In our notation the above expressions can be written as:

Monad 10 ≫= (x → Monad (x + 1)) = Monad 11

For Promise Monad:

    Monad 10           ::  Promise.resolve(10)
    ≫=                 ::  .then(…)    
    x → Monad (x + 1)  ::  x => new Promise(resolve => resolve(x + 1))

For Callback Monad:

    Monad 10           ::  Callback.resolve(10) // = Callback.pure(10)
    ≫=                 ::  .then(…)             // = Callback.bind(…)
    x → Monad (x + 1)  ::  x => new Callback(cb => cb(x + 1))

Monads encapsulate a value that can only be retrieved by executing the monad. For Promise monad we retrieve the result of the computation (11) by calling then() function and for our Callback monad we retrieve the result by run().

Monads have this interesting feature that they can be used even if their encapsulated value is not computed yet. We are able to call then() on a Promise and chain it with a function or another Promise even if it’s not completed and the value that it encapsulates is not computed yet. This fact is even more pronounced for our Callback monad. We had seen earlier that Callback doesn’t even bother start computing its result before we call run() (Repl.it demo).

More generally both computations might be denoted as:

Monad x ≫= (x → Monad y) = Monad y

x and y can be of any type. Here they’re Numbers, but they can be String, Boolean, JSON objects, … or even functions or other monads!

What is a Monad?

For our purpose any class that has these two features is a Monad:

• The class must have a way of encapsulating a value (using a static pure() or resolve() function)

• It must provide a way to bind itself with a function that returns another instance of it (using bind() or then())

Monads add extra structure to the value that they’re encapsulating. Different types of Monads provide different structures. The implementation of the pure function is the place to look for these structures.

For Promise:

    Promise.resolve = x => new Promise(res => res(x))

For Callback:

    Callback.pure = x => new Callback(cb => cb(null, x))

For Array:

    Array.of = x => [x] 

For Reader:

    Reader.pure = x => new Reader(env => x)

Click on the links to see the definitions and play with these monads. In this post we only study Promise and Callback.

We can indeed define a monad that has almost no extra structure. This minimum monad is called Identity Monad:

    Identity.pure = x => new Identity(x)

How Identity is useful can be the subject of another post.

Categories

Functional programming focuses on What as opposed to How. We write our program by declaring What we want instead of implementing the procedures step-by-step, detailing how the program works.

For example in this code snippet:

    const myLongOperation = userId => 
      Callback.pure(userId)
      .bindTo(getUser).bindTo(genMesssage).bindTo(sendMessage)

    myLongOperation(123456).run((error, result) => ...)

When we call run() we know that under the hood callbacks and error handling are involved. But we don’t see it and we don’t have to care about these details either. Instead here we wrote our program by describing what we want:

• get a user

• generate a message for that user

• send that message *(and asynchronously return *SendMessageResult)

myLongOperation is a function from userId to Callback of SendMessageResult.

myLongOperation :: userId -> Callback SendMessageResult.

Monadic abstraction focuses on the result of the operation. For instance Callback SendMessageResult only tells us about the result of the action (that is SendMessageResult) not where it comes from. Monads don’t deal with input. They just define a way of composing the outputs using bind (then).

Now let’s try to create an abstraction that takes both input and output into account.

Good old functions

The simplest construct that has an input and an output is a plain simple function.

    const plus1  = x => x + 1
    const times2 = x => x * 2

We can compose functions using function composition, in Math notation:

f o g = x → f(g(x))yeah

In JavaScript (demo):

    const compose = (f, g) => x => f(g(x))

Function composition is a right-to-left operation. compose(f, g)(x), first applies g to x and then f to g(x), hence:

    compose(plus1, times2)(10) == 21

But here I prefer left-to-right composition using pipe operator instead:

f ⋙ g = x → g(f(x))

    const pipe = (f, g) => x => g(f(x))

    pipe(plus1, times2)(10) // == 22

Function composition is not commutative in general by which I mean:

f ⋙ g ≠ g ⋙ f

We have seen that (snippet):

    pipe(plus1, times2)(10) != pipe(times2, plus1)(10)

But there is a special function for which function composition is always commutative, we name this function id:

f ⋙ id = id ⋙ f

And we define it as

    const id = x => x

Easy yeah!

Let’s try it (snippet):

    pipe(times2, id)(10) // == 20
    pipe(id, times2)(10) // == 20

Similar to functions there are other constructs that have these two properties:

• They’re composable (pipe-able)

• They have a special id instance for which the composition is commutative

We call these constructs Category.

Func Category

Let’s make a Category class for normal functions:

class Func {
  constructor(f) {
    // this.run = f
    this.run = x => f(x)

    // this :: Cat (x ↣ y)
    // Cat (y ↣ z) -> Cat (x ↣ z)
    this.pipe = g => new Func(x => g.run(this.run(x)))

    // utility function that pipes Func to a normal function
    // this :: Cat (x ↣ y)
    // (y -> z) -> Cat (x ↣ z)
    this.pipeTo = g => new Func(x => g(this.run(x)))
  }
}
// Cat (x ↣ x)
Func.id = new Func(x => x)

I use funky arrow ↣ to emphasize that Category abstracts a construct with an input and an output.

Func.id is indeed commutative over Func.pipe():

    Func.id.pipe(new Func(x => x * 2)).run(10) // = 20
    new Func(x => x * 2).pipe(Func.id).run(10) // = 20

Note that there’s one and only one instance of Func.id. Func.id is not a function, it is an instance of (member of) Func class.

Func might look like a boring Category since it only wraps normal functions:

    new Func(x => x * 2).run(5) == (x => x * 2)(5)

But Func enables us to pipe (compose) functions in a natural way in JavaScript (JSBin demo):

    new Func(x => x * 2)
      .pipe(new Func(x => x + 1))
      .pipe(new Func(x => Math.sqrt(x)))
    .run(12)  // == 5

Let’s compare the above snippet with a similar code for Promise monad:

    Callback.pure(12)
      .then(x => Promise.resolve(x * 2))
      .then(x => Promise.resolve(x + 1))
      .then(x => Promise.resolve(Math.sqrt(x)))
    .run((error, result) => console.log(result) /* result == 5 */)

There are a couple of structural differences between these two:

With the Categorical operation we’ve been able to feed the input at the end (with run(12)) but with the Monadic operation we had to feed the input at the beginning by Callback.pure.

Second, Monadic bind has the form of:

(Monad x) ≫= (x → Monad y) = Monad y

.bind(x => new Callback(cb => cb(null, x + 1)))

But Categorical pipe has the form of:

Cat (x ↣ y) ⋙ Cat (y ↣ z) = Cat (x ↣ z)

.pipe(new Func(x => x + 1))

It is apparent that Categories abstract constructs with an input and an output. Here Func is an abstraction of a function from x to x + 1.

Functions that return a Monad form a Category

We saw that normal functions (x → y) form a Category which we called Func. The right hand side of monadic bind is a function that takes an x and returns a Monad of y: (x → Monad y). These functions also form an important Category called Kleisli Category:

class Kleisli {

  // given f :: x -> Monad y, constructs a category of type:
  // Cat (x ↣ y)
  constructor(f) {

    // this.run = f
    this.run = x => f(x)

    // this :: Cat (x ↣ y)
    // Cat (y ↣ z) -> Cat (x ↣ z)
    this.pipe = g => new Kleisli(x => this.run(x).then(g.run)) // then == bind

    // utility functon:
    // this :: Cat (x ↣ y)
    // (y -> Monad z) -> Cat (x ↣ z)
    this.pipeTo = g => new Kleisli(x => this.run(x).then(g)) // then == bind
  }
}

// Monad => Cat (x ↣ x)
Kleisli.id = monad => new Kleisli(x => monad.resolve(x))

We might use Kleisli Category for Promise monad like (JSBin):

const times2Plus1 = new Kleisli(x => Promise.resolve(x * 2))
  .pipeTo(x => Promise.resolve(x + 1))

times2Plus1.run(10)
.then(x => console.log(x)) // == 21
.catch(error => console.error(error))

The result of calling times2Plus1.run(10) is a Promise which we consumed by its usual then and catch methods.

pipeTo() is a utility function that pipes a Kleisli (x ↣ y) to a normal function from ( y → Monad z) and produces a new Kleisli (x ↣ z)

Kleisli (x ↣ Monad y) .pipeTo( y → Monad z ) = Kleisli (x ↣ Monad z)

Kleisli (x ↣ Monad y) .pipe( Kleisli (y ↣ Monad z) ) = Kleisli (x ↣ Monad z)

Without pipeTo, we could have manually casted (y → Monad z) to Kleisli (y → Monad z) by new:

const times2Plus1 = new Kleisli(x => Promise.resolve(x * 2))
  .pipe(new Kleisli(x => Promise.resolve(x + 1)))

The beauty of Kleisli Category is that it is defined for any type of Monad. Here is an example for Promise monad: (JSBin).

Another demo showing that the same Kleisli class works for both Promise and Callback monad:

Using Kleisli Category our myLongOperation can be implemented as:

// myLongOperation :: Category (userId ↣ Promise SendMessageResult)

const myLongOperation = new Kleisli(getUser)
  .pipeTo(genMesssage)
  .pipeTo(sendMessage)

myLongOperation.run(123456).then(sendMessageResult => …)

We’re retrieving the final result by calling then(result => ) because the underlying monad in our Kleisli category is the Promise monad, and we assumed getUser, genMessage and sendMessage are functions that return a Promise:

    getUser     = userId => new Promise(resolve => … resolve(user))
    genMessage  = user   => new Promise(resolve => … resolve(msg))
    sendMessage = msg    => new Promise(resolve => … resolve(SendResult))

myLongOperation only deals with Kleisli Category, the type of the underlying monad is irrelevant to it. Try it:

See how we can feed the same myLongOperation function a Kleisli Category of Promise monad or a Kleisli Category of Callback monad.

Our latest implementation of myLongOperation is minimal. This implementation describes what myLongOperation does without any additional noise and it also matches our notation: getUser ⋙ genMessage ⋙ sendMessage.

In conclusion

Composability is the essence of any solution to the callback hell problem.

We implemented the Callback class as an alternative solution and we discovered that our Callback class has actually something in common with Promises. They both provide a then().function that binds them to functions that return a new instance of Promise or Callback. We named these constructs monad.

Monad x ≫= (x → Monad y) = Monad y

(Monad x) .then(x => Monad y) = Monad y

    Callback.pure(10).then(x => new Callback(cb => cb(null, x + 1)))

    Callback.resolve(10).then(x => new Promise(res => res(x + 1))

Monads deal with the result of the operation. Promise.resolve(10).will result in 10 (wrapped in a Promise).

But Categories deal with both input and output of the operation (we denoted them as Cat (x ↣ y)). Func is the simplest category (which corresponds to normal functions).

Categories provide a pipe() function which is akin to Monad.then(). then() receives a function in its argument, but in contrast pipe() takes another instance of Category:

Cat (x ↣ y) ⋙ Cat (y ↣ z) = Cat (x ↣ z)

Cat (x ↣ y) .pipe( Cat (y ↣ z) ) = Cat (x ↣ z)

    Func(x => x + 1).pipe(new Func(x => x * 3)).run(10)

“Functions that return a monad” form a category (which is called Kleisli category).

Using Kleisli category we’ve been able to reduce the noise and redundancy in our async program. Generally in functional programming, instead of dealing with how the program works, our goal is to describe what the program does. Abstractions (like categories or monads) will take care of the details.

Demo Links:

• Callback Monad usage

• Callback Monad Laziness vs Promises

• Callback Monad as solution to the callback hell problem

• Function Composition is not Commutative

• (id = x => x) Function

• Func Category

• Func Category usage for piping functions

• Kleisli Category for Promise Monad

• Kleisli.id function

• Kleisli Category usage demo for both Callback and Promise monad

Whether you liked this post or if I lost you earlier somewhere in the text, you might want to check Mostly adequate guide to FP (in javascript) open source book.

Although we didn’t need to use any library, but for me Ramda is the standard bearer of JavaScript FP libraries.