Finding the right abstraction

This series is about sharing some of the challenges and lessons I learned during the development of Prism and how some functional concepts lead to a better product.

Note: As of January 2021, I no longer work at Stoplight and I have no control over the current status of the code. There is a fork on my GitHub account that represents the state of the project when I left the company.

In the previous post we had a 30k feet view of Prism and some of its technical features and challenges.

In this specific post, I will explain the series of events that ended up me deciding to give a try to a functional library that apparently would fit the Prism's use case perfectly.

A validator in TypeScript

Prism was not the first project that I was working on when I joined Stoplight; before landing there I have been bounced around different projects while trying to understand where I could best contribute.

During that time, the development of the new version of Prism was just getting started by a different team, and I was totally unaware of the project's status, direction and technical choices that were made.

However, we can say that Prism was developed as a typical TypeScript web application; throwing exception where things were going unexpectedly wrong (somewhere returning null or undefined), using classes holding state for the validators, a strong classical inheritance model to let the software expand in the future and so on.

Months pass. I am working on Stoplight Studio and I was actively looking for a validation library that would be TypeScript friendly and would be able to type the returned value in case the validation would pass.

I googled the relative keyword around ending up in solutions using decorators that were not what I was looking for.

I do not like decorators. They're not even standards. I do not like classes neither.

During the search though, I stumbled on this Tweet that gave me some hope:

Julien Richard-Foy

@julienrf

TypeScript is fine as a language but it seems that the whole ecosystem is not used to designing libraries taking advantage of types to improve productivity. I saw mostly types around JS libs. I believe that types can deliver more value! An interesting example is the io-ts lib.

12:40 PM - 02 May 2019

I headed to the mentioned project and after a quick read to the README (which, I would agree with you, is not that…eloquent) I felt it looked promising and so I tried create a sample project to try it out.

First impressions

Note: this image is relative to io-ts 1.x. The current version is 2.x. Your experience is likely going to be different.

There are two things that are very evident:

1. There's a bunch of methods whose name aren't familiar at all:
   • inspect
   • isLeft
   • isRight
   • map (well actually this was not unknown)
   • mapLeft
   • orElse
   • reduce
   • …the list goes on and on
2. I can't find my data. Once I've defined a type and decoded an unknown payload, where is my data stored? I could only see functions.

I was already on the point of ditching the whole thing and look for the next alternative, but I then noticed I actually knew the author (we've been hanging out in the same Slack channel for a while, but mostly ignoring each other), so I decided to ask what was the deal with io-ts.

After some chit chats, it turned out that io-ts is using fp-ts, providing the functional primitives that the library is using and exposing, and that was the primary reason for the "weird" set of functions returned from the decode process.

Ultimately we ended up philosophically discussing what kind of software I have been writing all these years by ignoring the functional concepts.

At the end of the day, I learned that the decode function of io-ts would return an Either — a monad representing the effect that a computation might fail. So, in order to continue my journey, I had to master that.

It's not my intention here to explain to you functional programming. I am not qualified for that and if you're interested I really suggest you to read the amazing series of articles written by Giulio Canti directly:

Giulio CantiFollow

/gcanti

However, I just wanted to give a basic explanation of the Either monad and how I felt it would be just what I'd need for Prism

Either 101

An Either is a tagged union that holds a value whose type depends on the tag property.

type Either<E, A> = Left<E> | Right<A>

interface Left<E> {
  readonly _tag: 'Left'
  readonly left: E
}

interface Right<A> {
  readonly _tag: 'Right'
  readonly right: A
}

What you want to store in such a structure it's ultimately up to you, but what's mostly used for in all the code I saw so far in the wild is essentially to hold errors (usually put on the Left) and the value (usually on the Right) in as an alternative way to throw an exception when something is going wrong.

For instance, consider this function:

function createParamUriTemplate(name: string, style: HttpParamStyles, explode: boolean) {
  const starOrVoid = explode ? '*' : '';
  switch (style) {
    case HttpParamStyles.Simple:
      return `{${name}${starOrVoid}}`;

    case HttpParamStyles.Label:
      return `{.${name}${starOrVoid}}`;

    case HttpParamStyles.Matrix:
      return `{;${name}${starOrVoid}}`;

    default:
      throw new Error(`Unsupported parameter style: ${style}`);
  }
}

and now let's rewrite it to use the Either tagged union:

import { left, right } from ‘fp-ts/lib/Either’;

function createParamUriTemplate(name: string, style: HttpParamStyles, explode: boolean) {
   switch (style) {
     case HttpParamStyles.Simple:
+      return right(`{${name}${starOrVoid}}`);
-      return `{${name}${starOrVoid}}`;

     case HttpParamStyles.Label:
+      return right(`{.${name}${starOrVoid}}`);
-      return `{.${name}${starOrVoid}}`;

     case HttpParamStyles.Matrix:
+      return right(`{;${name}${starOrVoid}}`);
-      return `{;${name}${starOrVoid}}`;

     default:
+      return left(new Error(`Unsupported parameter style: ${style}`));
-      throw new Error(`Unsupported parameter style: ${style}`);
   }
 }

Such changes inevitably will mandate a change in the function signature:

import * as E from 'fp-ts/lib/Either';

 declare function createParamUriTemplate(
   name: string,
   style: HttpParamStyles,
   explode: boolean
- ): string;
+ ): E.Either<Error, string>;

And it also made me observe something:

• While the first signature does not communicate in any way that the function could error and throw an exception (the only way to get that is to inspect the source code), the second version is very clear about the possibility of a failure.
• While in the first case I can pretend (and pray) that no error will happen and carry on the happy path only, the second function will not let you access the data if you don't perform explicit checks
• It makes the Error an explicit part of your domain model instead of an exceptional thing that should never happen (that WILL happen one day, believe it or not)
• Since Either admits instances for Monad, Applicative and a Functor, it can leverage some good tooling to restore composability even if processes that might error (more about that later)

Obviously, these points didn't come in my mind all at once; it required some effort and education to get past the "meh, does not look so good" phase and understand the concepts and the benefits it would bring (and make the coworkers excited about it, but we'll discuss this later).

I was still super skeptical about it, but the bell was ringing. At that time, I had the two considerations in mind:

• If Prism's job is to return errors then treating them as a domain entities instead of a undesired effect that should never happen (aka: exceptions) would make a lot of sense
• By having to handle all the errors no matter what, I could make sure that I would always return the correct status code and not leave anything unhandled (and returning an incorrect 500).

It was worth giving it a try; but how do you introduce such stuff in a codebase that's already in progress, with other people working on it and that's close to a major release?

We'll explore that in the next article.