Hyper-ts is my Cyborg Brain

EDIT: Hyper-ts has updated since this was written, here is an updated version of the sample server

I have a confession. I have been programming professionally for 17 years, but for all my experience and efforts, I am not a good programmer. I make mistakes constantly. My tests have holes. I break production. I get confused. I forget things that seem trivial until they explode into fireballs of doom. I get pulled into various offices to explain why the team needed to spend the weekend scraping data off the walls with a spatula.

Being a good programmer means interpreting the code you are reading in your head as though you were the computer, and ultimately I am just a bag of meat. An attractive and erudite meat bag perhaps, but still not the sort of thing you want sticking its grubby fingers into your multi-million dollar production system.

This is why I love typed functional programming. Typed FP lets me offload much of the interpretation of code to the compiler, so that it can tell me where my meat brain screwed it up. Extracting side-effects means I can have certainty about what the code will do, and even more importantly what the code won’t do. Enforcing completeness at the compiler level means I can’t forget to handle all the various edge cases that make things go BOOM in the night. Composition means I can take small functions that I know work, slap them together and know that the whole complex thing will work. Then the project manager can get off my back and I can go read my book in peace.

This is why I am so freaking excited about hyper-ts. Hyper-ts takes things a step further - it enforces not just type safety, but protocol correctness at compile time. Holy. Freaking. What.

Hyper-ts keeps us safe from:

• Incorrect ordering of header and body writing
• Writing incomplete responses
• Writing multiple responses
• Trying to consume a non-parsed request body
• Consuming a request body parsed as the wrong type
• Incorrect ordering of, or missing, error handling middleware
• Incorrect ordering of middleware for sessions, authentication, authorization
• Missing authentication and/or authorization checks

This might seem impossible, so it’s time to break it down. Hammertime. Let's talk about State.

In pure FP you can’t just say, hey let’s update this variable. That would be mutation, which is gross, unsafe and terribly uncool - maybe something your Dad who writes VisualBasic would do. Instead, put on your fire aviators and maintain immutability by passing your current state into your functions and have the functions return the new state along with whatever computation they did. It’s easier to understand with an example.

Let’s model a DVD player.

type dvdstate = 'PLAYER_EMPTY' | 'TRAY_OPEN' | 'PLAYER_FULL'

const openTray = () => (state:dvdstate):[undefined, dvdstate] =>
 tuple(undefined, 'TRAY_OPEN')

const loadTray = (dvdTitle:string) => (state:dvdstate):[string, dvdstate] =>
 tuple(
   state === 'TRAY_OPEN' ? `Loaded DVD ${dvdTitle}` : 'EXPLODE',
   state === 'TRAY_OPEN' ? 'PLAYER_FULL' : state)

const startDVD = (dvdTitle:string) => (state:dvdstate) =>
 tuple(`Playing ${dvdTitle}`, state)

const startState = 'PLAYER_EMPTY'
const [_, state] = openTray()(startState)
const [title, state1] = loadTray('Die Hard')(state)
const [movie, state2] = startDVD(title)(state1)
console.log([movie, state2])

This is the most basic way to do state transitions in pure FP. We’ve got a dvdstate, which is what states our DVD player can be in. We’ve got some functions, openTray, loadTray and startDVD that do some state transitions. And finally, you can see here:

const startState = 'PLAYER_EMPTY'
const [_, state] = openTray()(startState)
const [title, state1] = loadTray('Die Hard')(state)
const [movie, state2] = startDVD(title)(state1)

how we thread state through our program. When we run this code we get:

[ 'Playing Loaded DVD Die Hard', 'PLAYER_FULL' ] 

Look Ma, no mutations!

But, this is really a hassle. While I was writing this I messed up which state variable went where like five times. If only there were an easier way.

import * as S from 'fp-ts/lib/State'

const playDisk = (movie:string) => pipe(
 openTray(),
 S.chain(() => loadTray(movie)),
 S.chain((title) => startDVD(title))
)('PLAYER_EMPTY')
console.log(playDisk('Elf'))

Well well well, what have we here. Our old friend the State Monad. We can use the built-in State Monad to thread the state through each function automatically. This is exactly the same as what we did before, but we don’t have to make new variables on every function call. And we get:

[ 'Playing Loaded DVD Elf', 'PLAYER_FULL' ]

However, this is still fragile. Imagine that you were up late the night before reading Bridge to Terabithia and having your heart ripped out. Your eyes are bloodshot and you’re not thinking clearly. You forget to open the dvd player before loading the DVD.

const brokenLikeMyHeart = (movie:string) => pipe(
 loadTray(movie),
 S.chain((title) => startDVD(title))
)
const result = brokenLikeMyHeart('Big Trouble in Little DVD Player')('PLAYER_EMPTY')
console.log(result)

What happens?

[ 'Playing EXPLODE', 'PLAYER_EMPTY' ]

Oof. The player tried to start the DVD and exploded. Maybe you can return it and they won’t notice.

Ok, so we have some kinds of safety, but your meat brain can still screw things up. This is where Indexed Monads come in. Indexed Monads are like State Monads, but with guard rails.

The type of the State Monad is State<A, S>, where A is your result and S is your state. But the type of an Indexed Monad is IxMonad<I, O, E, A>. Vowel soup right? Bear with me, I promise it makes sense.

I stands for Input - the input state. O stands for Output - the output state. E is for Error, and A is any type. Here’s what’s so cool. When you use an IxMonad, you specify - statically! - what the input and output state of your function must be. So we could define an IxMonad such that the brokenLikeMyHeart function would break at compile time because it does not adhere to the specified protocol. Sounds like space magic. Let’s see how hyper-ts uses this wizardry to enforce correctness.

import * as E from 'fp-ts/lib/Either'
import { pipe } from 'fp-ts/lib/pipeable'
import * as t from 'io-ts'
import { failure } from 'io-ts/lib/PathReporter'
import { toRequestHandler } from 'hyper-ts/lib/express'
import express from 'express'
import * as H from 'hyper-ts'
import pgPromise from 'pg-promise'
import * as TE from 'fp-ts/lib/TaskEither'
import { IntFromString } from 'io-ts-types/lib/IntFromString'

const pgp: pgPromise.IMain = pgPromise({})
const db = pgp('postgres://tim:password@localhost:5432/test_db');

const QuestionCodec = t.strict({
 name: t.string,
 text: t.string
})

export type Question = t.TypeOf<typeof QuestionCodec>

function badRequest(message: Error) {
 return pipe(
   H.status(H.Status.BadRequest),
   H.ichain(() => H.closeHeaders()),
   H.ichain(() => H.send(message.message))
 )
}

const paramDecode = pipe(
 H.decodeParam('question_id', IntFromString.decode),
 H.mapLeft((e) => new Error(failure(e).join('\n')))
)

const doAPIWork = (id:number) =>
 H.fromTaskEither(TE.tryCatch(() =>
   db.any<Question>('select * from screening.question where question_id = $1', [id]), E.toError ))

const questionHandler = pipe(
 paramDecode,
 H.ichain((id) => doAPIWork(id)),
 H.ichain((questions:Question[]) =>
   pipe(
     H.status<Error>(H.Status.OK),
     H.ichain(() => H.json(questions, E.toError))
   )
 ),
 H.orElse(badRequest)
)

const app = express()

app
 .get('/questions/:question_id', toRequestHandler(questionHandler))
 .listen(3000, () => console.log('Express listening on port 3000. Use: GET /'))

Here we have a little API that lets us find questions in a database. Let's break it down.

questionHandler is where we do the work of handling our request. First thing, we decode the parameter like so:

 H.decodeParam('question_id', IntFromString.decode),

If we’re unable to do the decode, like if someone passed in the string ‘i am not a number’ instead of a number, we generate a human readable error string and wrap it in an error like this:

H.mapLeft((e) => new Error(failure(e).join('\n')))

Next up, we do our actual API work. We hit the database and grab the data. We know for sure that id is a number so we can do:

H.fromTaskEither(TE.tryCatch(() =>
   db.any<Question>('select * from screening.question where question_id = $1', [id]), E.toError ))

Nice and safe with our error cases covered (we could decode the db response with io-ts but...there is such a thing as being too paranoid you know?) So far so good, but the mind blowing magic hasn’t happened quite yet.

H.ichain((questions:Question[]) =>
   pipe(
     H.status<Error>(H.Status.OK),
     H.ichain(() => H.json(questions, E.toError))
   )
 ),

Oh yeah, that’s the stuff. The state transitions defined in hyper-ts are:
StatusOpen -> HeadersOpen -> BodyOpen -> ResponseEnded

These states correspond to the parts of the HTTP response that you are writing:

HTTP/1.1 200 OK <- **STATUS**
Content-Type: text/plain <- **HEADERS**
Transfer-Encoding: chunked 
7\r\n <- **BODY (streaming)**
Mozilla\r\n 
9\r\n 
Developer\r\n 
7\r\n 
Network\r\n 
0\r\n 
\r\n <- **END BODY**

In this case, hyper-ts is enforcing (at compile time) that your code must have written a status, header and body - and in that order.

ichain is just like the chain method from the State example - state is automatically threaded through in the same way. Here's the cool part that's different: if you look at the type of the first line, H.ichain((questions:Question[]), you'll see:

H.Middleware<H.StatusOpen, H.ResponseEnded, Error, void>

The input state is H.StatusOpen. This tells us that when this function is called, we have not written our status line yet. The output state is H.ResponseEnded. That tells us that when this function returns we will have completed writing our entire response. In the case of an error we’ll get an Error, and if everything goes right we’ll get a void. Already we’re in better shape - we know that going into this function we can’t have written anything, and coming out we must have closed the request. Next, let’s look at:

H.status<Error>

the type is:

H.Middleware<H.StatusOpen, H.HeadersOpen, Error, void>

The input state is H.StatusOpen and the output state is H.HeadersOpen, which means that by the end of this function we must have written our status to the response. The next line:

H.ichain(() => H.json(questions, E.toError))

has type:

H.Middleware<H.HeadersOpen, H.ResponseEnded, Error, void>

This combines the header and body writing for convenience, but we must have written the headers and finished writing the body by the time this function returns. And, if you look at the implementation of H.json:

pipe(
    contentType<E>(MediaType.applicationJSON),
    ichain(() => closeHeaders()),
    ichain(() => send(json))
)

That is exactly what happens. Ok, let’s break it.

 H.ichain((questions:Question[]) =>
   pipe(
     H.ichain(() => H.json(questions, E.toError))
   )
 ),

We didn't write a status and what happens? BOOM compiler error!

Type '(ma: Middleware<unknown, HeadersOpen, Error, unknown>) => Middleware<unknown, ResponseEnded, Error, void>' is not assignable to type 'Middleware<StatusOpen, unknown, Error, unknown>'.

H.json has input type H.HeadersOpen, and we are passing H.StatusOpen, which isn’t going to fly.

From now on, I’ll never forget to send a status. I’ll never forget to write a body. I’ll never get my protocol out of order - because the compiler does that thinking for me. Welcome to my new cyborg brain.