The last REST API client library that you will ever need

Countless services on the Internet are being exposed over REST API. Most if not all REST API service providers have client libraries for various programming languages to interface with their API.

While all of that is nice, that also means that for every REST API service provider we have to learn a new API interface of that particular client library.

And if that's not the worst, then what if we have to interface with multiple REST API service providers using multiple REST API client libraries in a single code base?

It becomes a mess

The reason why is because we are dealing with client libraries that were never designed to interoperate between each other, even though they are doing roughly the same operations under the hood. The solution to this is to go one layer below and create the client library ourselves.

But how?

Purest is a generic REST API client library for building REST API client libraries. It's a tool for abstracting out REST APIs.

Introduction

Lets take a look at some basic configuration for Google:

{
  "google": {
    "default": {
      "origin": "https://www.googleapis.com",
      "path": "{path}",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  }
}

With it we can instantiate that provider:

var google = purest({provider: 'google', config})

Then we can request some data from YouTube:

var {res, body} = await google
  .get('youtube/v3/channels')
  .qs({forUsername: 'GitHub'})
  .auth(token)
  .request()

The above example demonstrates how a REST API provider can be configured and used in Purest by accessing its default endpoint.

Lets have a look at another example:

{
  "google": {
    "default": {
      "origin": "https://www.googleapis.com",
      "path": "{path}",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    },
    "youtube": {
      "origin": "https://www.googleapis.com",
      "path": "youtube/{version}/{path}",
      "version": "v3",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  }
}

This time around we have an explicit endpoint called youtube for accessing YouTube only:

var {res, body} = await google('youtube')
  .get('channels')
  .qs({forUsername: 'GitHub'})
  .auth(token)
  .request()

The provider configuration is just a convenience for extracting out the request options that we don't want to specify for every request. The auth method is used for replacing the {auth} token found in your configuration, get is the HTTP method being used and its value is the substitute for the {path} token. The qs method is sort of a convention for naming a querystring object that is then being encoded and appended to the request URL.

The above request results in:

GET https://www.googleapis.com/youtube/v3/channels?forUsername=GitHub
authorization: Bearer access_token

What else?

So far we have used Purest like this:

var google = purest({provider: 'google', config})

This allows us to have a configuration and a provider instance for it. Any other dynamic option that is needed for the request have to be passed for every request.

Sometimes, however, we may want to configure certain dynamic values per instance:

var google = purest({provider: 'google', config,
  defaults: {auth: token}
})

Then we no longer need to set the access token for every request:

var {res, body} = await google('youtube')
  .get('channels')
  .qs({forUsername: 'GitHub'})
  .request()

Cool, but what if we want to make our API more expressive?

What if we want to make it our own?

var google = purest({provider: 'google', config,
  defaults: {auth: token},
  methods: {get: ['select'], qs: ['where']}
})

Yes we can:

var {res, body} = await google('youtube')
  .select('channels')
  .where({forUsername: 'GitHub'})
  .request()

Every method in Purest can have multiple user defined aliases for it.

Lastly, accessing an endpoint defined in your configuration can be done by using the explicit endpoint method or its default alias called query:

var {res, body} = await google
  .query('youtube')
  .select('channels')
  .where({forUsername: 'GitHub'})
  .request()

Congratulations!

Now you know the basics.

But the possibilities are endless ...

Lets have a look at another example.

Refresh Token

One very common thing to do when working with REST API providers is to refresh your access token from time to time:

{
  "twitch": {
    "oauth": {
      "origin": "https://api.twitch.tv",
      "path": "kraken/oauth2/{path}"
    }
  }
}

Using the above configuration and the default aliases defined in Purest we can refresh the access token like this:

var {res, body} = await twitch
  .query('oauth')
  .update('token')
  .form({
    grant_type: 'refresh_token',
    client_id: '...',
    client_secret: '...',
    refresh_token: '...'
  })
  .request()

Again query is just an alias for the endpoint method used to access the oauth endpoint in your configuration. The update method is an alias for post and 'token' replaces the {path} in the path configuration. The form method is sort of a convention for naming application/x-www-form-urlencoded request body object that then is being encoded as request body string.

The above request results in:

POST https://api.twitch.tv/kraken/oauth2/token
content-type: application/x-www-form-urlencoded

grant_type=refresh_token&client_id=...&client_secret=...&refresh_token=...

Kinda nice!

Alright, but lets take a look at something more practical:

{
  "twitch": {
    "refresh": {
      "origin": "https://api.twitch.tv",
      "path": "kraken/oauth2/token",
      "method": "POST",
      "form": {
        "grant_type": "refresh_token",
        "refresh_token": "{auth}"
      }
    }
  }
}

Then we can set the application credentials for the entire instance:

var twitch = purest({provider: 'twitch', config, defaults: {
  form: {
    client_id: '...',
    client_secret: '...'
  }
}})

And refresh the access token like this:

var {res, body} = await twitch('refresh')
  .auth('the-refresh-token')
  .request()

Each one of your users will have their own refresh_token, but most likely all of them will be authenticated using a single OAuth application. So it makes sense to configure the provider to use your app credentials by default and only supply the refresh token on every request.

OpenID Connect

OpenID Connect is a popular framework for user authentication and user identity.

One very common theme about it is verifying your JSON Web Token (JWT) that can be either access_token or id_token:

{
  "auth0": {
    "discovery": {
      "origin": "https://{subdomain}.auth0.com",
      "path": ".well-known/openid-configuration"
    }
  }
}

The above configuration is about the discovery endpoint of Auth0 that contains a JSON document outlining certain settings being set for that tenant. The {subdomain} is your tenant name or tenant.region where region applies:

var auth0 = purest({provider: 'auth0', config,
  defaults: {subdomain: tenant}
})

var {body:doc} = await auth0('discovery').request()
var {body:jwk} = await auth0.get(doc.jwks_uri).request()

We request the discovery endpoint and store that document as the doc variable. Then we request the absolute jwks_uri returned in that JSON document and store it as jwk variable. The jwks_uri endpoint returns yet another JSON document containing a list of public keys that can be used to verify a token issued from that tenant:

var jws = require('jws')
var pem = require('jwk-to-pem')

var jwt = jws.decode('id_token or access_token')
var key = jwk.keys.find(({kid}) => kid === jwt.header.kid)

var valid = jws.verify(
  'id_token or access_token', jwt.header.alg, pem(key)
)

We use two additional third-party modules to decode the JSON Web Token, find the corresponding key id (kid), and then verify that token by converting the public key to a PEM format.

OAuth 1.0a

Last but not least

Some providers are still using OAuth 1.0a for authorization. One popular provider that comes to mind is Twitter:

{
  "twitter": {
    "default": {
      "origin": "https://api.twitter.com",
      "path": "{version}/{path}.{type}",
      "version": "1.1",
      "type": "json",
      "oauth": {
        "token": "$auth",
        "token_secret": "$auth"
      }
    }
  }
}

For convenience we set the application credentials for the entire instance:

var twitter = purest({provider: 'twitter', config, defaults: {
  oauth: {
    consumer_key: '...',
    consumer_secret: '...'
  }
}})

And then we pass the user's token and secret with every request:

var {res, body} = await twitter
  .get('users/show')
  .qs({screen_name: 'github'})
  .auth('...', '...')
  .request()

That works, but having to remember all those weird configuration key names every time is hard. Why not put all of them in the default endpoint configuration once and forget about them:

{
  "twitter": {
    "default": {
      "origin": "https://api.twitter.com",
      "path": "{version}/{path}.{type}",
      "version": "1.1",
      "type": "json",
      "oauth": {
        "consumer_key": "{auth}",
        "consumer_secret": "{auth}",
        "token": "{auth}",
        "token_secret": "{auth}"
      }
    }
  }
}

Then all we need to do is pass them as array of strings:

var twitter = purest({provider: 'twitter', config, defaults: {
  auth: ['...', '...', '...', '...']
}})

And focus only on what's important:

var {res, body} = await twitter
  .get('users/show')
  .qs({screen_name: 'github'})
  .request()

Streaming and Multipart

Lets upload some files:

{
  "box": {
    "upload": {
      "method": "POST",
      "url": "https://upload.box.com/api/2.0/files/content",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  },
  "drive": {
    "upload": {
      "method": "POST",
      "url": "https://www.googleapis.com/upload/drive/v3/files",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  },
  "dropbox": {
    "upload": {
      "method": "POST",
      "url": "https://content.dropboxapi.com/2/files/upload",
      "headers": {
        "authorization": "Bearer {auth}",
        "content-type": "application/octet-stream"
      }
    }
  }
}

As usual we have to instantiate our providers:

var box = purest({provider: 'box', config, defaults: {auth: token}})
var drive = purest({provider: 'drive', config, defaults: {auth: token}})
var dropbox = purest({provider: 'dropbox', config, defaults: {auth: token}})

The file upload endpoint for Box expects a multipart/form-data encoded request body:

var {res, body} = await box('upload')
  .multipart({
    attributes: JSON.stringify({
      name: 'cat.png',
      parent: {id: 0},
    }),
    file: fs.createReadStream('cat.png')
  })
  .request()

This is a common way for transferring binary files over the Internet. Every time you submit a Web form that allows you to pick a file from your local file system, the browser is then encoding that data as multipart/form-data, which is what the multipart method does when an object is passed to it.

We are also using the default fs module found in Node.js to stream that cat photo. Imagine that being a really big and fluffy cat that also happens to weight a lot of megabytes.

This is how we upload our cat photos to Google Drive instead:

var {res, body} = await drive('upload')
  .multipart([
    {
      'Content-Type': 'application/json',
      body: JSON.stringify({name: 'cat.png'})
    },
    {
      'Content-Type': 'image/png',
      body: fs.createReadStream('cat.png')
    }
  ])
  .request()

Note that we are still using the multipart method, but this time around we are passing an array instead. In that case the request body will be encoded as multipart/related, which is yet another way to encode multipart request bodies. You can read more about that endpoint here.

Lastly to upload our cat photo to DropBox we stream it as raw request body:

var {res, body} = await dropbox('upload')
  .headers({
    'Dropbox-API-Arg': JSON.stringify({path: '/cat.png'}),
  })
  .body(fs.createReadStream('cat.png'))
  .request()

No additional encoding is expected for the upload endpoint in DropBox.

Great!

But lets do something a bit more dynamic:

{
  "box": {
    "upload": {
      "method": "POST",
      "url": "https://upload.box.com/api/2.0/files/content",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  },
  "dropbox": {
    "download": {
      "url": "https://content.dropboxapi.com/2/files/download",
      "headers": {
        "authorization": "Bearer {auth}"
      }
    }
  }
}

var {res:download} = await dropbox('download')
  .headers({
    'Dropbox-API-Arg': JSON.stringify({path: '/cat.png'}),
  })
  .stream()

await box('upload')
  .multipart({
    attributes: JSON.stringify({
      name: 'cat.png',
      parent: {id: 0},
    }),
    file: {
      body: download,
      options: {name: 'cat.png', type: 'image/png'}
    }
  })
  .request()

We are making the download request using the .stream() method. This instructs Purest to return the raw response stream.

Then we are piping the response stream from DropBox to the request stream for Box by passing it to the multipart file key. This time, however, we need to pass a few additional options because Purest cannot reliably determine the file name and the mime type to embed into the multipart body.

Conclusion

Simple problems need simple solutions

Purest allows us to go one layer below and elegantly compose our own REST API client.

Purest is a tool for creating abstractions without having to create one.

Purest is a primitive for writing HTTP clients.

Happy Coding!