Extending JSON for fun and profit

header image by Steel's Fudge

In the early days of JavaScript when asynchronous requests first enabled web authors to make requests to HTTP servers and receive a readable response, everyone was using XML as the standard for data exchange. The problem with that was usually parsing; you'd have to have a beefy parser and serializer to safely communicate with a server.

That changed as Douglas Crockford introduced JSON as a static subset of the JavaScript language that only allowed strings, numbers and arrays as values, and objects were reduced to just key and value collections. This made the format robust while providing safety, since unlike JSONP, it would not allow you to define any executable code.

Web authors loved it [citation needed], API developers embraced it, and soon, standardization brought the JSON API into the fold of web standards.

Parsing JSON

The parse method takes just two arguments: the string representing a JSON value, and an optional reviver function.

With parsing, you may only have used the first argument to parse a function, which works just fine:

But just what does that reviver argument do, exactly?

Per MDN, the reviver is a function that will be passed every key and value during parsing and is expected to return a replacement value for that key. This gives you the opportunity to replace any value with anything else, like an instance of an object.

Let's create an example to illustrate this point. Say you have a fleet of drones that you'd like to connect to, and the API responds with an array of configuration objects for each drone. Let's start by looking at the Drone class:

For simplicity, all the class does is provide the name property. The symbols defined are there to hide the private members from public consumers. Let's see if we can make a factory function that will convert the configurations into actual objects.

Our imaginary API server responds with the following JSON object:

[
  { "$type": "Drone", "args": ["George Droney", { "id": "1" } ] },
  { "$type": "Drone", "args": ["Kleintank", { "id": "2" } ] }
]

We want to turn each entry that has a $type property into an instance by passing the arguments to the constructor of the appropriate object type. We want the result to be equal to:

const drones = [
  new Drone('George Droney', { id: '1' }),
  new Drone('Kleintank', { id: '2' })
]

So let's write a reviver that will look for values that contain the $type property equal to "Drone" and return the object instance instead.

The nice thing about the reviver function is that it will be invoked for every key in the JSON object while parsing, no matter how deep the value. This allows the same reviver to run on different shapes of incoming JSON data, without having to code for a specific object shape.

Serializing into JSON

At times, you may have values that cannot be directly represented in JSON, but you need to convert them to a value that is compatible with it.

Let's say that we have a Set that we would like to use in our JSON data. By default, Set cannot be serialized to JSON, since it stores object references, not just strings and numbers. But if we have a Set of serializable values (like string IDs), then we can write something that will be encodable in JSON.

For this example, let's assume we have a User object that contains a property memberOfAccounts, which is a Set of string IDs of accounts it has access to. One way we can encode this in JSON is just to use an array.

const user = {
  id: '1',
  memberOfAccounts: new Set(['a', 'b', 'c'])
};

We'll do this by using the second argument in the JSON API called stringify. We pass the replacer function

In this way, if we want to parse this back into its original state, we can apply the reverse as well.

Completing the cycle

But before we verify that the reverse mapping works, let's extend our approach so that the $type can be dynamic, and our reviver will check to the global namespace to see if the name exists.

We need to write a function that will be able to take a name of a class and return that class' constructor so that we can execute it. Since there is no way to inspect the current scope and enumerate values, this function will need to have its classes passed to into it:

const createClassLookup = (scope = new Map()) => (name) =>
  scope.get(name) || (global || window)[name];

This function looks in the given scope for the name, then falls back onto the global namespace to try to resolve built-in classes like Set, Map, etc.

Let's create the class lookup by defining Drone to be in the scope for resolution:

const classes = new Map([
  ['Drone', Drone]
]);

const getClass = createClassLookup(classes);

// we can call getClass() to resolve to a constructor now
getClass('Drone');

OK, so let's put this all together and see how this works out:

Et voilá! We've successfully parsed and revived the objects back into the correct instances! Let's see if we can make the dynamic class resolver work with a more complicated example:

const jsonData = `[
  {
    "id": "1",
    "memberOf": { "$type": "Set", "args": [["a"]] },
    "drone": { "$type": "Drone", "args": ["George Droney", { "id": "1" }] }
  }
]`;

Ready, set, parse!

If you drill down into the object structure, you'll notice that the memberOf and drone properties on the object are actual instances of Set and Drone!

Wrapping up

I hope the examples above give you a better insight into the parsing and serializing pipeline built into the JSON API. Whenever you are dealing with data structures for incoming data objects that need to be hydrated into class instances (or back again), this provides a way to map them both ways without having to write your own recursive or bespoke functions to deal with the translation.

Happy coding!