Becoming a Fastify JSON Schema GURU

by Manuel Spigolon

Fastify v4.3.0 has landed with new features! In detail, there are new functions available to the request and reply objects that allow you to easily work with JSON Schema.

Let's see what has changed!

The issue

Many developers have complained about the fact that Fastify uses ajv and fast-json-stringifyunder the hood to provide validation and serialization of JSON data, but those instances have never been exposed. In this way, it is impossible for developers to use those instances to process JSON schemas on their routes' handlers.

Because of this, devs were forced to build their own JSON schema validator and serializer, which is a huge pain.

Why does Fastify not expose the instances?

Fastify has an abstraction layer that allows it to be agnostic about the JSON schema validator and serializer. This abstraction layer is called:

• Validator Compiler
• Serializer Compiler

On top of those compilers, there is another component, the Schema Controllerthat manages when the validator and serializer are created and how they should be initialized.

This layer is structured to let you customise everything in your Fastify application and to keep the performance and the memory footprint as low as possible.

So, why Fastify does not expose the instances? Because nobody did it before, till now!

The solution

One day, @metcoder95, a member of the Fastify community, decided to open a PR to expose the validator and serializator instances to all the developers!

The awesome result was that the following code is now possible:

• read the compiled functions from the request and reply objects
• compile new functions at runtime
• validate and serialize JSON data by using a new JSON schema at runtime

So, let's see how it works!

Read the compiled functions

Let's jump into the code to understand how this feature works.

First, we need some JSON schema that we will use across the following code examples:

const bodySchema = {
  type: 'object',
  properties: {
    name: { type: 'string', minLength: 2 }
  },
  required: ['name']
}

const responseSchema = {
  type: 'object',
  properties: {
    user: {
      type: 'object',
      properties: {
        id: { type: 'integer', minimum: 1 }
      }
    }
  }
}

Now, let's use them by:

1. creating a route that uses the schemas
2. reading the compiled functions from the request and reply objects

// 1. create a route that uses the schemas
fastify.post('/:userId', {
  schema: {
    body: bodySchema,
    response: {
      200: responseSchema
    }
  }
},
async function handler (request, reply) {
  // 2. read the compiled body function from the `request` object
  const bodyValidationFunction = request.getValidationFunction('body')
  const validationResult = bodyValidationFunction({ name: 'John' })
  console.log(validationResult)

  // 3. read the compiled serialization function from the `reply` object
  const responseSerializationFunction = reply.getSerializationFunction(200)
  const serializedResponse = responseSerializationFunction({ user: { id: 1 } })

  return serializedResponse
})

As you can see, now you can retrieve the compiled functions generated by the route's schema configuration!

Compile new functions at runtime

Another feature that has been added to Fastify is the ability to compile new functions at runtime.

Let's see how it works:

fastify.post('/foo',
  async function handler (request, reply) {
    const bodyValidationFunction = request.compileValidationSchema(bodySchema)
    const validationResult = bodyValidationFunction({ name: 'John' })
    console.log(validationResult)

    const responseSerializationFunction = reply.compileSerializationSchema(responseSchema)
    const serializedResponse = responseSerializationFunction({ user: { id: 1 } })

    return serializedResponse
  })

In this new example, the route's schema configuration is not used anymore, but the compiled functions are generated at runtime.

This will let you to generate dynamic schemas during the handler execution!

Security Notice

Treat the schema definition as application code. Validation and serialization features dynamically evaluate code with new Function(), which is not safe to use with user-provided schemas.

Validate and serialize JSON data

The last new feature is the ability to validate and serialize JSON data. This one is just a shortcut to writing even less code:

fastify.post('/light-foo',
  async function handler (request, reply) {
    const validationResult = request.validateInput({ name: 'John' }, bodySchema)
    console.log(validationResult)

    const serializedResponse = reply.serializeInput({ user: { id: 1 } }, responseSchema)
    return serializedResponse
  })

Performance

Fastify is always thinking about the performance of your application. The new features we just read above have this mindset too!

• request.getValidationFunction
• request.compileValidationSchema
• request.validateInput
• reply.getSerializationFunction
• reply.compileSerializationSchema
• reply.serializeInput

All the new functions implement a WeakMap cache to avoid recompiling the functions every time. Compiling a new function is a very expensive operation, so it may impact your application performance.

To avoid this, Fastify caches the compiled functions in a WeakMap so every time you call the above functions with the same JSON schema, it will use the cached version.

There is a fundamental thing you must do to benefit from this cache: reuse the same JSON schema objects. Let's see how to do it!

The WeakMap cache

To understand how the cache works, let's take a look at the code:

const cache = new WeakMap()

const schema = { id: 'foo' }
cache.set(schema, function () { console.log('foo') })

console.log(cache.has({ id: 'foo' })) // prints false
console.log(cache.has(schema)) // prints true

As you can understand, the WeakMap cache is a map that stores the JSON as key and the compiled function as function.

If the key is not the same object reference, the cache will not be able to find the compiled function.

That said, to benefit from the cache, you must use the same JSON schema object:

const aSchema = {
  type: 'object',
  properties: {
    name: { type: 'string', minLength: 2 }
  }
}
fastify.get('/',
  async function handler (request, reply) {
    // we call the compileValidationSchema with the same schema object
    const validationFn = request.compileValidationSchema(aSchema)
    const validationResult = validationFn({ name: 'John' })
    console.log(validationResult)
    return validationResult
  })

Instead, by writing a new schema object, the cache will not be able to find the compiled function:

fastify.get('/',
  async function handler (request, reply) {
    // we call the compileValidationSchema with a new JSON object every handler call
    // so the cache will not be able to find the compiled function
    const validationFn = request.compileValidationSchema({
      type: 'object',
      properties: {
        name: { type: 'string', minLength: 2 }
      }
    })
    // ...
  })

Summary

Thanks to the community contribution, Fastify is now able to compile new functions at runtime and let you to validate and serialize JSON data in your handler without the burden of creating your own ajv and fast-json-stringify instances.

You should now be able to use the new features to generate dynamic schemas and validate and serialize JSON data.

If you have any questions or suggestions, please join the Fastify discord and feel free to drop a message.

Acknowledgements

Thank you very much to @metcoder95 for the contribution.