Preface

In 2023, Typescript is rarely questioned as an important tool for modern JavaScript developers, but one of its biggest limitations is the lack of added runtime type safety, particularly when dealing with IO at the boundaries of your application.

To solve this problem a number of popular runtime validation and type-safety tools have popped up, usually competing on runtime parsing speed, API expressiveness, and a TypeScript vs JSON-schema based core.

Compile-time and Editor Performance Pain

What seems to be degrading is performance of the developer experience. TypeScript drives the code intelligence of many popular editors, and as you add type complexity and size to an application or monorepo, code-completion and type-checking starts to draaaaag. This is because driving features like autocomplete means compiling your code on the fly repeatedly, and the more types & files needed to populate autocomplete for a line of code, the longer it takes to get a response.

This is compounded by the growing movement of fully and deeply typed libraries like tRPC, and Tanstack's Query & upcoming Router, which utilise new layers of generics and mapped types over your DTOs, deepening your types, and surfacing compile-time performance issues which weren't as noticeable before.

What's prompted me to look at this is my tRPC+Zod editor experience at work has become unusable. 2-3 seconds to just get autocompletion options on a tRPC path, and then another 2-3 for the next path, repeat. When investigated using TypeScript's tracing tools the data entirely points back to my team's Zod DTOs. What I learned is that Zod's performance is okay at the start, but when you start using methods like .extend/.pick/.omit (and so on) the performance regresses in the order of a magnitude. Rather than making this into a "Zod considered bad" post, I wanted to investigate how the alternatives which can be integrated with tRPC fare, and see whether I can do better.

Analysis Setup

For these benchmarks I'm using a Macbook Pro with M2 Pro and 32GB of RAM.

I've worked on 2 codebases in an Nx monorepo to set up this experiment.

1. Light Type - an experimental runtime type-checker with feature parity in a number of key areas (and an inspired API) to Zod
2. Benchmarks - a tRPC API and React SPA which uses a number of popular and tRPC-compatible type-checkers

The type-checkers under test are:

• Zod 3.20.6
• Yup 1.0.0
• Superstruct 1.0.3
• TypeBox 0.25.21
• Light-Type (Experimental Version)

There may be more libraries which can be integrated with tRPC, but the first 3 are recommended by the project, and TypeBox has had some effort made to integrate it. Light-Type is designed to fit the necessary interfaces of Zod so it's also compatible natively.

The Benchmarks are each made up of a tRPC router with types co-located, and a React Component using tRPC's @tanstack/query abstraction which consumes the router.

TypeScript is pretty good at optimising its compiles. It caches where it can and where there's a single path back to a type it doesn't spread itself wider. So when performance tracing this setup I've seen that compile-times for each router/component don't bleed onto the others, making benchmarking the compiler much easier.

The Actual Types

Each library has two benchmarks, a 'simple' one, and a 'complex' one which adds usage of the equivalent to extend/omit in each library. Except Yup which doesn't have the necessary features to join the complex round.

// simpleZodRouter.ts

import { z } from 'zod'
import { publicProcedure, router } from '../trpc'

const PersonDto = z.object({
  id: z.number(),
  firstName: z.string(),
  lastName: z.string(),
  tel: z.string().optional(),
})

const CarDto = z.object({
  id: z.number(),
  name: z.string(),
  age: z.number(),
  brand: z
    .union([
      z.literal('Volvo'),
      z.literal('Mercedes'),
      z.literal('BMW'),
      z.literal('Ferrari'),
      z.literal('Bazmus'),
    ])
    .optional(),
  previousOwners: z.array(PersonDto).default([]),
})

type DbCar = typeof CarDto['_input']
type Brand = typeof CarDto['_output']['brand']

export const simpleZodRouter = router({
  list: publicProcedure
    .input(z.object({ count: z.number().default(100) }))
    .output(z.array(CarDto))
    .query((opts) => {
      return new Array(opts.input.count).fill(null).map<DbCar>((_, idx) => {
        return {
          id: idx,
          name: 'Foobarmo',
          age: 3,
          brand: 'Bazmus' as Brand,
          previousOwners: [
            {
              id: idx * 100,
              firstName: 'Bob',
              lastName: 'Owneverythingman',
              tel: undefined,
            },
          ],
        }
      })
    }),
  get: publicProcedure
    .input(z.object({ id: z.number() }))
    .output(CarDto)
    .query((opts) => {
      return {
        id: opts.input.id,
        name: 'Foobarmo',
        age: 3,
        brand: 'Bazmus' as Brand,
        previousOwners: [
          {
            id: opts.input.id * 100,
            firstName: 'Bob',
            lastName: 'Owneverythingman',
            tel: undefined,
          },
        ],
      }
    }),
  create: publicProcedure
    .input(CarDto)
    .output(CarDto)
    .mutation((opts) => {
      return opts.input as DbCar
    }),
})

// complexZodRouter.ts

import { z } from 'zod'
import { publicProcedure, router } from '../trpc'

const EntityDto = z.object({
  id: z.number(),
})

const PersonDto = EntityDto.extend({
  firstName: z.string(),
  lastName: z.string(),
  tel: z.string().optional(),
})

const CarDto = EntityDto.extend({
  id: z.number(),
  name: z.string(),
  age: z.number(),
  brand: z
    .union([
      z.literal('Volvo'),
      z.literal('Mercedes'),
      z.literal('BMW'),
      z.literal('Ferrari'),
      z.literal('Bazmus'),
    ])
    .optional(),
  previousOwners: z.array(PersonDto).default([]),
})

type DbCar = typeof CarDto['_input']
type Brand = typeof CarDto['_output']['brand']

export const complexZodRouter = router({
  list: publicProcedure
    .input(z.object({ count: z.number().default(100) }))
    .output(z.array(CarDto))
    .query((opts) => {
      return new Array(opts.input.count).fill(null).map<DbCar>((_, idx) => {
        return {
          id: idx,
          name: 'Foobarmo',
          age: 3,
          brand: 'Bazmus' as Brand,
          previousOwners: [
            {
              id: idx * 100,
              firstName: 'Bob',
              lastName: 'Owneverythingman',
              tel: undefined,
            },
          ],
        }
      })
    }),
  get: publicProcedure
    .input(z.object({ id: z.number() }))
    .output(CarDto)
    .query((opts) => {
      return {
        id: opts.input.id,
        name: 'Foobarmo',
        age: 3,
        brand: 'Bazmus' as Brand,
        previousOwners: [
          {
            id: opts.input.id * 100,
            firstName: 'Bob',
            lastName: 'Owneverythingman',
            tel: undefined,
          },
        ],
      }
    }),
  create: publicProcedure
    .input(CarDto.omit({ id: true }))
    .output(CarDto)
    .mutation((opts) => {
      return opts.input as DbCar
    }),
})

Running the benchmark

# Set up the monorepo
yarn install

# The app/api can served as a test of the integration
yarn light-type-benchmark:start

# Run a traced compile of the repo
yarn tsc:trace
> tsc -p tsconfig.base.json --noEmit --incremental false --jsx react-jsx --allowSyntheticDefaultImports --generateTrace ./.trace

tsc:trace produces ./.trace/trace.json which can be analysed in your browser's dev tools or Perfetto.

This does produce a lot of tracing output which we don't need, but since the benchmarks are isolated in their own projects TypeScript produces reliable results for them. I've edited down the trace file just to the files we're concerned with.

Results

And here are the results:

You can download and explore the trace yourself here: https://gist.github.com/Nick-Lucas/ee964bf004f40ea6601bd2543869efee

While compile-times do vary slightly from run to run, the scale of difference between each library remains the same.

Conclusions

The big outlier here is Zod, which seems to have some structural issues with its types, ballooning in cost by 10+ times after just 2 .extend and 1 .omit call. All the other libraries perform pretty well in the same situation, though Superstruct seems to balloon by about 4x, and TypeBox is slower to begin with but then degrades less poorly. Yup doesn't compete on complex features but performs very well at what it's designed for.

• It's clear that all the type processing for the libraries is happening in the processing of router files where those types are located, and those are quite varied in cost. These results are reproducible over and over again too, it's not a die-roll for the big differences. The react components themselves seem to be isolated and very fast to compile.

• Light Type is consistently the fastest across multiple benchmarks. This is going to be down to a couple factors:

  1. It has fewer features than the other libraries. I would predict it will slow slightly over time as I continue developing it.
  2. Light Type has a lot of focus put into producing the simplest possible types at every stage. Which for example Zod doesn't seem to prioritise.

Notes specifically on Zod: That big block in the flamegraph is Zod's Complex test. Basically don't try to use any features outside of primitives, object, array, or you may immediately gain 10x 🤒 to that type.

• The sum total of all type compilations is your editor experience, and 281ms is already about as high as you probably want your entire library of types to take for autocomplete.
• Those brown tags in the top of the flamegraph are labelled recursiveTypeRelatedTo_DepthLimit, I'm really not an expert on TypeScript's insides, but that sounds like it's trying to tell us something important, and Zod has been guilty of causing "Type instantiation is excessively deep and possibly infinite" errors.
• Zod right now, looks to be a big footgun and in need of rethinking its types, at least when combined with other modern tools like tRPC. If you're starting a new project it might one to avoid for now, though there's really not much alternative if you love Zod's API