Recreating Vue 3 Reactivity API (roughly)

This article is roughly based off the talk I gave on 20th November 2019 at Vue.js London #13 meetup. You can find the video of the talk here and the repo here.

Typescript will be used in this article so we can look at the problem in a slightly different perspective. If you hate typescript, you can watch my talk instead which was in Javascript.

Introduction to Vue 3 Reactivity API

You can read about the Official Vue 3 Reactivity API. But here is a brief introduction with examples.

There are 4 functions in the reactivity API:

1. reactive()
2. ref()
3. computed()
4. watch()

Consider example 1:

import { reactive, watch } from '@vue/runtime-core'

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const car = reactive({
  position: 0,
  speed: 2,
})

setInterval(() => {
  car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
}, 1000)

watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]
  road[car.position] = '🚗'

  console.clear()
  console.log(road.reverse().join(''))
})

This code uses reactive() and watch() from the reactivity API. reactive() create an reactive object, i.e. the retrieval and setting of any properties will be tracked. watch() takes in a callback that will be executed immediately; whenever the callback's dependencies are changed, the callback will be evaluated again.

So in this example, car.position is updated every 1000ms. And we will see the car moving from the right to the left.

Consider example 2

import { ref, computed, watch } from '@vue/runtime-core'

const counter = ref(0)

const squared = computed(() => counter.value ** 2)

setInterval(() =>  counter.value += 1, 1000)

watch(() => {
  console.log(`counter: ${counter.value}`, `counter²: ${squared.value}`)
})

This code uses ref(), computed() and watch(). ref() and computed() both returns a Ref. A Ref is simply defined as:

interface Ref<T> {
  value: T
}

From the example, ref(0) returns { value: 0 } where the value will be reactive. computed() takes in a function returns a Ref whose value is whatever the function returns.

Hopefully this quick introduction by examples makes sense. If you are in doubt, make sure you read the official description of the Vue 3 Reactivity API before reading the rest of the article.

Quick introduction to ES6 Proxy

Proxy is an ES6 feature; it is the real magic behind Vue 3's reactivity. You can see the full documentation here.

In this introduction, I am just going to include the parts we need from proxy to create reactivity.

Proxy is an object which allow us to programmatically control how it behaves on native operations.

Consider example 3

const target: Record<any, any> = {}

const p = new Proxy(target, {
  set(setTarget, key: string | number, value) {
    console.log(`=== start p.${key} = ${value} ===`)

    console.log(`setTarget === target -> ${setTarget === target}`)
    console.log({ key, value })

    setTarget[key] = value
    console.log(`=== end p.${key} = ${value} ===`)
    return true
  },

  get(setTarget, key: string | number) {
    console.log(`=== start getting p.${key}} ===`)

    console.log(`getting setTarget[${key}]: ${setTarget[key]}`)

    console.log(`=== end getting p.${key}} ===`)
    return 'nope'
  }
})

p.x = 3 // will trigger set trap
console.log()

target.x = 5 // will not trigger trap

console.log(`p.x: ${p.x}`) // will trigger get trap
console.log()

console.log(`p.y: ${p.y}`) // will trigger get trap
console.log()

console.log(target)

Here is the output:

=== start p.x = 3 ===
setTarget === target -> true
{ key: 'x', value: 3 }
=== end p.x = 3 ===

=== start getting p.x} ===
getting setTarget[x]: 5
=== end getting p.x} ===
p.x: nope

=== start getting p.y} ===
getting setTarget[y]: undefined
=== end getting p.y} ===
p.y: nope

{ x: 5 }

Please note that the reason for key: string | number is because Typescript currently cannot handle symbols as keys in objects. This is so stupid and there is a 5-year-old issue created regarding this. key will be typed as string | number | symbol otherwise.

As you can see in the example, we have set up the set and get trap for the proxy p. Whenever p's property is set or retrieved, our traps will be called and we can change how it behaves.

In this example, we always return 'nope' in the get function. This is why we see 'nope' for both p.x and p.y.

If you are still unsure about how Proxy works, make sure you read more into it in the mdn documentation.

Let's recreate Vue 3's reactivity API

You should be familiar with Vue 3's reactivity API and Proxy by now. Let's now try to recreate Vue 3's reactivity API.

reactive() and watch()

Let's recall example 1:

import { reactive, watch } from '@vue/runtime-core'

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const car = reactive({
  position: 0,
  speed: 2,
})

setInterval(() => {
  car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
}, 1000)

watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]
  road[car.position] = '🚗'

  console.clear()
  console.log(road.reverse().join(''))
})

Our aim in this section is to make example 1 work with our customreactive() and watch().

Brute-force "reactivity"

We can quickly make example 1 work as expected by simply calling the watchers (watch() callbacks) whenever a reactive property is set. Let's implement this first and see where we can depart from there.

First, let's keep track of the watchers in watch().

const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback() // this is what Vue 3 watch() will do
  watchers.push(callback)
}

Pretty straightforward. Now we have a list of watchers. Next we have to trigger them whenever a reactive property is changed.

We can achieve this by having reactive() to return a proxy whose set trap will trigger all watchers.

const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback() // this is what Vue 3 watch() will do
  watchers.push(callback)
}

const reactive = <T extends object>(t: T): T => {
  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      watchers.forEach(watcher => watcher())

      return true
    },

    get(target, key: keyof T) {
      return target[key]
    },
  })
}

Two things to note about the types:

1. Please note that the reason for key: keyof T is because Typescript would require key to be a key of T before being able to do target[key] = value. Without : keyof T, key will be typed as stirng | number | symbol which introduces another problem with the 5-year-old issue mentioned earlier.
2. Previously string | number was sufficient because the target was a Record<any, any>, so typescript knows that the target can be extended.

An example to illustrate how the type works.

const r = reactive({ a: 3 }) // r is of type `{ a: number }`
r.a = 5
r.b // this will throw a type error as `b` is not a key of `{ a: number }`

Exporting our watch() and reactive(), we can combine them with example 1:

Example 4:

And the car is moving! ✅

There are couple of problems with this approach:

1. Watchers will be called N times if we trigger mutate reactive object N times

Watchers should only be fired once after a series of consecutive mutation. Currently each mutation will trigger the watchers immediately.

1. Watchers will be called even when it doesn't need to

Watchers should only be reevaluated whenever their dependencies changes. We currently do not care and call the watchers whenever somethings is mutated.

Brute-force reactivity (fixing problem 1)

We aim to solve the first problem in the last section.

To illustrate the problem, I have modified the code to add one more car which will trigger another mutation in the interval. You can see the code in example 5.

import { reactive, watch } from './reactivity';

// % in Javascript is remainder operator, e.g. -1 % 5 gives -1.
// The following is an implementation for modulus.
const mod = (x: number, y: number) => ((x % y) + y) % y

const MAX_ROAD_LENGTH = 10

const cars = [
  reactive({
    position: 0,
    speed: 2,
  }),
  reactive({
    position: 2,
    speed: 1,
  }),
]

setInterval(() => {
  for (const car of cars) {
    car.position = mod(car.position + car.speed, MAX_ROAD_LENGTH)
  }
}, 1000)

let callCount = 0;
watch(() => {
  const road = [...'_'.repeat(MAX_ROAD_LENGTH)]

  for (const car of cars) {
    road[car.position] = '🚗'
  }

  console.clear()
  console.log(road.reverse().join(''))
  console.log(`callCount: ${++callCount}`)
})

You can see how the callCount increments by 2. This is because there are two mutations happening every 1000ms so the watcher was called twice every 1000ms.

Our aim is to have the watchers only called once after a series of consecutive mutations.

How do we achieve this? "Firing something only once after a series of invocation"? Does this sound familiar? We actually have probably encountered this already in many places. For example, showing search suggestions only after user has stopped typing for a while; firing scroll listener once only after the user has stopped scrolling for a while?

Debounce! Yes, we can just debounce the watchers. This will allow a series of mutation finish before triggering the watcher. And it will only do it once! Perfect for this use case!

I will just use lodash's debounce here so we won't need to implement it.

See example 6:

import debounce from 'lodash.debounce'

const watchers: (() => any)[] = []
const watch = (callback: () => any) => {
  callback()
  watchers.push(debounce(callback, 0)) // debouncing callback
}

const reactive = <T extends object>(t: T): T => {
  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      watchers.forEach(watcher => watcher())

      return true
    },

    get(target, key: keyof T) {
      return target[key]
    },
  })
}

You can see how the callCount only increment by 1 every 1000ms.

Dependency tracking

The second problem: "watchers will be called even when it doesn’t need to", can be solved with dependency tracking. We need to know what a watcher depend on and only invoke the watcher when those dependencies are mutated.

In order to illustrate the problem, I have modified the index.ts.

import { reactive, watch } from './reactivity';

const r1 = reactive({ x: 1 })
const r2 = reactive({ x: 100 })

setInterval(() => {
  r1.x++
}, 1000)

setInterval(() => {
  r2.x--
}, 5000)

watch(() => {
  console.log(`r1.x: ${r1.x}`)
})

watch(() => {
  console.log(`r2.x: ${r2.x}`)
})

With this example, we can see the problem clearly. We expect r1.x to be logged every second and r2.x every 5 seconds. But both values are logged every second because all watchers are called.

Here are the steps we can implement dependencies tracking:

1. We can keep track of the dependencies of a watcher in a Set, which helps avoid duplications. A dependency is a property in a reactive. We can represent each property in a reactive with a unique identifier. It could be anything unique but I'll use a Symbol() here.
2. Clear the dependencies set before calling the watcher.
3. When a reactive property is retrieved, add the symbol representing that property to the dependencies set.
4. After the watcher callback finishes, dependencies will be populated with symbols that it depends on. Since each watcher now relates to a set of dependencies, we will keep { callback, dependencies} in the watchers list.
5. Instead of triggering all watchers as a property is being set, we could trigger only the watchers that depend on that property.

import debounce from 'lodash.debounce'

const dependencies = new Set<symbol>() // 1. keep track of dependencies

const watchers: ({
  callback: () => any,
  dependencies: Set<symbol>,
})[] = []
const watch = (callback: () => any) => {
  dependencies.clear() // 2. clear dependencies 
  callback()
  // 4. dependencies is populated
  watchers.push({
    callback: debounce(callback, 0),
    dependencies: new Set(dependencies), // make a copy
  })
}

const reactive = <T extends object>(t: T): T => {
  const keyToSymbolMap = new Map<keyof T, symbol>()
  const getSymbolForKey = (key: keyof T): symbol => {
    const symbol = keyToSymbolMap.get(key) || Symbol()
    if (!keyToSymbolMap.has(key)) {
      keyToSymbolMap.set(key, symbol)
    }
    return symbol
  }

  return new Proxy(t, {
    set(target, key: keyof T, value) {
      target[key] = value 

      // 5. only trigger watchers depending on this property
      watchers
        .filter(({ dependencies }) => dependencies.has(getSymbolForKey(key)))
        .forEach(({ callback }) => callback())

      return true
    },

    get(target, key: keyof T) {
      dependencies.add(getSymbolForKey(key)) // 3. add symbol to dependencies
      return target[key]
    },
  })
}

With this we can see the result matches our expectation and this means dependency tracking is working!!!

Update dependencies on the fly

A watcher may change its dependencies. Consider the following code:

const r1 = reactive({ isReady: false })
const r2 = reactive({ x: 1 })

setTimeout(() => {
  r1.isReady = true
}, 1000)

setInterval(() => {
  r2.x++
}, 500)

watch(() => {
  if (!r1.isReady) return
  console.log(`r2.x: ${r2.x}`)
})

In this example, we expect the log to happen after 1 second and every 500ms afterwards.

However our previous implementation only logs once:

This is because our watcher only access r1.x at its first call. So our dependency tracking only keeps track of r1.x.

To fix this, we can update the dependencies set every time the watcher is called.

const watch = (callback: () => any) => {
  const watcher = {
    callback: debounce(() => {
      dependencies.clear()
      callback()
      watcher.dependencies = new Set(dependencies)
    }, 0),
    dependencies: new Set<symbol>(),
  }

  watcher.callback()

  watchers.push(watcher)
}

This wraps the dependency tracking into the watcher to ensure the dependencies is always up to date.

With this change, it is now fully working! 🎉

ref(), computed()

We can build ref() and computed() pretty easily by composing reactive() and watch().

We can introduce the type Ref as defined above:

interface Ref<T> {
  value: T
}

Then ref() simply returns a reactive with just .value.

const ref = <T>(value: T): Ref<T> => {
  return reactive({ value })
}

And a computed() just return a ref which includes a watcher that updates the value of the ref.

const computed = <T>(fn: () => T): Ref<T> => {
  const r = ref<T>(undefined as any)

  watch(() => {
    r.value = fn()
  })

  return r
}

See the following example:

Conclusion

Thanks for reading this tedious article and hope you have gain some insights about how the magic behind Vue's reactivity works. This article has been worked on across months because I travelled to Japan in the middle of writing this. So please let me know if you spot any mistakes/inconsistency which can improve this article.

The reactivity we have built is just a really rough naive implementation and there are so many more considerations put into the actual Vue 3 reactivity. For example, handling Array, Set, Map; handling immutability etc. So please do not use these code in production.

Lastly, hopefully we will see Vue 3 soon and we can make use of this amazing api to build awesome things! Happy coding!