Note: This was originally a script for a video. As it takes months at times to release a video, I've decided to turn what I have into an article as well.
I'm a fairly big proponent of the idea that certain programming topics are best taught by discussing the low level fundamentals rather than the high level API. In the same way that Dan Abramov teaches Redux or Francis Stokes teaches just about anything.
In this article we're going to discuss a fairly hip JavaScript topic: Reactive Data Structures. Let's first get into a use case.
The Problem
For the most basic of use cases, let's try to share data between modules. Perhaps we're creating a game and we want our score to be able to be changed via multiple different modules containing their own functions.
For this purpose we usually create somewhat of a function hierarchy (see React's Data Flow), but this may require us to change our main function when we want a change in a smaller function. It also leads to highly nested code with data being passed through multiple levels for simple updates (known in React as Prop Drilling). So we're not going to go with that method.
Frameworks like Solid.js and Svelte.js solve this problem using Reactive Data Structures, often called Stores or Signals. Other frameworks may have slightly differing approaches, like React's Context and Vue's Vuex. We're going to implement the Solid/Svelte approach without using the framework.
Let's set up our code. We'll store all data, such as our score, in a file called data.js. Our main file, index.js, will be responsible for taking the score and displaying it, as well as importing the buttons.js file which contains the code for our buttons.
We could just create another script tag instead of an import, but I prefer this method.
Code below available at: https://codesandbox.io/s/reactor-p1-nu3ik
├── index.html
├── index.js
├── buttons.js
└── data.js
<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<title>Reactor Example</title>
<meta charset="UTF-8" />
<script type="module" src="index.js"></script>
</head>
<body>
<h1 class="score">0</h1>
<button class="score-increase">Increase Score</button>
<button class="score-decrease">Decrease Score</button>
</body>
</html>
// index.js
import './buttons.js';
import { score } from './data.js';
const h1 = document.querySelector('h1.score');
h1.textContent = `score: ${score}`;
Our first instinct here is just to export a variable called score that points to a number.
// data.js
export const score = 0;
// buttons.js
import { score } from './data.js';
const b1 = document.querySelector('button.score-increase');
b1.addEventListener('click', () => score++);
const b2 = document.querySelector('button.score-decrease');
b2.addEventListener('click', () => score--);
We're unfortunately going to run into a problem immediately. We cannot assign to any imported variables. They're defined as constant binding values when imported. Changing it to let won't help either, as it will only be mutable to the module it's exported from.
One option might be to use export let and also export a changeScore function which should have edit access. There's a simpler solution, however.
Using Objects
As with all constant variables in JavaScript, we actually can change its properties if it's an object. Moving score to an object with a value property is an easy fix there.
Code below available at: https://codesandbox.io/s/reactor-p2-5obug
// data.js
export const score = { value: 0 };
// buttons.js
// ...
b1.addEventListener('click', () => score.value++);
// ...
b2.addEventListener('click', () => score.value--);
// ...
// index.js
// ...
h1.textContent = `score: ${score.value}`;
Now this actually works. Our value is changed and the changes carries over from module to module. We're not seeing any change visually, however. When we click our buttons, the h1 does not update.
This is because our code in index.js is only ran once. It has no idea when our data has changed. We can probably start an interval which sets our value ever few milliseconds, but this really isn't a viable option for everywhere that we end up using our score.
A better alternative is to have our score tell everyone when its value changes. Like a newspaper, we can give people the option to subscribe and we'll notify them when we get a new issue... or value.
Subscribers
This requires us to know when we've been mutated. We usually use functions for this thing, but we can preserve using .value by turning our object into a class and creating getters and setters.
Note that, with the exception of Vue.js and a few others, this isn't often how reactivity libs work - we often just use functions for updating. For this article, I prefer the OOP method as it cuts down on some code complexity. We don't need a separate read, set, and update method (update takes a function, whereas set only takes a value). I advise you to look up getters and setters in JS, however, if you're unfamiliar.
Code below available at: https://codesandbox.io/s/reactor-p3-e8dxg
// reactor.js
export class Reactor {
constructor(value) {
// private value for where it's really stored
this._val = value;
// private list of functions to be notified
this._subscribers = [];
}
// return value when requested
get value() {
return this._val;
}
// set value and then notify everyone
set value(newVal) {
this._val = newVal;
for (const subscribeFunc of this._subscribers) {
subscribeFunc(newVal);
}
}
// add function to subscriber list and immediately invoke
subscribe(func) {
this._subscribers.push(func);
func(this._val);
}
}
One way that we differ from a newspaper is that subscribers get a value instantly upon subscription. This lets our score counter work without having to set it an additional time right before subscribing, but it's also important to keep this in mind for a feature we're going to add later.
// data.js
import { Reactor } from "./reactor.js";
export const score = new Reactor(0);
// index.js
// ...
score.subscribe(val => {
h1.textContent = `score: ${val}`;
});
At this point we've already created a reactive data structure. The fact that this reacts to changes and updates its subscribers is the reactivity we've been looking for. We can have one reactive value update another reactive value and create chains of reactivity.
const score = new Reactor(0);
const halfScore = new Reactor(0);
score.subscribe(val => halfScore.value = val/2);
One thing we can't really do as easily though is have one value change in response to any of multiple values changing. What if we want to generate a high score out of multiple reactive scores? We might do something like this:
// example.js
import { Reactor } from './reactor.js';
const scores = new Reactor([]);
const highScore = new Reactor(0);
// finds highest reactive score and changes highScore to it
function setHighScore(val) {
// we use this for scores as well, so check if it's a number
let highestNum = typeof val === "number" ? val : 0;
for (const score of scores.value) {
if (score.value <= highestNum) continue;
highestNum = score.value;
}
highScore.value = highestNum;
}
// adds new score and makes it reactive when changed
function addScore(num = 0) {
const score = new Reactor(num);
score.subscribe(setHighScore);
// we cannot use .push() - we need to use = for it to react
scores.value = [...scores.value, score];
}
addScore(0);
addScore(45);
addScore(26);
This looks a bit messier than I'd like it to. We're forced to have our addScore also subscribe each score individually. Since our subscribe function is called immediately, we're also updating the highScore when add add a new one, but if we added one any other way, it wouldn't update the high score.
Computed Values
There's a cleaner way - computed values. At the cost of more complex library code, we get a cleaner user experience. Here's what a computed version of that code might look like.
import { Reactor, computed } from './reactor.js';
const scores = new Reactor([]);
const highScore = computed(() => {
let highestVal = 0;
for (const score of scores.value) {
if (score.value <= highestVal) continue;
highestVal = score.value;
}
return highestVal;
});
highsScore.subscribe(num => console.log('high score: ' + num));
// high score: 0
scores.value = [new Reactor(0)];
// high score: 0
scores.value = [...scores.value, new Reactor(45)];
// high score: 45
scores.value = [...scores.value, new Reactor(26)];
// high score: 45
const firstScore = scores.value[0];
firstScore.value = 103;
// high score: 103
I'm not sure if we're all looking at the same code here, but this looks like magic to me.
Our high score will change whenever a new value is added or when any value inside of it changes its own value.
...how?
We're not subscribing to anything. How does the computed function know about which variables are inside of it? We're not stringifying anything and we're not doing static analysis. We're using an array, so there aren't any unique variable names. Is this something specifically with arrays?
Nope! Here's a sample with some other values:
import { Reactor, computed } from './reactor.js';
const num1 = new Reactor(45);
const num2 = new Reactor(92);
const unusedVal = new Reactor(34);
const num4 = computed(() => num1.value + num2.value);
num4.subscribe(num => console.log('num4: ' + num));
// num4: 137
num1.value = 8;
// num4: 100
num2.value = 2;
// num4: 10
unusedVal.value = 17;
// num4 is unchanged and doesn't console.log since we never used unusedVal for num4
A computed value is like a regular subscription, but it allows us to subscribe, dynamically, to multiple values. It knows exactly which reactive variables are inside it and only has them specifically subscribed.
This seems impossible unless computed and Reactor are communicating in some way. They're separate, but they must share some sort of local state or else there's no way this is possible.
And that's right on the mark. The trick to all of this working is the following:
- We automatically run subscriptions once after subscribing.
- There is a single (non-exported, but top-level) variable in the same module as both
computedandReactorthat may or may not have a value at any given time.
The Trick
So computed is able to communicate with Reactor by the following method:
- Set our local variable (
computeFunc) to the function passed tocomputed. - Run the function passed to
computedonce. - Have
Reactorvalues automatically subscribe tocomputeFuncwhen they're read from andcomputeFuncis not empty. - Set
computeFuncback to what it was before.
This way, we're able to communicate with all reactive values in the function without knowing specifically what they are, since it's the job of the reactive values themselves to check this variable.
To reiterate, since this is perhaps the most complex part of this article - both computed and Reactor have computeFunc in scope. computeFunc is usually empty. As JS, in this context, is single threaded, the only time it ever contains a value is exactly when computed initially runs. This way we're ensuring that every Reactor inside the function passed to computed subscribes to this function. If we did not set computeFunc back to what it was before (usually undefined), then every reactive value would subscribe to it - even ones not related to any computed.
We set it back to "what it was before" and not undefined because computed values can contain computed values. This means we may be getting deep into some stack and since every computed uses the same variable, computeFunc, we need to set it back to was before, as it may have not been undefined, but just some other function.
That was a lot of talk and perhaps it may be clearer in code. A computed value is just a regular Reactor, so let's set that up first.
// reactor.js
export function computed(func) {
// we can give it anything, since we're changing it momentarily
const reactor = new Reactor(null);
// run it immediately to get a new value
reactor.value = func();
return reactor;
}
// ...
This doesn't look like much yet. Let's add our local variable and change Reactor to check for it.
Code below available at: https://codesandbox.io/s/reactor-p4-1tcij?file=/reactor.js
// reactor.js
// initially undefined. We can set it to null instead.
let computeFunc;
export function computed(func) {
const reactor = new Reactor(null);
// THIS is the function we subscribe to, which updates the reactor
const fn = () => reactor.value = func();
// set computeFunc to fn and store previous value for later
const prevVal = computeFunc;
computeFunc = fn;
fn();
// set computeFunc back to previous value
computeFunc = prevVal;
return reactor;
}
export class Reactor {
// ...
get value() {
// If it exists, we add it to the subscribers.
// Do not call it, unlike a regular subscriber.
if (computeFunc) this._subscribers.push(computeFunc);
return this._val;
}
// ...
}
And now computed works! We can create new reactive values from other ones.
We're not quite done yet, however. We'll find that our array example does not work yet. This is because our computed function does not account for dynamically added values.
Accounting For Arrays & Cleanup
We're only setting computeFunc on the initial function creation, so only the Reactors that are inside the computeFunc on initial creation will subscribe to fn. With our array example, we're adding reactive values even after computed is initially called. We need to change fn to account for that.
Code below available at: https://codesandbox.io/s/reactor-p5-cdx10?file=/reactor.js
export function computed(func) {
const reactor = new Reactor(null);
// move the local variable assignment into the subcribed function
const fn = () => {
const prevVal = computeFunc;
computeFunc = fn;
reactor.value = func();
computeFunc = prevVal;
};
fn();
return reactor;
}
The problem with this is that we're now going to run into an infinite loop. Whenever a reactive value in the computed is changed, we loop through our subscribed functions and call them.
Then the function we're subscribing to is setting ComputeFunc and calling our get value method. Doing that forces us to add a subscriber to ourself. We're adding a subscriber while looping through subscribers, so we always have another subscriber to loop over. Thus, an infinite loop.
A quick solution is making sure we have no duplicates of any functions in our array. Move our array to a new Set().
export class Reactor {
constructor(value) {
// ...
this._subscribers = new Set();
}
get value() {
// change from .push() to .add()
if (computeFunc) this._subscribers.add(computeFunc);
// ...
}
subscribe(func) {
this._subscribers.add(func);
// ...
}
}
At this point we may want to add some more cleanup code. Different reactive libs have different sort of safe guards and differing ways to do similar things. We may want to firstly add an unsubscribe function, which is usually just returned from the subscribe function.
subscribe(func) {
this._subscribers.add(func);
func(this._val);
// remove the subscriber
return () => this._subscribers.delete(func);
}
Using Set makes this process super clean.
We also may want to add some infinite loop protection. That can be done by checking if the function we're in (fn) is equal to computeFunc.
if (fn === computeFunc) {
throw Error("Circular computation detcted");
}
Now doing the following throws an error instead of lagging the page until your tab crashes:
import { Reactor, computed } from './reactor.js';
const num1 = new Reactor(0);
// ERROR: Circular computation detected
const num2 = computed(() => {
num1.value++;
return num1.value + 1;
});
Practical Application - Mini Framework
At this point I was going to see if I could describe how RxJs's approach differs from ours. Instead I think I'm going to show how we can turn our library into a mini framework, to illustrate the effectiveness of this approach.
We often want frameworks to be fairly reactive - where changes to variables are reflected in the DOM and vice versa. Our reactive system is perfect for this.
Code below available at: https://codesandbox.io/s/reactor-p6-ynq3h
import { Reactor, computed } from './reactor.js';
import { get, create } from './framework.js';
const num1 = new Reactor(0);
const num2 = new Reactor(0);
const total = computed(() => num1.value + num2.value);
const inputOptions = {
rejectOn: isNaN,
mutator: Number,
};
const input1 = create('input')
.bind('value', num1, inputOptions);
const input2 = create('input')
.bind('value', num2, inputOptions);
const span = create('span')
.bind('textContent', total);
get('body')
.append(input1)
.append(' + ')
.append(input2)
.append(' = ')
.append(span);
Our framework exposes 2 functions - get and create which wrap HTMLElements in a class called El. This class exposes the methods bind, append, and on. With simple rules, we can create a 2-way binding between our reactive values and input elements.
get simply uses document.querySelector(). create is a simple call to document.createElement(). on is .addEventListener() and append is .appendChild().
bind is the interesting one here.
bind(name, funcOrReactor, options = {}) {
// grab reactor from function, if it isn't a reactor
const reactor = funcOrReactor instanceof Reactor ? funcOrReactor : computed(funcOrReactor);
// if editing value, apply 2-way binding
if (name === 'value') {
this.on('input', e => {
const val = options.mutator ? options.mutator(e.target.value) : e.target.value;
if (options.rejectOn && options.rejectOn(val)) return;
reactor.value = val;
});
// change property when reactive value changes
reactor.subscribe(val => this._el[name] = val);
} else if (name === 'textContent') {
reactor.subscribe(val => this._el[name] = val);
} else {
// if not textContent or value, it's probably an attribute
reactor.subscribe(val => this._el.setAttribute(name, val));
}
// allow method to be chained
return this;
}
bind just adds a subscription unless the name is value in which case it also tries to change the reactive value with an eventListener. In such a case, we can mutate the value and possibly prevent updates with rejectOn. Here we're using it to prevent non-valid numbers from getting in our reactive values.
Conclusion
I hope you learned a bit from this walk through. Special thanks to Ryan Carniato and Jimmy Breck-McKye who were instrumental in my understanding of all of this. I ended up rewriting Jimmy's library to fully understand some concepts. You can see that here if you'd like to improve your understanding of some concepts.
If you're up to it, let me know what you liked and didn't, so that I can improve my technical writing for future publications!
Top comments (4)
This code in p5 (Accounting For Arrays & Cleanup) doesn't make sense to me becase it seems you are incurring circular computation in
fnand usingnew set()in subscribers to avoid it, whereas the computed function should be subscribed for only once on creation imo. Why are we adding this subscriber and avoiding adding it every time it's run? My two cents: p4 code just works fine.My another confusion is on this line:
Is there any reason on adding
prevVal? Shouldn't it always benullorundefined? Unless there is some race conditions, likefnis being called whilecomputedFuncalready has a value but nothing seems to be concurrent hereThanks for sharing this. The writing and code are both neat and well-explained. This is a good example of observer and publisher pattern. However, I think debugging can get quite challenging while codebase grows larger with this pattern.
I don't quite understand the problem statement in Accounting For Arrays & Cleanup. At a quick glance, I just copied the code in reactor.js from p4 to p5 and it works.
After going through the code, I still don't understand the issue mentioned in below:
The p4 code just works fine, and the logic flow is
highScorefunction, it obtains the latest scores array and finds the highest scorehighScorereactor