Divergent States in a "Single Source of Truth" Framework

I'll tell you what I've learnt from struggling with a bug that made me lose a couple of weeks. The application framework used in this post is Hyperapp, but I guess the same problem can be found in frameworks based on transforming the state of "Single Source of Truth" with pure functions (such as Elm, Redux, so on) if we use them in a wrong way.

Introduction to Hyperapp with an Example App without the Bug

Hyperapp is a minimalistic framework that enables us to create virtual DOM based apps with architecture similar to The Elm Architecture (TEA). Hyperapp gives us both a view framework based on virtual DOM and a Redux-like state management framework without learning Elm with its smaller-than-favicon size.

Here is an example app of Hyperapp written in TypeScript¹. I'll use this app to demonstrate the problem. So it's much simpler than the app I'm actually developing, but may look too complicated as an example of Hyperapp. Sorry!

const buildInit = (): State => {
  const result = new Map();
  for (let i = 0; i < 3; ++i){
    const key1 = randomKey();
    result.set(key1, {});
    for (let j = 0; j < 3; ++j){
      const key2 = randomKey();
      result.get(key1)[key2] = false;
    }
  }
  return result;
};

// Action functions to update the State object
const SetTrue = (state: State, [key1, key2]: [string, string]): State => {
  const newState = new Map(state);
  const childState = newState.get(key1);
  newState.set(key1, {
    ...childState,
    [key2]: true,
  });
  return newState;
};
const SetFalse = (state: State, [key1, key2]: [string, string]): State => {
  const newState = new Map(state);
  const childState = newState.get(key1);
  newState.set(key1, {
    ...childState,
    [key2]: false,
  });
  return newState;
};

const view = (state: State) => {
  const children1 = [];
  // Side note: I should use the `map` method here, but I couldn't find
  // it's available just until I write up most of this post....
  for (const [key1, childState] of state.entries()) {
    const children2 = [];
    for (const key2 of Object.keys(childState)) {
      children2.push(
        h(
          "span",
          {
            onmousemove: [SetTrue, [key1, key2]],
            onmouseleave: [SetFalse, [key1, key2]],
            class: { elem: true, true: childState[key2] },
          },
          text(key2),
        )
      );
    }
    children1.push(
      h(
        "div",
        { class: "elem" },
        [
          text(key1),
          ... children2,
        ]
      )
    );
  }
  return h("div", {}, children1);
};

// Run the app
app({
  init: buildInit(),
  view: view,
  node: document.getElementById("app"),
});

First of all, a basic application in Hyperapp like this is split into three parts: State, View, and Action. Each of them corresponds to Model, View, Update of TEA:

• State: The application's internal state. Updated by Actions.
• View: Function which takes a State to return a virtual DOM object.
• Action: Function which takes a State to return a new State.

Like a View in TEA generates messages from its event handlers (event handlers set in the virtual DOM returned by the View), a View in Hyperapp dispatches Actions from its event handlers. This is a code to show how event handlers are set in the View, excerpted from the example view function above:

h(
  'span',
  {
    onmousemove: [SetTrue, [key1, key2]],
    onmouseleave: [SetFalse, [key1, key2]],
    class: { elem: true, false: state[key1][key2] },
  },
  text(key2)
)

The h function is the construtor of a virtual DOM object in Hyperapp, which takes the name of the tag (here it's span), attributes of the element as an object, and the element's child node(s) (here it's text(key2)). Actions dispatched by the event handlers are set in the second argument's on*** attributes, along with their extra argument (called "payload"). In the extracted code, the Action SetTrue with the payload [key1, key2] is dispatched by a mousemove event, and SetFalse with [key1, key2] is dispatched by a mouseleave.

What happens after SetTrue or SetFalse gets dispatched? Hyperapp calls the dispatched Action with the current State, its payload, and the event object created by the mousemove/mouseleave event (but the event object is not used in the example!). Then, it passes the State returned by the Action to the View to get the refreshed virtual DOM to update the actual DOM tree. And the (re)rendered DOM tree again waits for new events to dispatch Actions.

Okay, those are the important points of Hyperapp that I want you to know before explaining the problem.

Compare the Behavior of the Buggy App and Non-Buggy One

Now, let's see how the example app works and how it gets broken by the mistake I made. I created a StackBlitz project containing both one without the bug and one with the bug in a single HTML file:

https://stackblitz.com/edit/typescript-befryd?file=index.ts

And this is the screenshot of the working app:

The example app is as simple as it receives mouseover events to paint the <span> element blue, then gets it back by mouseleave events.

By contrast, the example app I injected the bug into doesn't restore the color of the mouseleaveed <span> element:

The Change that Caused the Problem

What's the difference between the buggy example and the non-buggy example? Before diggging into the details, let me explain the motive for the change. The function that made me anxious is the SetTrue Action (and SetFalse did too):

const SetTrue = (state: State, [key1, key2]: [string, string]): State => {
  const newState = new Map(state);
  const childState = newState.get(key1);
  newState.set(key1, {
    ...childState,
    [key2]: true,
  });
  return newState;
};

The expression newState.get(key1) may return undefined since newState is a Map in the standard library and its get method returns undefined if the Map doesn't contain the associated value. TypeScript doesn't complain about this because { ...undefined } returns {} without a runtime error!, but catching undefined here is not expected. And in my actual app, it isn't evident that the get method always returns a non-undefined value.

Checking if the result of newState.get(key1) is undefined or not is trivial enough, but looking out over the view and SetTrue/SetFalse functions, you will find that the value equivalent to newState.get(key1), childState, is available as the loop variable of the outermost for ... of statement in view:

const view = (state: State) => {
  // ...
  for (const [key1, childState] of state.entries()) {
    // ...
  }
  // ...
};

That's why I decided to pass childState as one of the payload of SetTrue/SetFalse Action. I modified them as following:

const SetTrue = (
  state: State,
  [childState, key1, key2]: [ChildState, string, string]
): State => {
  const newState = new Map(state);
  newState.set(key1, {
    ...childState,
    [key2]: true,
  });
  return newState;
};

Note that the line const childState = newState.get(key1); is removed, then the local variable childState is defined as the part of the second argument instead. Now the view function gives the childState loop variable to SetTrue/SetFalse :

const view = (state: State) => {
  // ...
  for (const [key1, childState] of state.entries()) {
    // ...
          {
            onmousemove: [SetTrue, [childState, key1, key2]],
            onmouseleave: [SetFalse, [childState, key1, key2]],
            // ...
          }
    // ...
  }
  // ...
};

ℹ️ In the StackBlitz project I showed before, the view, SetTrue and SetFalse functions containing theses changes are renamed into viewBuggy, SetTrueBuggy, and SetFalseBuggy, respectively.

These changes freed SetTrue/SetFalse from undefined-checking for every child of the state. That would improve the app's performance a little (But too little to see. Don't be obcessed with that!).

Unfortunately, this is the very trigger of the weird behavior! The application won't handle the mouseleave event correctly anymore. It leaves the mouseleaveed element blue if the mouse cursor moves onto another element which also reacts to the mousemove event.

What Hyperapp does After a DOM Event Occurs

Why does the change spoil the app? Learn how Hyperapp updates the state first of all to get the answer. According to the source code, Hyperapp handles DOM events as follows:

• (1) Call the Action assigned to the event with the state, the payload, and event object.
  • In the case of this article's app, the Action is SetTrue or SetFalse, and the payload is [key1, key2] or [childState, key1, key2].
• (2) Update the internal variable named state if the Action returns a state different from the original state (compared by !==).
  • Do nothing and stop the event handler if the returned state has no change.
• (3) Unless the render function (introduce later) is still running, enqueue the render function by requestAnimationFrame (or setTimeout if requestAnimationFrame is unavailable).
• (4) The render function runs: call the view function with the updated state to produce renewed virtual DOM object, then compares each child of the old and new virtual DOM tree to patch the real DOM.

This is sumarrized into the pseudo JavaScript code below:

// Variables updated while the app is alive
let state, virtualDom, rendering = false;

const eventHandlers = {
  onmousemove: [SetTrue, payload],
  onmouseleave: [SetFalse, payload],
};

// (1)
const [action, payload] = eventHandlers[eventName];
const newState = action(state, payload);

if (state !== newState){
  // (2)
  state = newState;

  // (3)
  if (!rendering){
    rendering = true;
    enqueue(render);
  }
}

// (4)
function render(){
  const newVirtualDom = view(state);
  compareVirtualDomsToPatchTheRealDom(virtualDom, newVirtualDom);
  virtualDom = newVirtualDom;
  rendering = false;
}

Recall how browsers handle JavaScript tasks. Task here is a function call associated with the event (by addEventListener etc). The browser keeps running the function until the call stack gets empty without any interruption. In respect to the event handler of Hyperapp, from (1) to (3) is recognized as a single task (if the state changes). Because (3) just calls requestAnimationFrame to put off calling the render function. What requestAnimationFrame does is only registering the given function as a new task to execute later, then finishes its business. So the task initiated by the event finishes after calling requestAnimationFrame. As (3) does, calling requestAnimationFrame and do nothing afterwards is a typical way to let the browser process another task. You will find that the browser treats requestAnimationFrame as a special function according to the call stack by setting a break point to step in to the render function:

This is an example call stack in Microsoft Edge's DevTools². This shows that requestAnimationFrame switched the task running. The replaced task doesn't techinacally share the call stack with the older one, but Edge's debugger artificially concatenates them for convinience.

Consecutive Events Update the State in Order

The flow I described in the last section can be interrupted between (3) and (4), if a new task is enqueued while Hyperapp is performing (1)-(3). Such interruptions unsurprisingly happen when a pair of events occur simultaneously like the example app. That interruptions are caused by say, mouseover after mouseleave, focus after blur, and so on. As long as your app's Actions just receive the state and return the updated one, there are no problem because (2) definitely updates the state before yielding the control to the subsequent event. When a couple of serial events take place, the pseudo JavaScript code is rewritten as this:

// ... omitted...

// (1) `eventName` can be `mouseleave`, `blur` etc.
const [action, payload] = eventHandlers[eventName];
const newState = action(state, payload);

if (state !== newState){
  // (2)
  state = newState;

  // (3)
  if (!rendering){
    rendering = true;

    // (1'): (1) for another event.
    // `anotherEventName` can be `mouseenter`, `focus` etc.
    const [action, payload] = eventHandlers[anotherEventName];
    const newState = action(state, payload);
    if (state !== newState){
      // (2')
      state = newState;

      // (3') is omitted because `rendering` here must be true.
    }

    // Now, `render` renders `state` updated by (2').
    // So the `render`ed State is up-to-date even after a sequence of
    // simultaneous events are dispatched.
    render();
  }
}

// (4)
function render(){
  // ... omitted...
}

State Diverges by Unexpected References in the View

The point in the previous pseudo code is that state doesn't get stale: State passed as the first argument to the View and any Actions is always up-to-date. They wouldn't refer the State before updated by older events. Thus Hyperapp achieves "State is the Single Source of the Truth" --- as far as the Actions and their payload use the State correctly.

Now, review the example app's view function with the problem:

const view = (state: State) => {
  // ...
  for (const [key1, childState] of state.entries()) {
    // ...
          {
            onmousemove: [SetTrue, [childState, key1, key2]],
            onmouseleave: [SetFalse, [childState, key1, key2]],
            // ...
          }
    // ...
  }
  // ...
};

To make sure childState is available for the SetTrue and SetFalse Action, I put it in their payload. Payload is set as the value of on*** attribute of the virtual DOM node with its Action. So childState remains unchanged until the view function is called with the updated state. That means Actions can take the State updated by the precedent event and one not-yet updated. The State has diverged!

Let's see the details by revising the pseudo JavaScript:

// Variables updated while the app is alive
let state, virtualDom, rendering = false;

// Up until now, `eventHandlers` is defined as an independent variable for
// simplicity. But it's actually set as properties of `virtualDom` by the
// `view` function.
virtualDom.eventHandlers = {
  onmousemove: [SetTrue, state.childState],
  onmouseleave: [SetFalse, state.childState],
};

// (1). `eventName` can be `mouseleave`, `blur` etc.
const [action, payload] = eventHandlers[eventName];
const newState = action(state, payload);

if (state !== newState){
  // (2)
  state = newState;

  // (3)
  if (!rendering){
    rendering = true;

    // (1') ⚠️`state` is already updated in (2), but `virtualDom` is not yet!
    // So `virtualDom.eventHandlers` with its payload aren't either!
    const [action, payload] = virtualDom.eventHandlers[anotherEventName];
    const newState = action(state, payload);
    if (state !== newState){
      // (2')
      state = newState;

      // (3') is omitted because `rendering` here must be true.
    }

    render();
  }
}

// (4)
function render(){
  const newVirtualDom = view(state);
  compareVirtualDomsToPatchTheRealDom(virtualDom, newVirtualDom);
  // `virtualDom.eventHandlers` are finally updated here. But too late!
  virtualDom = newVirtualDom;
  rendering = false;
}

The virtual DOM object is updated by View, and the State is updated by an Action. Due to the difference in when they are called, the Action can process an outdated state left in the virtual DOM as payload.

Conclusion and Extra Remarks

• Don't refer to (some part of) the State in Actions except as their first argument.
  • In addition to payload, we have to take care of free variables if we define Actions inside the View.
• I suspect we might encounter the same problem in other frameworks with the similar architecture (e.g. React/Redux, Elm).

1. Hyperapp doesn't force you to use TypeScript of course. But I'll use TypeScript for easier description of the type of the state. ↩

2. I'm afraid I failed to reproduce a call stack like this with the debugger of my favorite Firefox's DevTools. 😞 ↩