Aral Roca

Posted on Apr 25 • Originally published at aralroca.com

SPA-Like Navigation Preserving Web Component State

Original article: https://aralroca.com/blog/spa-navigation

In this third and final article in the series on HTML Streaming, we will explore the practical implementation of the Diff DOM Streaming library in web browsing. This approach will allow any website using web components to retain its state during browsing. We will discuss in detail how to achieve this step by step using VanillaJS and Bun.

It is important to note that it is not necessary to have a complex server, as we can use static files. However, we will also explore how this technique works with HTML streaming and we will stream from Bun.

1. Creating "Hello World" page
2. Adding more than one page
3. Adding a Web Component
4. Maintaining the state during navigation
5. Manage new scripts of the page to be navigated to
6. Working with streaming and suspense
7. Transitions between pages (View Transition API)
8. Caching the navigation
9. Prefetch the navigation
10. Pros and Cons of Using Diff DOM Streaming for Web Navigation
Final conclusions
References

1. Creating "Hello World" page

The first thing we are going to do is to create the home page with a "Hello World" with Bun:

const server = Bun.serve({
  port: 1234,
  fetch() {
    return new Response(
      `
      <html>
        <head>
          <title>Hello, World!</title>
        </head>
        <body>
          <h1>Hello, World!</h1>
        </body>
      </html>
    `,
      {
        headers: {
          'Content-Type': 'text/html',
        },
      }
    )
  },
})

console.log(`Done! http://${server.hostname}:${server.port}`)

Now if we run it with:

bun run index.ts

And we open http://localhost:1234 We will see our page with an h1 that says Hello, World!. Ok, all good.

Hello World with Bun

2. Adding more than one page

Let's create the page /foo and the page /bar and instead of "Hello World" put "Foo" and "Bar".

To do this, we must get the pathname from the url of the request:

// Hardcoded examples of pathname-dependent example
const names = {
  '/foo': 'Foo',
  '/bar': 'Bar',
}

const server = Bun.serve({
  port: 1234,
  fetch(req: Request) {
    // request URL:
    const url = new URL(req.url)
    // Getting the hardcoded name via the pathname
    const name = names[url.pathname] ?? 'Hello, World'

    return new Response(
      `
      <html>
        <head>
          <title>Hello, World!</title>
        </head>
        <body>
          <nav>
            <a href="/">Home</a>
            <a href="/foo">Foo</a>
            <a href="/bar">Bar</a>
          </nav>
          <h1>${name}!</h1>
        </body>
      </html>
    `,
      {
        headers: {
          'Content-Type': 'text/html',
        },
      }
    )
  },
})

console.log(`Done! http://${server.hostname}:${server.port}`)

If we run it and go to http://localhost:1234/foo we will see our page with an h1 that says Foo!, and if we go to http://localhost:1234/bar we will see our page with an h1 that says Bar!. Ok, all good.

In addition, we have added three a elements to navigate between pages 👌

<nav>
  <a href="/">Home</a>
  <a href="/foo">Foo</a>
  <a href="/bar">Bar</a>
</nav>

For now, there is nothing special about this type of navigation, it is the default navigation of the browsers.

Hello World using MPA with Bun

3. Adding a Web Component

For the example, we are going to add a web component with VanillaJS that acts as a counter, since it is a simple example to test a client interaction:

// Counter Web Component
class CounterComponent extends HTMLElement {
  connectedCallback() {
    const shadowRoot = this.attachShadow({ mode: 'open' })
    let count = 0

    shadowRoot.innerHTML = `
      <button id="inc">Increment</button>
      <button id="dec">Decrement</button>
      <p id="count">Count: ${count}</p>
    `
    const countEl = shadowRoot.querySelector('#count')
    shadowRoot.querySelector('#inc').addEventListener('click', () => {
      count++
      countEl.textContent = `Count: ${count}`
    })
    shadowRoot.querySelector('#dec').addEventListener('click', () => {
      count--
      countEl.textContent = `Count: ${count}`
    })
  }
}

// Register Counter Web Component
if (!customElements.get('counter-component')) {
  customElements.define('counter-component', CounterComponent)
}

To add it, after using the CustomElementRegistry, we can consume it as another HTML element with <counter-component></counter-component>.

What we are going to do is to add it under the heading of each page:

import path from 'node:path'

const names = {
  '/foo': 'Foo',
  '/bar': 'Bar',
}

const server = Bun.serve({
  port: 1234,
  fetch(req: Request) {
    const url = new URL(req.url)
    const name = names[url.pathname] ?? 'Hello, World'

    if (url.pathname === '/code') {
      return new Response(Bun.file(path.join(import.meta.dir, 'code.js')))
    }

    return new Response(
      `
      <html>
        <head>
          <title>Hello, World!</title>
        </head>
        <body>
          <nav>
            <a href="/">Home</a>
            <a href="/foo">Foo</a>
            <a href="/bar">Bar</a>
          </nav>
          <h1>${name}!</h1>
          <counter-component></counter-component>
          /* 👇 Counter Web component code here */
          <script src="https://aralroca.com/code.js"></script>
        </body>
      </html>
    `,
      {
        headers: {
          'Content-Type': 'text/html',
        },
      }
    )
  },
})

console.log(`Done! http://${server.hostname}:${server.port}`)

Now if we run it again we will see that there is a counter on each page and we can interact with it to change its value!

Our first web component

However, when we navigate between pages, the counter is reset to 0 😬

The thing is that each navigation loads the page again and therefore the web component script is loaded again and returns to its initial state.

The browsers traditional navigation what they do is to replace all the current document, for the new one. What we want is that it only updates what has changed of the DOM, if the web component does not change, why reset it?

4. Maintaining the state during navigation

There are many strategies to preserve the state, such as saving the value in sessionStorage so that at startup it loads its initial value again, but that is not what we are going to do, we are going to do something better, and that is to update during navigation only the DOM changes that change, and not the rest, therefore, only the heading should change, preserving the web component between pages so that it maintains its state.

To do this, we cannot use the browser's natural navigation, we have to apply a diff between the current document and the HTML stream, for this to be possible, we have to register the navigate event for the browsers that support it:

if ('navigation' in window) {
  window.navigation.addEventListener('navigate', spaNavigation)
}

Now what would be missing would be to implement the logic of the spaNavigation function to do the diff. As we have commented previously, we are going to use the library Diff DOM Streaming. For it, we will only intercept the navigations that are with the same origin:

function spaNavigation(event) {
  const url = new URL(event.destination.url)

  // Avoid intercepting different origins
  if (location.origin !== url.origin) return

  event.intercept({
    /* TODO */
  })
}

The event.intercept allows us to add a handler to implement our own navigation, the browser will change the URL automatically, but instead of processing the request and updating the DOM by the browser here it will be done by us:

function spaNavigation(event) {
  const url = new URL(event.destination.url)

  if (location.origin !== url.origin) return

  event.intercept({
    async handler() {
      // Fetch new HTML Streaming document
      const res = await fetch(url.pathname, { signal: event.signal })

      if (res.ok) {
        // Lazy import of the diff-dom-streaming library
        const diffModule = await import(
          'https://unpkg.com/diff-dom-streaming@latest'
        )
        const diff = diffModule.default

        // Scroll to the top of the page
        document.documentElement.scrollTop = 0

        // Applies the diff between the current document and the stream reader.
        await diff(document, res.body.getReader())
      }
    },
  })
}

In the handler the first thing we do is request the new page, once the request has been processed, we load the library in a lazy way (it will only do it the first time), make the page scroll up and apply the diff with the reader.

In this way, it now only updates the parts that change during browsing and therefore keeps the web components alive without resetting them again.

Now if we run the file and check the navigation, we see that it keeps the last state of the counter!

Preserving Web Component State

Ok, we are doing well!

5. Manage new scripts of the page to be navigated to

If we put another script with a console.log of the name (what you see as heading) <script>console.log('${name}')</script>, our expectation is that this console.log will come up every time you browse, since the script is different, in fact it could even be another totally different script that needs the page to work.

If we add it, we see that this does not happen, it only runs on the initial loading of the first page, but then during navigation the console.log is not repeated again!

New scripts are not executed

How can we solve it?

The Diff DOM Streaming, only makes a change at DOM level, however, to execute the new scripts in this one we can do it by means of an extension:

registerCurrentScripts()

await diff(document, res.body.getReader(), {
  onNextNode: loadScripts,
})

Where registerCurrentScripts and loadScripts are:

const scripts = new Set()

function registerCurrentScripts() {
  for (let script of document.scripts) {
    if (script.id || script.hasAttribute('src')) {
      scripts.add(script.id || script.getAttribute('src'))
    }
  }
}

// Load new scripts
function loadScripts(node) {
  if (node.nodeName !== 'SCRIPT') return

  const src = node.getAttribute('src')

  if (scripts.has(src) || scripts.has(node.id)) return

  const script = document.createElement('script')

  if (src) script.src = src

  script.innerHTML = node.innerHTML

  // Remove after load the script
  script.onload = script.onerror = () => script.remove()

  document.head.appendChild(script)

  // Remove after append + execute (only for inline script)
  if (!src) script.remove()
}

What the previous code does is that during the application of the diff, all the scripts that have been detected that have some change and at the same time have not been executed previously, we execute them.

New scripts now are executed

If we now run it again, we will see that:

It preserves the state of the web component when browsing.
The console.log is executed every time we navigate.

That is to say, now we have a functional SPA, without changing the navigation history, that works in streaming and it works both with <a href="/foo" /> and with imperative navigation as location.assign('/foo') 🥳.

6. Working with streaming and suspense

In the previous examples we have used a normal Response from Bun, without streaming. With streaming we could use a Suspense as explained in the first article and keep it running during navigation.

let suspensePromise
// ...
return new Response(
  new ReadableStream({
    async start(controller) {
      // Some initial chunks...

      // Add "Suspense" placeholder
      controller.enqueue(encoder.encode('<div id="suspense">Loading...</div>'))

      // Expensive chunk:
      suspensePromise = Bun.sleep(2000).then(handleExpensiveChunk)

      // "Unsuspense" code
      function handleExpensiveChunk() {
        controller.enqueue(
          encoder.encode(`
            <template id="suspensed-content"><div>Expensive content</div></template>
          `)
        )
        controller.enqueue(
          encoder.encode(`
            <script>
              function unsuspense() {
                const suspensedElement = document.getElementById('suspense');
                const ususpensedTemplate = document.getElementById('suspensed-content');

                if (suspensedElement && ususpensedTemplate) {            
                  suspensedElement.replaceWith(ususpensedTemplate.content.cloneNode(true));
                  ususpensedTemplate.remove();
                }
              }
              unsuspense();
            </script>
          `)
        )
      }

      // ... Rest of chunks
      await suspensePromise
      controller.close()
    },
  })
)

In order to create the suspense, it's important to take into account the following:

Add the chunk with placeholder so that it occupies the place that the expensive chunk will occupy later.
Save the promise that loads the content of the expensive chunk and do not make the await until the end of the streaming. In this example I have used Bun.sleep of 2 seconds to simulate this load.
In the .then of the promise that makes the modification of the temporary chunk by the end.

Suspense with streaming by default works both for the initial load, as well as during normal browser navigation. Nevertheless, with our navigation implementation it also works 🥳

SPA that works with suspense and streaming

So we already have a SPA that works with suspense and streaming! And we haven't had to make any changes to our client code. However, we have benefits, while navigating to the other page with suspense mode, we can interact with our web components, this is wonderful!

7. Transitions between pages (View Transition API)

By using our own implementation, apart from making SPA-like navigation behave without having to handle the navigation history with pushState and being able to use more standard things like location.assign or HTML hyperlinks (a), we can also add styled transitions thanks to the View Transition API.

The View Transitions API provides a mechanism for easily creating animated transitions between different DOM states while also updating the DOM contents in a single step, and yes, they can be used during streaming, and divide it into different steps. To activate it, it is only necessary to pass the transition: true setting to Diff DOM Streaming library:

await diff(document, res.body.getReader(), {
  onNextNode: loadScripts,
+ transition: true
})

The chunk of the "unsuspense" script can also be set to use the View Transition API:

<script>
-  function unsuspense() {
+  async function unsuspense() {
+    if (window.lastDiffTransition) await window.lastDiffTransition.finished;
    const suspensedElement = document.getElementById('suspense');
    const ususpensedTemplate = document.getElementById('suspensed-content');

    if (suspensedElement && ususpensedTemplate) {
-      suspensedElement.replaceWith(ususpensedTemplate.content.cloneNode(true));
+      document.startViewTransition(() => {
+        suspensedElement.replaceWith(ususpensedTemplate.content.cloneNode(true));
+      });
       ususpensedTemplate.remove();
    }
  }
  unsuspense();
</script>

The Diff DOM Streaming library exposes the window.lastDiffTransition so we can get the ViewTransition and do things like wait for it to be done to make sure the template is rendered.

SPA with suspense, streaming and View Transition API

To better visualize the transition animation, we can exaggerate its timing with CSS:

<style>
  ::view-transition-old(root),
  ::view-transition-new(root) {
    animation-duration: 0.5s;
  }
</style>

Incremental vs full transition

Many times it will make more sense to use a complete transition instead of incremental, especially if we do not use suspense and we want a single transition at once instead of several, in this case, instead of using the configuration, we can use the View Transition API directly:

+ document.startViewTransition(async () => {
await diff(document, res.body.getReader(), {
  onNextNode: loadScripts,
-  transition: true,
});
+});

8. Caching the navigation

Naturally, if you have already visited the page before, it should be cached during navigation, so that it would not be necessary to display the suspense again and show the actual content of the page directly.

To make this possible, from Bun we can add the following header to the Response:

return new Response(
  new ReadableStream({
    /* ... chunks of html ... */
  }),
  {
    headers: {
      'cache-control': 'public, max-age=31536000, immutable',
    },
  }
)

The suspense is no longer shown when caching

When being cached instead of receiving different chunks in streaming and processing the suspense while waiting for the expensive chunk, the diff have everything at once and therefore the suspense stops coming out because the browser has already cached the real content.

👉 In development it will be better to set "no-store, must-revalidate" to be able to work and see the changes without being cached.

9. Prefetch the navigation

Sometimes it is necessary to use prefetch, because we can know what will be the next page that the user will visit. An example would be if an e-commerce checkout process has different steps on different pages, we know what the next page will be and we want that when the user performs the action it is displayed as easy as possible. In this case, we could add using the normal prefetch of the platform that would continue to work with our SPA-like implementation:

<link rel="prefetch" href="/foo"></link>

In this case, even if we clear the cache, navigating to /foo will always go fast, because on the previous page its HTML is requested to improve navigation.

Prefetching the /foo page

If we look at the Network tab, refreshing the home page already downloads the foo page, so that navigating to it is instantaneous without showing the suspense.

10. Pros and Cons of Using Diff DOM Streaming for Web Navigation

Pros:

State Maintenance: The implementation allows for maintaining the state of web components during navigation, enhancing user experience by avoiding data loss.
Load Optimization: Utilizing Diff DOM Streaming can significantly improve page load times by streaming and updating only the elements that have changed, rather than reloading the entire page. This avoids reloading a lot of resources.
Smooth Transitions: Integration with the View Transition API enables adding animated transitions between different page states, improving aesthetics and the feeling of fluidity in the application.
Navigation Caching: The implemented caching strategy optimizes user experience by caching previously visited pages, reducing load times on future visits.
Prefetching: Prefetching capability anticipates user actions and prepares corresponding pages in advance, enhancing responsiveness and load speed.

Image from lexica.art

Cons:

Limited Compatibility: The functionality of intercepting navigation with the navigate event may have limited compatibility, as it currently only supports certain browsers. This may limit portability and widespread adoption of the proposed solution.
Limitations in Using Component Libraries: Those familiar with development using component libraries like React may find limitations in not being able to fully leverage the potential of HTML Streaming over the wire. Instead, they may need to resort to other strategies to maintain component state during navigation.

Image from lexica.art

Final conclusions

Concluding this series on HTML Streaming, we've explored the practical application of the Diff DOM Streaming library in web browsing. By integrating this approach, websites utilizing web components can maintain their state during browsing sessions. Throughout this journey, we've delved into detailed implementations using VanillaJS and Bun.

If you want to see the final code that we have been talking about in the article, you can access here:

Code example

As a note, we have disabled the cache and prefetch so you can comfortably test the suspense and see how the animations look when browsing, but feel free to play and test.

Final note:

In Brisa, the framework we are developing, one of the features is that you can write web components as JSX components to make it more comfortable, in fact it is very similar to Brisa server components, the only difference is that instead of web components they are translated to pure HTML, and using signals you can share state between pages to make the pages react. I invite you to subscribe to my blog newsletter to be notified when we release Brisa open-source in a few months and you can try it.

References

diff-dom-streaming library - Library used for implementing HTML Streaming in web browsing.
CustomElementRegistry - Documentation on the CustomElementRegistry interface for registering and defining custom elements for use in a document.
View Transition API - Information about the View Transition API, which provides a mechanism for creating animated transitions between different DOM states.
pushState - Documentation on the pushState method of the History interface, used for adding states to the browser's session history stack.

Top comments (6)

Danny Engelman • Apr 27 • Edited

Maybe a better pattern for the Web Component could be:

jsfiddle.net/WebComponents/47ovs52f/

<my-counter count="42"></my-counter>

<script>
  customElements.define("my-counter", class extends HTMLElement {
    static get observedAttributes() {
      return ["count"]
    }

    constructor() {
      const createElement = (tag, props = {}) => Object.assign(document.createElement(tag), props);
      super()
        .attachShadow({mode:"open"})
        .append(
          createElement("style", {
            innerHTML: `b{padding:0 1em}`
          }),
          createElement("button", {
            innerHTML: "dec",
            onclick: e => this.count--
          }),
          this.counter = createElement("B", {
            innerHTML: "0"
          }),
          createElement("button", {
            innerHTML: "inc",
            onclick: e => this.count++
          })
        )
    }

    get count() {
      return Number(this.counter.textContent);
    }

    set count(newValue) {
      this.setAttribute("count", this.counter.textContent = newValue);
    }

    attributeChangedCallback(name, oldValue, newValue) {
      if (oldValue !== newValue) this.count = newValue;
    }
  });

</script>

```

`

Aral Roca • Apr 27

Thank you for your comment. I haven't really focused much in this article on the details of the web component itself, more on the navigation issue, but thanks for your input. In Brisa, the framework we are developing transforms JSX to web components, a counter would be like this:

src/web-components/custom-counter.tsx

export default function Counter({ }, { state }) {
  const count = state(0);

  return (
    <div>
      <button onClick={() => count.value++}>+</button>
       {count.value}
      <button onClick={() => count.value--}>-</button>
    </div>
  );
}

This is an example without props, but you can use props similar to React. Also is filename-based, so custom-counter.tsx then you can consume in your page with <custom-counter />. However, the Brisa framework is still in a rather experimental phase, we are going to make it public in a few months.

Oliver Williams • Apr 26

I associate HTML streaming with progressive UI updates whereas for a view transition apparently the entire new view should be ready before being rendered. ericportis.com/posts/2023/view-tra...

Aral Roca • Apr 26

Thanks for the comment, I can clarify this in the article, you are right that there may be many cases where you want to make a total transition and not an incremental one. In this case, it can be done as follows:

+ document.startViewTransition(async () => { 
await diff(document, res.body.getReader(), {
  onNextNode: loadScripts,
-  transition: true,
});
+});

It is true that if you don't use suspense and the page is fast, most transitions are better to be complete and not incremental, however, the good thing is that you can choose IMHO :) What do you think?

Ankesh Kapil • Apr 27

Looks like jsf to me. Java was ahead of time 😅

Aral Roca • Apr 27

I don't know if this is good or bad, I have no experience with jsf, could you give me more context? ☺️

DEV Community