Getting Started with Sematext Browser SDK for Front-end Performance Monitoring

Open-sourcing a code base for the world to see after working on it for a long time is a great experience. You should care about what your users want. You want your users to have a great experience using your product. Everything has to fall into place. Performance, responsiveness, user experience, etc. all have to be exceptional. That’s why I think front-end performance metrics are crucial.

What Metrics Should You Monitor to Improve Front-end Performance?

You want to know your page load times, how fast (or slow!) your HTTP requests are. You want to measure the Apdex score for DOM elements. User-centric performance metrics like largest contentful paint measures the page load performance, first input delay measures its interactivity and the cumulative layout shift gives you information about the visual stability of your web application. You can read more about these metrics in Tips and tricks about how to optimize website performance.

But those are only basic examples.

You can use the Sematext Browser SDK to implement more complex monitoring solutions and collect the exact metrics you want from real users in real browsers in real time. Woah, so many uses of "real" in one sentence! Want another one? What is Real User Monitoring? Another? 5 Best Practices for Real User Monitoring. I could go like this all night.

Front-end Performance Monitoring with Sematext Browser SDK

While working on open-source projects at Sematext we’ve learned how crucial it is to see the codebase. It allows you to understand the solution better and edit it to meet your own needs. It’s also valuable as you get contributors and bug fixes from the community using the product.

That’s why we open-sourced Sematext Browser SDK. It’s the library powering our Sematext Experience data collection. It is now available on Github under the Apache 2.0 license. This means you can get insight into how the SDK collects metrics, how it ships data to Sematext Cloud, our cloud monitoring solution, and even modify the script itself if you wish to do so.

Getting Started

To get started with the Sematext Browser SDK head over to Github and clone it. The project is developed using ECMAScript 2015 and uses various tools, like:

• npm package manager
• yarn package dependency manager
• flow static type checker
• Cypress.io integration tests framework

SDK Customization Examples

Open-sourcing of Sematext Browser SDK brings new possibilities. You can not only see what the script is doing with your website or web application but also modify the experience script behavior.

Some of the example modifications that are possible include, but are not limited to:

• including page loads and HTTP requests happening in the background
• record every page load – even the ones that were caused by the refresh issued by the user when using Ctrl+F5
• avoid sending part of the data – you don’t want to collect info about element timing, web vitals, or memory usage? You can do that!
• change how the URL is parsed and omit parts of it – for example, the query part of the URL

These are just a few examples of what you can do with the Sematext Browser SDK now available as open-source.

Let’s look at one of these customizations. For instance, how tos include page loads and HTTP requests happening in the background as data and send that to Sematext Experience, our real user monitoring tool.

How to Include Page Loads and HTTP Requests Happening in the Background

By default, the Sematext Browser SDK stops listening to page loads, HTTP requests, and element timing metrics when the tab of your web browser is in the background. But you may want to include that. Here’s how you do it.

Start with the index.js file. During the startup of the script you create a so-called DocumentVisibilityObserver which is responsible for informing metrics collection mechanisms whether the content is visible or hidden. The code for this looks as follows:

const visibilityObserver = new DocumentVisibilityObserver();
const pageLoadDispatcher = new PageLoadDispatcher();
const ajaxDispatcher = new AjaxDispatcher();
const elementTimingDispatcher = new ElementTimingDispatcher();

visibilityObserver.addListener(pageLoadDispatcher);
visibilityObserver.addListener(ajaxDispatcher);
visibilityObserver.addListener(elementTimingDispatcher);

PageLoadDispatcher, AjaxDispatcher, and ElementTimingDispatcher are responsible for creating commands and sending metrics. Once you create their instances you add them into the created DocumentVisibilityObserver:

visibilityObserver.addListener(pageLoadDispatcher);
visibilityObserver.addListener(ajaxDispatcher);
visibilityObserver.addListener(elementTimingDispatcher);

If you want to keep measuring page loads, HTTP requests, and element timing metrics when the page is in the background you need to remove the code above. That’s all. You could of course clear up the code and remove the DocumentVisibilityObserver completely, but let’s not complicate our lives and do one change at a time.

How to Build the Experience Script

Once you’re happy with the changes you need to do the following:

• Check if the code passes lint by running yarn lint.
• Check if the code passes static type checking by running yarn flow.
• Check if the integration tests are passing by running yarn e2e.

If everything is OK you can just run:

$ yarn build

The output should be similar to the following one:

➜ browser-sdk git:(master) ✗ yarn build
yarn run v1.22.5
$ webpack --mode production
Hash: 801dbe739470081f7282
Version: webpack 4.44.1
Time: 1885ms
Built at: 10/13/2020 5:05:23 PM
 Asset Size Chunks Chunk Names
 experience.js 135 KiB 0 [emitted] main
experience.js.LICENSE.txt 1.6 KiB [emitted]
Entrypoint main = experience.js
 [0] ./src/common.js 9.38 KiB {0} [built]
 [2] ./src/CommandExecutor.js 2.07 KiB {0} [built]
 [6] ./src/element/utils.js 4.41 KiB {0} [built]
 [8] ./src/constants.js 1.03 KiB {0} [built]
 [34] ./src/index.js 3.6 KiB {0} [built]
 [88] ./src/dispatchers/PageLoadDispatcher.js 5.02 KiB {0} [built]
 [93] ./src/RumUploader.js 9.09 KiB {0} [built]
 [95] ./src/bootstrap.js 3.37 KiB {0} [built]
 [96] ./src/dispatchers/AjaxDispatcher.js 3.6 KiB {0} [built]
 [97] ./src/ajax/xhr.js 1.52 KiB {0} [built]
 [98] ./src/ajax/fetch.js 1.81 KiB {0} [built]
 [99] ./src/DocumentVisibilityObserver.js 2.32 KiB {0} [built]
[100] ./src/dispatchers/ElementTimingDispatcher.js 3.04 KiB {0} [built]
[101] ./src/dispatchers/WebVitalsDispatcher.js 2.43 KiB {0} [built]
[103] ./src/dispatchers/MemoryUsageDispatcher.js 2.91 KiB {0} [built]
 + 90 hidden modules
✨ Done in 3.21s.

That means that the Experience script is now ready to be used and you can find the experience.js file in the dist directory of the project.

Because you have now modified the Sematext Browser SDK behaviour and built your own version, you need to host your custom experience.js file somewhere. You can host it along with your application or a web page or push it to a CDN of some kind – for example, Cloudflare.

Once that is done, instead of adding:

(window,document,"script","//cdn.sematext.com/experience.js","strum");

to the head section in your application or a web page you just need to specify the location of the modified file. For example, if you host the modified experience.js file at awesome.website.com it would be:

(window,document,"script","https://awesome.website.com/experience.js","strum");

Please note that if you are shipping front-end metrics to Sematext Experience with your customized script, you may want to pull the changes from the Sematext Browser SDK repo from time to time and merge them with your cloned repo. That way you will benefit from any improvements, bug fixes, or additional data Sematext started collecting and exposing in Experience.

Next Steps

You can see that modifying the Experience script by working with the Sematext Browser SDK is not complicated and you can adjust it to your needs. We encourage you to try and experiment with the code.

If you seek more detailed information regarding the Experience script we encourage you to head over to the dedicated documentation in the official Sematext docs pages. On Github, you will find the list of supported commands, global constants, and the installation instruction for the Browser SDK.

We are open to questions, suggestions, enhancements, and potential fixes. Feel free to open a Github issue with questions or pull requests with improvements. After all, we made it for you 🙂

Get Started with Experience

You can make your front-end better by using the Sematext Browser SDK and sending performance metrics to Sematext. You get insight into all key front-end metrics and how to improve them. Think of it as Lighthouse on steroids.

To get started head over to Sematext Cloud, sign up, and follow the getting started instructions for Experience. You’ll see front-end performance metrics show up in Sematext in no-time.