Cleaning & speeding up the JS ecosystem - Journey so far

#javascript #e18e #typescript #npm

For some time now, I have been hard at work at increasing performance across the ecosystem. Usually through cleaning up aged dependency trees, reducing install footprints, and improving CPU/memory performance of commonly used dependencies.

This blog post is just my brief attempt at explaining some of the journey that lead to e18e and the ecosystem cleanup.

Thoughts on micro-utilities

Micro-utilities are a major contributor to the install size and, more, the dependency tree complexity of many projects.

Packages such as is-number and is-nan fit into this category.

Importantly, many of these packages do have a use in some situations but certainly not in the common use case.

Usually they can be replaced in these situations:

They were written in a time when equivalent native functionality didn't exist, but now does
They do more than what the consumer needs

Example: is-number

For example, is-number determines if a value is a number or a number-like string which is not NaN or +/-Infinity.

In some projects, this is a repeated piece of validation which may be better extracted into its own module or package.

However, in the common case, many consumers never needed the NaN and Infinity validation, or even to support number-like strings.

This opens up some possible improvements in various projects. We can replace the use of this package in those projects with much simpler inline logic (e.g. in many real-world projects, we safely switched to typeof n === 'number' since those projects later assumed and relied on the value being an actual number anyway).

Another example: is-regexp

You can test that something is a regular expression using instanceof: v instanceof RegExp.

However, in some situations (e.g. using virtual contexts), this won't work since RegExp isn't the same class which v originated from. In those cases, we need to use something like this:

Object.prototype.toString.call(obj) === '[object RegExp]'

If you don't want to inline such code, and need to support virtual contexts or similar, maybe a library makes sense.

However, in many cases, the projects did not support virtual contexts anyway (and did not want to). That opens us up to opportunities to simplify again to a simple instanceof.

Supporting older runtimes

Another potential area of improvement is that of older runtime support.

Quite a lot of very popular packages have very deep dependency trees of various polyfill-like modules. These generally exist for one or both of the following reasons:

To protect against global namespace tampering
To maintain support in runtimes which lack this functionality

Many of us do not need this level of backwards compatibility and could get large performance gains if we could trim it away.

It is important to note, of course, there are still people who do need to work in those constraints. This is why in this area, we often have been providing forks or alternatives rather than trying to change the existing packages (which wouldn't be open to such changes anyway).

Starting to improve things

I started to think about these particular areas back in 2018 or so, after seeing how large and deeply nested my node_modules were for even the smallest projects.

My first few attempts at change were to create some kind of ESLint plugin which can detect these packages and suggest they be removed. Every several months, I would have this same idea and try again but never really got where I wanted to be.

Through this time, I was at least contributing to various large projects to clean up and improve what I could (e.g. one i've contributed to for a long time is storybook).

Ecosystem cleanup

I then created the ecosystem cleanup, a repository for raising possible performance improvements across the ecosystem (basically an issue tracker). Again, this was basically me and my own personal issue tracker for some time, but at least it was visible in the open.

Soon after, I started to see people turning up in the issues and contributing to upstream projects. I was so happy to see this as I'd spent many years chipping away at this by myself, and wondering if I'm even making a difference. To see others joining in helped so much in knowing someone else cares.

Module replacements

While the cleanup project was and still is incredibly useful, we didn't really have anywhere to share with the rest of the community what good alternatives exist.

To solve this, I created the module replacements project.

This project basically holds some JSON lists of modules which can possibly be replaced and their suggested alternatives. These are currently split into three levels of "strictness" or "opinionatedness": native (modules which can be replaced with native functionality), micro-utilities (modules which may be replaceable by simple inline code), and preferred (modules which we think should be replaced with more performant alternatives).

Codemods

The next step of the replacements project was to create a set of codemods so we can automate the replacement of some of these modules.

Driven by @passle, this project quickly received a huge amount of contributions in the form of various codemods.

The codemod team have also done some good work porting these to the codemod platform. In future, we also want to provide them through some kind of CLI or auto fix rules.

e18e

The turning point where I feel all of us who care about this stuff found each other was e18e.

Bjorn was doing some great work improving the performance of astro, and Marvin had been similarly writing about speeding up the ecosystem. Eventually, our paths crossed and we had some great discussions on the side.

A small group of us worked together to see if we're all on the same page and if there's a community to be built out of this. Then came e18e!

Built to be a community space for people to collaborate on improving ecosystem performance, this has shown us how many people care about these things. So many have joined and have contributed massive amounts already. We're seeing speed ups and size reductions almost every day across the ecosystem.

Some thanks

The community is growing quickly and has too many people to thank for the contributions. However, I'd like to thank these people in particular for helping make this community possible:

Similarly, the people who were already working on projects contributing to the same goals in parallel: