bundlejs: An online esbuild based bundler & npm package size checker

• Introduction
  • Quick feature run down
• Bundling, Treeshaking, and Minification
• Console
• Input and Output Tabs
• Configuration
  • CDN Hosts
  • Compression Algorithms
  • The Brotli Problem
  • Default to Gzip
  • LZ4, Gotta Go Fast
  • Compression Quality
  • Aliases and Externals
  • Esbuild Config Options
• Editor Buttons + Extra Features...
• Drag Handles...Interactive Fun
• JSX Support
• Sharing Bundle Sessions
• Bundle Analysis
• Analytics
• Discussions and Support
• Security and Performance
• Tips and Tricks
• Contribute
• Conclusion

Introduction

bundlejs (pronounced bundle js) is a quick and easy way to treeshake, bundle, minify, and compress (in either gzip or brotli) your typescript, javascript, jsx and npm projects, while receiving the total bundles' file size.

bundlejs aims to generate more accurate bundle size estimates by following the same approach that bundlers use:

• Doing all bundling locally
• Outputing the treeshaken bundled code
• Getting the resulting bundle size

The benefits of using bundlejs are:

1. It's easier to debug errors
2. You can verify the resulting bundled code
3. The ability to configure your bundles
4. The ability to treeshake bundles
5. The ability view a visual analysis of bundles
6. You can bundle offline (so long as the module has been used before)
7. Supports different types of modules from varying Content Delivery Networks (CDNs), e.g. CDNs ranging from deno modules, to npm modules, to random github scripts, etc...

This blog post is meant to highlight some of the most important changes as well as to give some insight into how bundlejs works in the background, and to act as the docs for bundlejs.

📒Note: There will be a follow up article to this one, going into the technical nitty gritty on how bundlejs works and how you can use what I've learned from this project to either create your own online bundler or an es build-wasm backed js repl.

😅 TL;DR: this blog post is rather long, so take a look at the bundlejs.com website first, then skim through this blog post making sure to check out the images and the code examples, those can help cut down on confusion and reduce the required reading time.

Quick feature run down

This video runs through all the major features of bundlejs (there is audio but I don't have a good mic 😅)

Bundling, Treeshaking, and Minification

• Bundling is the process of efficiently concatenating modules together into one file which we call a bundle.
• Treeshaking is the process of a bundler traversing the modules to be bundled and removing unused code.
• Minification is the process of shrinking the amount of code necessary to have a functional program, e.g. removing blank space or reducing variable names, etc...

bundlejs uses esbuild and it's incredible ability to bundle, transform, transpile, minify, treeshake and traverse files. Specifically, bundlejs uses esbuild-wasm which is able to access a subset of those features, with the key limitations being,

1. npm only runs on node, so no package.json or npm install (a-la, a joke about using StackBlitz WebContainers to run node on the browser)
2. browsers don't work the way nodejs does. They don't have a easy way to access file system, so storing and accessing files isn't practical. The way esbuild would normally work when installed on node just causes issues on the web
3. due to the limitation of esbuild-wasm when running on a browser (no npm, and node nodejs), the only option is for modules to come from the web but esbuild doesn't natively support importing http(s)://... modules, so a different solution is required

To solve each of these problems esbuild's plugin system comes in clutch. I created a total of 4 plugins to solve these limitations, they are,

1. HTTP plugin - Fetches and caches modules
2. CDN plugin - Redirects npm package imports (sometimes referred to as bare imports) to Content Delivery Network (CDN) urls for fetching
3. EXTERNALS plugin - Marks certain imports/exports as modules to exclude from bundling
4. ALIAS plugin - Aliases certain imports/exports to modules of a different name

Content Delivery Networks (CDNs) are a great way to distribute code all over the world at fast speeds. In the context of bundlejs, CDNs represent online repositories of code that bundlejs can fetch from.

For example, unpkg.com is a fast global content delivery network for everything on npm. It's used to quickly and easily load any file from any package on npm using a URL like: https://unpkg.com/package-name@version/file.js, a similar thing would apply for skypack.dev, esm.sh, etc...

In a later blog post I will delve deeper into the technical details of how these plugins work, but for now just keep in mind that these plugins assist esbuild-wasm to create javascript bundles.

Console

In previous versions of bundlejs.com I encouraged devs to use the devtools console for viewing console logs, and for a while I thought it was an ok experience, but I started realizing that it was inconvenient and not very mobile friendly. Initially, I thought creating a virtual console would be a large undertaking, so I delayed adding a custom console for quite some time. Well in the March of this year inspired by @hyrious's esbuild-repl I finally did it 👏.

Input and Output Tabs

With the addition of the console I wanted to ensure that the editors weren't unwieldy, so I created a tab bar for the input, output and config editor. The tab bar allows for quick access to all editors while ensuring that the console is always available.

Configuration

In v0.2 I added support for custom configurations (configs), it supports most of esbuild's build options, as well as some added options to change the default CDN and the compression algorithm.

The default config is

{
  "cdn": "https://unpkg.com",
  "compression": "gzip",
  "esbuild": {
      "target": [ "esnext"],
      "format": "esm",
      "bundle": true,
      "minify": true,
      "treeShaking": true,
      "platform": "browser"
  }
}

When you click the share button, it will also share the custom config you've setup, e.g.

{
  "cdn": "https://unpkg.com",
  "compression": "lz4",
  "esbuild": {
      "target": [ "es2018" ],
      ...
  }
}

The config above would result in this share URL bundlejs.com/?q=@okikio/animate&config={"compression":"lz4","esbuild":{"target":["es2018"]}}.

Notice how cdn is missing from the share URL, that's because bundlejs smartly decides on which config to send as a part of the share URL based on how different the new config is from the default config.

📒 Note: There are 3 available compression algorithms, brotli, gzip, and lz4.

CDN Hosts

Content Delivery Networks (CDNs) are a great way to distribute code all over the world at fast speeds. In the context of bundlejs, CDNs represent online repositories of code that bundlejs can fetch from.

For example, unpkg.com is a fast global content delivery network for everything on npm. It's used to quickly and easily load any file from any package on npm using a URL like: https://unpkg.com/package-name@version/file.js, a similar thing would apply for skypack.dev, esm.sh, etc...

By default bundlejs lets you enter code like this,

export * from "@okikio/animate";

But behind the scenes bundlejs auto fetches that specific package from a CDN namely, unpkg.

In older versions of bundlejs the default CDN used to be skypack but because skypack doesn't have easy access to the package.json of node packages, I switched to using unpkg as the default CDN.

With later updates bundlejs recieved the ability to update the default cdn on a global or local scale.

*   `skypack:react` -> [https://cdn.skypack.dev/react](https://cdn.skypack.dev/react)

*   `unpkg:react` -> [https://unpkg.com/react](https://unpkg.com/react)

*   `esm.sh:react` or `esm:react` -> [https://cdn.esm.sh/react](https://cdn.esm.sh/react)

*   `deno:preact` -> [https://deno.land/x/preact](https://deno.land/x/preact)

*   `esm.run:react` -> [https://esm.run/react](https://esm.run/react)

*   `github:facebook/react/main/packages/react/index.js` -> [https://raw.githubusercontent.com/facebook/react/main/packages/react/index.js](https://raw.githubusercontent.com/facebook/react/main/packages/react/index.js)

*   `jsdelivr:react` -> [https://cdn.jsdelivr.net/npm/react](https://cdn.jsdelivr.net/npm/react)

*   `jsdelivr.gh:facebook/react/packages/react-dom/index.js` -> [https://cdn.jsdelivr.net/gh/facebook/react/packages/react-dom/index.js](https://cdn.jsdelivr.net/gh/facebook/react/packages/react-dom/index.js)

Compression Algorithms

bundlejs offers the options of bundling using:

1. brotli - results in the smallest bundle size but it's the slowest
2. gzip - results in the 2nd smallest bundle size but it's faster than brotli (default)
3. lz4 - results in the largest bundle size but it's the fastest bundle algorithm

📒 Note: Each compression algorithm has it's own story.

The Brotli Problem

brotli is a compression algorithm that compresses data really well, however, it's very slow compared to other alternatives. Adding brotli was quite an undertaking with lots of ups and downs, but thanks to Lewis Liu on Twitter I was able to use deno-brotli to include a WASM version of brotli in bundlejs.

deno-brotli does 2 things right, they are,

1. It compresses the huge WASM file required for deno-brotli into an lz4 compressed string, which can then be decompressed by lz4 allowing for easy storage of the WASM as a js file (the result is great build tools support as the WASM is just a string inside a JS file, plus it solves the ecosystem problem really well).

   For lz4 support bundlejs is using deno-lz4, which also runs via WASM, but the way deno-lz4 compress itself is slightly different than deno-brotli. I'd highly encourage you to take a look at the source code for deno-lz4 it's really informative.

2. By having the WASM as a js file you can actually preload the WASM as a js module 🤯

   

You can read through this tweet thread to learn more,

Default to Gzip

bundlejs has used gzip as the default for quite sometime, bundlejs used to use pako, but thanks to a discussion with @matthewcp who rightfully pointed out the (De)compression Stream API, I started looking into alternative to pako.

I eventually chose to replace pako with denoflate as the default compression algorithm for gzip, it's a bit faster and smaller than pako.

LZ4, Gotta Go Fast

In order to use WASM in a portable way, deno-brotli would compress the WASM binary file into a base64 string using lz4 as the compression algorithm. I saw the oppertunity to add another compression algorithm, so I used the lz4 (deno-lz4) implementation used to compress the brotli WASM binary string (see #the-brotli-problem for more info.), it was by far the easiest compression algorithm to add to bundlejs 🤣.

Compression Quality

You can set the quality of the compression (from a scale of 1 to 11, with 1 being the least compressed and 11 being the most compressed), you can set the compression quality for any of the compression algorithms mentioned above by setting the compression config option to,

{
    "compression": {
        "type": "brotli",
        "quality": 11
    }
}

You can check out a demo here, bundlejs.com/?config={"compression":{"type":"brotli","quality":11}}.

Aliases and Externals

Aliases are a way to redirect certain packages to other packages, e.g. redirecting the fs to memfs, because fs isn't supported on the web, etc... This wasn't a direct feature request but I felt it would be a good addition.

Externals are a direct feature request issue#13, it took a while but a good solution is finally a part of bundlejs, you use it the way you'd use the esbuild externals config option.

{
    "alias": {
        "@okikio/animate": "@babel/core"
    }
}

{
    "esbuild": {
        "external": ["@okikio/animate"]
    }
}

No one else can understand my pain...I'm adding more feature to bundlejs as I'm writing this blog post, so it's just getting longer and longer and longer, etc.... 😅

Esbuild Config Options

esbuild config options are exactly how they sound, however, with esbuild running on the browser there are some limitations on what esbuild can do, due to the lack of native filesystem access some options don't work or are rendered obsolete.

The supported esbuild build options are

Quite a bit to work with I'd say.

Editor Buttons + Extra Features...

The editor buttons add extra functionality to the editor, they enable easy access to common editor tasks.

The current list of editor buttons are:

Drag Handles...Interactive Fun

The new drag handles enable a more interactive experience with the editor, they are a little like the drag handles in vscode, but mobile friendly.

bundlejs being mobile friendly isn't a huge focus point for the project, but it's nice to have if you ever find yourself in the need for bundlejs while on a mobile or touch enabled device.

JSX Support

JSX is now officially supported in bundlejs 🎉.

Ignore the red error lines, for some reason the monaco code editor doesn't want to work well with JSX 😅

To use JSX you need to set the jsxFactory and the jsxFragment config options according to the JSX based framework you are using.

e.g. for Preact the following config would be used:

{
    "esbuild": {
        "jsxFactory": "h",
        "jsxFragment": "Fragment"
    }
}

Try out the preact demo

Sharing Bundle Sessions

To share bundle sessions* between multiple users (while avoiding the need for a server) we need a static and local way to store and share code between users. To solve this problem I decided to encode the bundle session* information right into the URL, this was because I wanted the entire project to run offline, and I didn't want the high maintenance cost of a server and database.

*sessions are the specific state of bundlejs at a specific time, they are not the entire bundle session history, just the input code and the bundle configuration at the time the share button is clicked.

/?q=(import)@okikio/emitter,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate&treeshake=[T],[{ animate }],[{ animate as B }],[ as TR],[{ type animate }],[],[{ animate as A }],[ as PR],[{ animate }]&text="export  as PR18 from \"@okikio/animate\";\nexport { animate as animate2 } from \"@okikio/animate\";"&share=MYewdgziA2CmB00QHMAUAiAwiG6CUQA&config={"cdn":"skypack","compression":"brotli","esbuild":{"format":"cjs","minify":false,"treeShaking":false}}&bundle

// Click Build for the Bundled, Minified & Compressed package size
import T from "@okikio/emitter";
import { animate } from "@okikio/animate";
import { animate as B } from "@okikio/animate";
import  as TR from "@okikio/animate";
import { type animate } from "@okikio/animate";
export  from "@okikio/animate";
export { animate as A } from "@okikio/animate";
export  as PR from "@okikio/animate";
export { animate } from "@okikio/animate";
console.log("Cool")
export  as PR18 from "@okikio/animate";
export { animate as animate2 } from "@okikio/animate";

{
    "cdn": "skypack",
    "compression": "brotli",
    "esbuild": {
        "target": ["esnext"],
        "format": "cjs",
        "bundle": true,
        "minify": false,
        "treeShaking": false,
        "platform": "browser"
    }
}

The URL breakdown is,

/?
q=(import)@okikio/emitter,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,(import)@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate,@okikio/animate&
treeshake=[T],[{ animate }],[{ animate as B }],[* as TR],[{ type animate }],[*],[{ animate as A }],[* as PR],[{ animate }]&
text="export * as PR18 from \"@okikio/animate\";\nexport { animate as animate2 } from \"@okikio/animate\";"&
share=MYewdgziA2CmB00QHMAUAiAwiG6CUQA&
config={"cdn":"skypack","compression":"brotli","esbuild":{"format":"cjs","minify":false,"treeShaking":false}}&
bundle

• q or query represents the module, e.g. react, vue, etc...

  You can add (import) in-front of a specific module to make it an import instead of an export

• treeshake represents the export/imports to treeshake.

  The treeshake syntax allows for specifying multiple exports per package, through this syntax
  

  "[{ x,y,z }],[*],[* as X],[{ type xyz }]" 
  // to
  export { x, y, z } from "...";
  export * from "...";
  export * as X from "...";
  export { type xyz } from "...";
  

  The square brackets represent seperate packages, and everything inside the squarebrackets, are the exported methods, types, etc...

• text represents the input code as a string (it's used for short input code)

• share represents compressed string version of the input code (it's used for large input code)

• config represents the bundle configuration to use when building the bundle

• bundle tells bundlejs to bundle the input code on start-up. This isn't on by default for security reasons. I want to discourage people from sending large complex bundles that crash browsers or that take a long time to load, especially before the input code is properly verified as non-malicious. So, if you want to bundle the code on startup, you have to manually add &bundle to the end of the url yourself.

The reason why I decided on this syntax is because it allows for a lot of flexibility, and transparency concerning what is being bundled. I also wanted to make it easy to share bundle session between users.

Bundle Analysis

bundlejs can analyze and visually represent bundles as easy to navigate and easy to understand charts.

Using a port of esbuild-visualizer and rollup-plugin-visualizer by @bardadymchik I added the ability to visualize bundles, this feature comes from a feature request by @atomiks on issue#22, the issue is still open you can make suggestions to improve this feature.

The bundle analysis charts are displayed right under the editor, like so,

The charts displayed comes in 3 distinct flavours:

Analytics

When I initially built the project I only used a simple page view counter, I wanted to view how popular the project was without violating user privacy, it worked but I felt it could be better, so I decided to also use umami as a privacy preserving, cookieless, open source, Google Analytics alternative, to which the analytics are public for anyone to view.

Setting up Umami with Vercel and Supabase

Jakob Bouchard ・ Feb 2 '21

#analytics #firstpost #webdev #tutorial

Discussions and Support

To encourage discussion, give support and to gain feedback, I added a comment section to bundlejs, I used giscus for this.

Initialy, when I created the bundlejs project I also created a GitHub Discussion for it as well. I didn't want to have the overhead of having to manage a Discord server, so I choose GitHub Discussions for chats about bundlejs. The problem is that no one really uses GitHub Discussions, so I to integrated it right into the website itself via giscus, this was so new users can easily interact with others, get support from me, and leave me feedback.

As of right now the comments section is looking really bare and basic, why not leave your mark. Leave a comment with what you love and what you think needs improvement in bundlejs, I'll go through them and try to integrate your ideas into bundlejs.

Security and Performance

Security and performance are critical quality areas for bundlejs. In order to bundle modules together, bundlejs has to fetch multiple sets of modules from all over the internet, while ensuring that malicious actors don't get involved, and that esbuild-wasm isn't taken advantage of to crash other devices.

Some really...really large modules can take up to 4+ GB of memory to be bundled properly by esbuild-wasm.

The security criterias I set for bundlejs were:

1. Don't leak personal user info.
2. Don't go through a central server, e.g. the ability to get node modules from various sources
3. Ensure people always know what packages they are bundling
4. Ensure people can't use bundlejs to maliciously slowdown browsers

To ensure I met the security criterias set,

1. I use strict Content Security Policies (CSP) for bundlejs, ensuring no unintended 3rd party can get involved.

2. I self-host as many of the 3rd party scripts I can. By enclosing the number of hands involved in bundlejs.com I reduce the chance that personal information is leaked.

3. I use sandboxing techniques e.g. Web Workers and Shared Workers to ensure the main thread runs at a smooth 60fps while avoiding access to the DOM.

   Web Workers and Shared Workers are scripts that run on a seperate thread, by using Workers I am able to isolate potentially malicious code while ensuring that the main-thread isn't affected.

   Most of the uses of Workers were Shared Workers. Shared Workers reduce the performance impact of multiple instances of the bundlejs site/web app running on the same device.

4. To reduce the chance of bundlejs being used to maliciously slowdown browsers, I ensure the share URL is easy to read and understand, and by default disable auto-bundling for shared URLs.

The current browser landscape for Shared Worker support is spotty at best.

The support table looks like this,

* Support for Safari is currently experimental, but should be coming in later versions

Module Workers are esmodules that run in Workers.

📒 Note: I built a Shared Worker polyfill @okikio/sharedworker. It's a small but simple polyfill that falls back to a regular Web Worker if Shared Workers are not supported. You can use it while waiting for the next version of Safari to support Shared Workers, or while supporting older versions of Safari.

⚠️ Warning: The Shared Worker polyfill doesn't handle module workers, you will still need to somehow compile your modules to non-esm versions to support workers in Firefox. You can view how bundlejs handles module workers in the bundlejs source code, you may also wish to view the astro-repl source code to see how it handles module workers.

Most of the other security policies are passive in nature, e.g.

• bundlejs only bundling on page load if the URL has ?bundle in it.
• bundlejs enforcing https:// for all requests, including for iframes, etc...
• Only have properly vetted CDN hosts for bundlejs by default.
• etc...

Tips and Tricks

Top tier tip, follow me (@okikio_dev) and bundlejs (@jsbundle) on twitter; shameless plug 🤣.

I do post announcments and updates on these accounts, as well as small tips and tricks that help in making the most use of bundlejs.

• When bundling packages that also export CSS and other external files, bundlejs.com now checks the gzip/brotli size of these external files, however, it won't output the external files' code, this behaviour may change in the future but for now that is the approach I am going with. Keep this in mind this isn't a bug, however, if it causes confusion I am willing to change this behaviour.

• Treeshaking is available, but not all CDNs support access to each packages package.json so there might be slight package version conflicts. The only verified CDN with access to the package.json is https://unpkg.com. The other CDN's that are used either pre-bundle the code for us (this is hit or miss depending on the package) or they aren't full npm CDN's e.g. https://deno.land or https://raw.githubusercontent.com.

• Check the full devtools console for error messages and warnings, if you are having trouble debuging an issue in bundlejs, or even better yet enable esbuilds verbose logging when trying to debug issues, e.g.
  

  {
      "esbuild": {
          "logLevel": "verbose"
      }
  }
  

  Check out a demo.

• You can use custom protocols to specify which CDN's specific imports/exports module should use. If an error occurs such that you can't bundle a package properly, I highly suggest switching CDN's via either custom protocols or by changing the cdn config option. I recommend using custom protocols instead of the cdn config option when trying to debug issues with a CDN:

  e.g.
  

  {
      "cdn": "unpkg"
  }
  

  or
  

  export * from "unpkg:typescript";
  

  Try using custom protocols to solve this example issue on bundlejs.

• For some packages a soft error occurs where the default export is excluded from the treeshaken bundle, the solution for this is to manually include the default export like so,
  

  export * from "skypack:solid-dismiss";
  // and
  export { default } from "skypack:solid-dismiss";
  

If you have a tip and trick you would like to share, post a comment below, or send me a tweet!

Contribute

The codebase is currently quite disorganized so, I'd suggest direct messaging me on Twitter or starting a GitHub Discussion to discuss ways to contribute.

There is a lot of stuff happening on the bundlejs project and it can be very overwhelming, if you think you can still contribute by all means please do! I will eventually get to writing detailed docs, on how to contribute, and how everything works in the backend, look forward to it.

You can use a pre-made Gitpod dev environment to quickly get started with the project or to contribute quick changes to the project.

If you love the project, I'd welcome if you'd spread the word, my goal is to make bundlejs a viable alternative/replacement for bundlephobia and even local bundlers, but right now the project is so small that most people who'd benefit from it don't know about it. I'd love to see people using it.

Last note, bundlejs is now on OpenCollective, so if you'd like to contribute to it financially, it'd be appreciated.

Conclusion

bundlejs, a quick and easy way to treeshake, bundle, minify, and compress (in either gzip or brotli) your typescript, javascript, jsx and npm projects, while receiving the total bundles' file size.

bundlejs aims to generate more accurate bundle size estimates by following the same approach that bundlers use:

• Doing all bundling locally
• Outputing the treeshaken bundled code
• Getting the resulting bundle size

The benefits of using bundlejs are:

1. It's easier to debug errors
2. You can verify the resulting bundled code
3. The ability to configure your bundles
4. The ability to treeshake bundles
5. The ability to view a visual analysis of bundles
6. You can bundle offline (so long as the module has been used before)
7. Supports different types of modules from varying Content Delivery Networks (CDNs), e.g. CDNs ranging from deno modules, to npm modules, to random github scripts, etc...

The next time you need to bundle a project or you need to know the bundle size of a project, give bundlejs.com a try.

📒Note: There will be a follow up article to this one, going into the technical nitty gritty on how bundlejs works and how you can use what I've learned from this project to either create your own online bundler or an esbuild-wasm backed js repl.

Photo by Okiki Ojo, you can find the image on Dropbox.

Originally published on blog.okikio.dev

Also, published on Hackernoon and dev.to