Speed up your build by putting node_modules in ram

After a day of diligently copying from Stackoverflow and Google searches, you finally ready to build that 10GB Frankenstein project. You type npm run build into the console then went home and enjoy the rest of your day. The next day you back to the office and surprised to find that the build is still in progress. "What should I do to speed this up?" you ask yourself. And then you came to the right place: This article.

Jokes aside, building JavaScript projects is a time consuming process, although we have more advanced build tools than ever before, it never make sense why your simple website take so much time to be build. Turned out there is a way to make it build faster: by putting the node_modules inside your ram.

Why? you may ask, it requires a bit of knowledge of OS, I'll explain the basics here. When building the project, there are a large amount underlying tasks involved, the main ones are parsing, bundling and minifying which are all I/O related. Since I/O is usually the bottleneck of an operating system, putting files that are frequently read/wrote in an extremely fast I/O device is very helpful to reducing the overall build time. Note we only put node_modules because data in ram is very volatile and maybe lost in case of system crash or power outage. Thus putting the code that you wrote in ram is not recommended.

How? It differs between operating system, the basic idea is to build up a ram disk and then link your node_modules there, here's the detail guide on each system.

Linux

Linux is the best suited for this task because of the system built-in tools.

tmpfs is a fantastic way of mounting your data into ram, simply type

sudo mount -t tmpfs -o size=2G tmpfs node_modules

there are a couple of caveats:

1. if your project is large, increase the size so that node_modules can fit in.
2. if your node_modules locates somewhere else, you need to change the path to the location.
3. files in tmpfs will sometimes get swapped out to disk, so the process may be slow down.

ramfs is preferred if you have a lot of ram space (i.e. 16G or more)

sudo mount -t ramfs ramfs node_modules

couple of things to note:

1. if you are using ramfs and your ram is full, your system will hang
2. you cannot limit the size of it
3. both ramfs andtmpfs behaves different from real ext4 filesystem in some way.

To release the ram space, simply use umount

sudo umount node_modules

macOS

macOS is a tricking one, your shining macbook lacks the tools of linux, so you need to manually manage your ram disk and system link.

First you need to create a ram disk

for macOS <= 10.12

diskutil erasevolume HFS+ “ramdisk” hdiutil attach -nomount ram://4194304

for macOS >= 10.13 and you are using APFS

diskutil partitionDisk $(hdiutil attach -nomount ram://4194304) 1 GPTFormat APFS 'ramdisk' '100%'

check device identifier of your ramdisk:

diskutil info /Volumes/ramdisk 

here we have /dev/disk1s2, yours might be different.

then you can mount node_modules into the ramdisk

diskutil mount -mountPoint node_modules /dev/disk1s2

Alternatively you can use symbolic link

mv node_modules node_modules_files &&
ln -s /Volumes/ramdisk ./node_modules &&
/bin/cp -r node_modules_files/* node_modules

To free up ram space, use this:

diskutil eject /Volumes/ramdisk

Windows

Windows is not suited for this task, install a linux virtual machine for this.

Well, no that's a joke. There's actually a way, it's just more difficult.

First you need to get ImDisk, since windows have no built-in tool to create a ram disk.

Then let's say the ramdisk is R:

you can create a symbolic link by using mklink

mklink /d C:\path\to\node_modules R:\node_modules

To free up ram space just unmount it in ImDisk, also you can save it as an image file for future use.

Use faster build tool?

All these time we've been trying to make our build faster by going one way. Maybe there is another way of achieving the same goal? Turned out there is: If you want absolute build speed, esbuild might be the tool that you are looking for. It's at least 10 times faster than all the current build tools. The main downside of it is that it's brand new and not a lot of people have used it so It's unwise to use it in production. However I believe it's a great choice for hobby/side projects where risk is low. It maybe the ultimate solution for build speed problems.

credits: the idea of this post came from this tweet.