2 Birds, 1 Stone – 10 Apps, 1 Node

A Better Way to Node_Modules via Symlinking

Prerequisites:

• A basic understanding of the 'node_modules' directory
• The 'node_modules' problem | Leoat12 | Dev.to

Refresher:

• NPM Folder Structures | NPM Docs
• NPM Packages and Modules | NPM Docs

When learning a new technology, I prefer to make multiple small projects focused on different concepts as opposed to cramming everything into one giga app.

I come from an Angular background where it's possible to create multiple projects using a single, root package.json; a single node_modules folder is shared across all the various projects under that Angular app.

A few days ago, I renewed my efforts to learn React and immediately ran into a major hurdle... the node_modules folder, or rather a growing hydra of node_modules folders.

Every time I created an app with 'create-react-app' (CRA), a new dependency folder appeared.

Why is having multiple node_modules a problem?

For a Hello World React app with only the core dependencies, i.e. react, react-dom, and react-scripts, the node_modules folder is 218 MB on disk. Now the size doesn't sound too bad, but in this directory are 5,300 folders and 35,000 files. WHY?!?! Am I building an OS here?

This exorbitant amount is for the most basic of apps w/ no other dependencies. It climbs even higher on adding other deps, e.g. w/ Tailwind CSS, 363 MB on disk across 6,100 folders!

You want me to copy or redownload 40,200 files spread across 6,100 folders in order to compile and serve EVERY SINGLE Tailwind React app I create?!

I say not! ヽ(ಠ_ಠ)ノ

The Investigation

On closer examination, the source code of this Hello World app is only 18 files across 2 folders for a total of 700 KB. Great, now this I can work with!

That means the size of the app directory is a problem specific to how dependencies are handled in Node.js.

What if we could reuse a node_modules folder across multiple projects?

After some research, I found that when using require(), Node.js will look into the node_modules of parent folders if the module isn't found in the root node_modules of the project. While useful, it doesn't solve the problem of serving the compiled app.

The Solution – Symbolic Links

I looked a little deeper and found my solution at the OS level.

Windows and Linux both have a concept called symbolic links (symlinking), which you can think of as a local reference/pointer to a file or directory elsewhere on the system.

The benefit of symlinking is that the link looks and acts like the real file/directory, but takes up no extra storage! And of course, the symlink is set up instantaneously as there's no copying to be done.

Give you any ideas? Yes, exactly! We can create a single node_modules directory containing the dependencies for all our React apps, and use symlinking inside of each app to reference this 'root' node_modules.

Luckily, it worked just as expected. Now instead of 10 React apps costing my hard drive 4 GB, 61,000 folders, and 402,000 files, I need only 10% of that storage. The savings only grow as I add more apps!

Further, by copying a set of 18 Hello World files, I also save a few minutes on each new React app I create.

Imagine 100 React apps without symlinking... 4 million files! 😬

A Minimal Example

react-playground/
├─ node_modules/            (root)
├─ package.json
│
├─ react-app-1/
│  ├─ package.json
│  └─ *node_modules/        (symlinked)
│
├─ app-group/
│  ├─ react-app-2/
│  │  ├─ package.json
│  │  └─ *node_modules/     (symlinked)
│  └─ react-app-3/
│     ├─ package.json
│     └─ *node_modules/     (symlinked)
│
└─ .gitignore

Keep in Mind:

• All dependency versions should be consistent between projects, e.g. react apps, using a shared node_modules.
• Individual projects can use deps not used in other projects, but these deps will still be installed in the root node_modules.

I've left resources on the implementation of symlinking below. The terminal commands are as follows:

• mklink (Windows)
• ln (Linux)

You can also reference Configuration.md in my React repo for an example implementation.

Ryo112358/react-playground

Now get Symlinking, your computer will thank you!

Until Next Time,
Pulkit

P.S. Shout-out to my girlfriend (@laulina) for the title idea ^.^

References:

1. Ln Command | Linuxize
2. Mklink Command | Microsoft Docs
3. Loading modules from the 'node_modules' folder | Node.js Docs
4. How can I make multiple projects share 'node_modules' directory?

Further Reading:

1. Lodash Library
2. What's really wrong with 'node_modules' | Hacker Noon
3. The Complete Guide to Symbolic Links

Published on Medium

• A Better Way to Node_Modules | CoffeelessProgrammer | Medium