A quick breakdown of the most useful JavaScript modules that I find myself using over and over again.
This is an opinionated article that focuses on general-purpose modules and utilities that Iâve found invaluable to Node.js and frontend JavaScript development. It wonât be exhaustive or include any special-purpose modules, as those types of awesome lists are indeed awesome but tend to be a bit overwhelming.
Command Line Tools
Letâs get started with some extremely useful command line tools.
np - A better
npm publish
.
If youâre an npm author, Iâd highly recommend checking out np, as it makes the process of bumping versions, adding git release tags, and publishing to npm a breeze, especially once you start having more than a couple of modules to maintain. Itâs also worth noting release by Zeit as a solid alternative.
yarn - An alternative package manager to
npm
.
Thereâs no clear winner between npm
and yarn
. The recent drama over yarn
v2 is probably enough to push many yarn
devs back to using npm
.
As a JS developer in 2020 just make sure that youâre at least familiar with both npm
and yarn
and be comfortable switching off between them.
prettier - An opinionated code formatter.
Prettier enforces a consistent style by parsing your code and re-printing it with its own rules that take the maximum line length into account, wrapping code when necessary.
I love eslint and have been a long-time user of JavaScript Standard Style in particular, but the idea behind automatic code formatters like prettier and gofmt is undeniably attractive.
As developers, we spend way too much time and mental energy worrying about code presentation and styling, whereas prettier alleviates the need for those thought processes and allows you to focus on what youâre writing instead of how youâre writing it.
now - Extremely simple deployments.
Now is absolutely the best free deployment system that exists today in terms of simplicity, reliability, and feature set. Itâs great for testing static and dynamic deployments and scales up nicely if and when you require more servers. Aaaaaaaaaand did I mention that itâs free until you want to scale up?!
It plays extremely well with Node.js and JS-powered webapps. Iâd also highly recommend checking out the rest of Zeitâs offerings as well, as their team is comprised by some of the best JS devs the community has to offer.
asciinema - Free tool for recording high quality terminal sessions.
See my previous blog post âMaking your Code Beautifulâ for a breakdown of how you can take advantage of asciinema to produce quality code demos & screencasts like the pros.
Promises
This section really deserves an entire article unto itself, especially now that async & await have started becoming the de facto standard paradigm for concurrent programming in JavaScript. With that being said, Iâd highly recommend checking out Sindre Sorhusâ excellent promise-fun module collection if you havenât already. My only gripe with these modules is that they likely wonât work out-of-the-box with most frontend toolchains like create-react-app or rollup.
Here are a few of the most useful gems which stick out for working with promises and async-style code in Node:
pify - Promisify a callback-style function.
There are a lot of ways to convert functions from old-school callback-style to Promise-style, but Iâve found pify to be the best. Itâs tiny and has some niceties like automatic method binding that the built-in util.promisify is lacking.
p-map - Map over promises concurrently.
Concurrency is great, but most of the time you want to set a practical limit on parallelism whether it be throttling network bandwidth or compute resources. This is where p-map really shines. I use it 99% of the time as a drop-in replacement for Promise.all(âŠ)
, which doesnât support limiting parallelism.
Before I was aware of p-map, I created my own version async-await-parallel, but you should use p-map as itâs better. đ
p-retry - Retry a promise-returning or async function.
I generally wrap any HTTP requests and external service calls with p-retry to add a basic level of robustness to them. Combined with p-map, you can process large batches of external requests with controlled parallelism without having to worry too much about an occasional transport error, socket hang-up, or server timeout.
p-timeout - Timeout a promise after a specified amount of time.
Alongside p-retry, p-timeout is a must for robustly working with third-party APIs and services. You can also specify an optional fallback, as oftentimes returning something is better than hanging indefinitely or returning after an inordinate amount of time.
p-cache or p-memoize - Memoize the results of an async function via an LRU cache.
The aim of many of these Promise utilities reminds me a lot of architecting robust microservices, where every external dependency can be treated with a common interface supporting retrys, timeouts, caching, circuit breakers, fallbacks, etc.
Graceful degradation of functionality is generally preferable to overwhelming the system or not responding at all, so if youâre not too familiar with microservices, check them out and see if their design decisions can help improve your Promise handling abilities as well.
Scraping
There are a lot of great scraping utilities out there, some of which operate on raw HTML like cheerio, and some of which simulate a full browser environment like puppeteer. What you decide to use really depends on your use case, as working with raw HTML is a lot quicker and more lightweight, whereas automating a headless browser is more robust at the cost of being heavier to get started.
cheerio - Fast, flexible, and lean implementation of core
jQuery
designed specifically for the server.
Cheerio is really great for quick & dirty web scraping where you just want to operate against raw HTML. It provides a robust jQuery-like syntax for traversing & manipulating HTML documents. Cheerio pairs particularly well with request-promise-native below for fetching remote HTML documents.
puppeteer - Headless Chrome Node API
Unlike cheerio, puppeteer is a wrapper for automating headless chrome instances, which is really useful for working with modern JS-powered SPAs. Since youâre working with Chrome itself, it also has best-in-class support for parsing / rendering / scripting conformance. Headless Chrome is still relatively new, but it will likely phase out older approaches such as PhantomJS in the years to come.
If you need to faithfully scrape websites, automate web-based workflows, or capture screenshots, puppeteer is a clear winner that will only become more popular with time.
Node.js
dotenv-safe - Load environment variables from
.env
and ensure theyâre all present.
This module extends the hugely popular dotenv module to enforce the existence of expected environment variables via a .env.example
file. Like the original, it provides fast, secure, and robust environment variable support for Node.
It also plays well with Zeitâs now.sh deployments with the âdotenvâ: true
option set in now.json.
request and request-promise-native - Simplified HTTP request client.
Making HTTP requests is an extremely common operation, and my goto module here is request-promise-native which wraps the original request module with native ES6 promise support. 95% of the time I want to await the result of a promisified HTTP request. The other 5% of the time I want to work with the response stream directly, in which case I use the underlying request module, foregoing Promise support.
For robustness, I will oftentimes wrap request-promise-native calls in some combination of p-retry, p-timeout, and p-cache.
Itâs also worth mentioning got as a newer alternative to request with promise support baked in, although I havenât used it much personally.
Example of downloading an HTML document with request-promise-native.
consolidate - Template engine consolidation library for Node.
Consolidate is great for handling any type of backend templating (emails, tweets, raw html, etc.). I generally use handlebars as my templating engine of choice, but no matter what, I always wrap my template usage in consolidate, because it provides a simple & consistent interface to templating regardless of the template engine you decide to use under the hood.
For example, I used consolidate in create-react-library to render the boilerplateâs templates with library-specific variables.
execa - A better
child_process
.
Extremely useful if you need to run a shell command or spawn a child process in general.
fs-extra - A better
fs
with additional methods and Promise support.
Itâs rare that I find myself using fs
directly anymore. Try fs-extra
and you wonât look back.
Math
D3 (Data-Driven Documents) is a popular frontend library for data visualization and animation. It also contains some of the best standalone packages for common math operations that I find myself consistently choosing over alternative modules.
d3-randomâââGenerate random numbers from various distributions.
When Math.random
doesnât cut it, give d3-random a try. It supports sampling from different common distributions, including uniform, normal, and exponential.
d3-ease - Easing functions for smooth animations.
d3-interpolate - Interpolate numbers, colors, strings, arrays, objects, whatever!
This module provides a variety of interpolation methods for blending between two arbitrary values. Values may be numbers, colors, strings, arrays, or even deeply-nested objects.
Testing
ava - Awesome JS test runner.
Itâs not surprising that my go-to for unit test runner for Node.js is yet another tool created by Sindre Sorhus. Ava is a newer unit test runner that takes a lot of what was good about mocha, tape, chai and other JS test runners, and bundles it all together into a quality project with sensible defaults that âjust worksâ.
Itâs worth noting that Avaâs tests are run in parallel by default, which you can disable at the file-level for use cases like database testing where the order your unit tests run may be important.
nock - HTTP mocking and expectations library.
Nock is great for testing modules that perform HTTP requests in isolation. If your Node module makes HTTP requests and you want to provide proper unit testing, then nock is the way to go.
sinon - Function spies, stubs and mocks for JS testing.
Sinon is a very useful utility library for writing isolated tests by taking advantage of dependency injection. It should be a part of every Node developerâs tool belt.
Wrapping Up
I hope youâve found this breakdown helpful, even if itâs just learning about one quality module that you werenât aware of before. I know a lot of aspiring and seasoned developers alike who end up rolling their own solutions to common problems, which can be a useful practice in & of itself, but itâs also good to know when there are quality, existing solutions you should be using instead of constantly reinventing the wheel.
The size & scope of NPMâs module library is unprecedented, and imho itâs one of JavaScriptâs biggest advantages compared with other programming languages. The better you become at taking advantage of npm modules, the faster and more productive youâll be as a developer. Higher-order âsoftâ skills like this are one of the hallmarks of becoming a mythical 10x programmer.
Have any favorite npm modules I left out? Let me know by sharing your favorite modules in the comments! â€ïž
Before you goâŠ
If you liked this article, please leave leave a â€
Top comments (8)
đŻ
Nice write up. Thank you!
Thanks, Zane đ
This is a great list! Glad to see got receive a mention, I've been using it for quite a while and will never look back
Yes! I've switched to using
got
for most things these days as well -- some of the upgrades from v9 to v10 were a little rough.Thanks for the kind words đ
Can you expand on Consolidate? Is it just a wrapper for a bunch of different view engines/templating libraries? Or is it an abstraction? I don't see why id use it.
Yeah, it's an abstraction over different templating libraries.
If you're currently using a particular templating engine, then there's probably no great reason to switch.
Hope this helps đ