DEV Community

Cover image for How to Evaluate NPM Packages
Sam Winter
Sam Winter

Posted on • Originally published at thecarrots.io

How to Evaluate NPM Packages

I recently wrote an article on how to speed up React development, advising to, among other things, use packages instead of reinventing the wheel. You'll almost certainly want to use a modal library, for example, instead of building your own implementation. (I say "almost certainly" to include masochists and sadistic instructors.)

Here follows a qualification to my prior recommendation. Using third-party software carries risk. Malicious packages make the news almost every week. Millions of developer hours have been spent migrating from a deprecated library.

As an example, I'll compare several popular React component libraries along various mostly quantitative axes that assess risks to sustainability, performance, and security. I won't delve into subjective concerns such as APIs and UI, though these are certainly major factors in choosing a package. Even among this lofty tier of open source software, the difference in risk profiles is stark:

Longevity

Developers love stars. They're the de facto measure of a repository's quality. But they're an incomplete and at times misleading metric of potential to be evergreen (I couldn't resist the pun on Segment's component library). Other common indicators are how many repositories use this project, how many contributors there are, and which if any organizations sponsor the project. Here's how our component libraries stack up:

Package Stars Used By Contributors Corporate Sponsor
Chakra UI 12.4k 638 246
Evergreen 10.3k 1626 105 Segment
Antd 64.8k 5000+ 1305
Base Web 5.6k 1012 177 Uber

All of these libraries are established, though Antd stands well above the rest. It's certainly not in danger of deprecation. A caveat to Antd's clear superiority is its age. It gained popularity long before these other libraries even had a package.json and hence acquired more stars, users, and so on. Its only major competitor for years was Material UI.

Before installing any package, I typically check its star history using Tim Qian's fantastic tool. Antd's advantage is clear:

Star History Comparison

Antd almost breaks the scale of our chart, so let's take a closer look at our newcomers:

Star History Comparison Sans Antd

Here we see drastically different trajectories. From star count alone, one might assume that Chakra UI and Evergreen are about equal in terms of adoption, but Chakra UI is clearly outstripping its competitors. All of these libraries received an initial burst of stars, yet only Chakra has sustained its growth for the past year.

Say you prioritize corporate sponsorship when evaluating packages and have narrowed your choice to Segment's Evergreen and Uber's Base Web. Evergreen might seem like a clear winner given its much higher star count and adoption. Where Evergreen falls short is in the distribution of its contributors and its commit frequency. Here's the activity from its top six contributors:

Evergreen Top Contributors

There have been few significant contributors to Evergreen outside of the top three developers. Even more alarming is that two of those three core contributors haven't been active in well over a year. Here's how Base Web compares:

Base Web Top Contributors

Although Base Web also has a somewhat top-heavy distribution of contributions, it's more equal than Evergreen's, and its top contributors are still active. Meanwhile, the entire weight of Evergreen seems to rest on mshwery's shoulders.

The code frequency charts tell a similar story. Contributions to Base Web have been much more consistent:

Evergreen
Evergreen Code Frequency

Base Web
Base Web Code Frequency

While libraries backed by successful tech companies tend to be more enduring (React itself is a prime example), they carry their own risks. Sometimes one or only a few engineers champion an open-source library within the company. When those advocates leave, the project can wither without institutional buy-in. More decentralized packages such as Antd and Chakra UI are resistant to political indifference.

Performance

Two pieces of software that do the same thing might radically differ in size. Moment.js is the most popular date manipulation library, with over twelve million downloads per week, yet it doesn't tree shake well and will add 300kB to your project. Day.js has almost the exact same API and is only 2kB. In fact, Moment.js now recommends using Day.js and other date libraries as alternatives.

Here's the cost of each component library according to the indispensable Bundlephobia:

Package Minified Size Tree Shakable Side Effects Dependencies
Chakra UI 363.3kB Yes No 44
Evergreen 795.3kB Yes No 17
Antd 1.2MB Yes Yes 45
Base Web 60.9kB Yes No 22

Antd is the obvious loser in terms of performance. By the way, Moment.js accounts for about 20% of its size. To download the entire Antd package would take almost seven seconds on emerging 3G. The fact that Antd has side effects is equally concerning. Side effects (code that performs some behavior external to its module) hinders treeshaking. A bundler cannot safely remove a module with side effects because it could have external effects that are required.

The data that Bundlephobia provides is instructive but incomplete. A package's impact on your application's size might be limited if much of that package is tree shaken. To accurately assess its size in the context of your application, use a tool like Webpack Bundle Analyzer, which generates a treemap visualization of bundle content:

Webpack Bundle Analyzer in Action

Security

The most secure packages are popular, well maintained, and require minimal downstream dependencies. Even if a library has millions of downloads, its risk is higher when it doesn't meet these additional criteria (as when the original, inactive maintainer of event-stream gave publishing rights to someone who added a malicious downstream dependency). If a package is popular, there are more users to quickly identify a security issue; if it's well maintained, there are more developers to patch it; and if it has few dependencies, there's less risk in the first place.

While all of the component libraries we've examined are probably secure, the numerous dependencies of Chakra UI and Antd make them more vulnerable. After downloading a dependency, you should perform an audit with yarn audit or npm audit. These CLI commands identify security vulnerabilities in your packages and recommend patches. Ideally, you would also include yarn audit or npm audit in your continuous integration pipeline to check dependencies against known vulnerabilities with every pull request. yarn audit did in fact reveal a low-risk issue in Evergreen:

┌───────────────┬──────────────────────────────────────────────────────────────┐
│ low           │ Denial of Service                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ node-fetch                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=2.6.1 <3.0.0-beta.1|| >= 3.0.0-beta.9                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ evergreen-ui                                                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ evergreen-ui > glamor > fbjs > isomorphic-fetch > node-fetch │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1556                        │
└───────────────┴──────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The above advisory was published on September 10th, and Evergreen's latest version was released on September 28th. The issue is likely inconsequential, but the fact that Evergreen did not fix it in that release or any subsequent commit might reflect poorly on their attention to security.

If software has eaten the world, then open-source software propels it forward. When evaluating packages, carefully select your chariot so that you find yourself astride a sleek, nimble unicorn instead of on your sofa on a Saturday night submitting pull requests for the ancient library upon which your entire application depends while praying for a review from a maintainer who's somewhere off the coast of Bermuda.

About Us

At Carrots we're building a hiring platform specifically for software engineers. You can connect your GitHub, Stack Overflow, and more to show off so much more than your resume. Our algorithm shows where you rank among world-class talent and surfaces your profile to top companies.

Top comments (0)