DEV Community

Using npx and npm scripts to Reduce the Burden of Developer Tools

Tierney Cyren on April 22, 2019

On Friday, I was working on a workshop-ified version of Step by Step Express for Flawless Hacks in Brooklyn. In the workshop-ified version, I've m...

Read full post

Bryan Hughes • Apr 22 '19

Cool article! 💜 npx!

I thought I'd throw out that in certain situations, there is another solution: install the tool as a dev depedency locally and call it from a run script.

npm does this trick that I don't think many folks are aware of: whenever you install a module with a "bin" entry in it's package.json, it installs a link to that bin entry in node_modules/.bin. Then, whenever you call a run script, npm will automatically append node_modules/.bin to your path before executing any commands. The upshot is that you don't have to require your users to do a manual step, you don't have to wait for npx to download the dependency on the fly, and you can control the version of prettydiff like you would for any other dependency.

Your package.json would effectively look like this:

{
  "devDependencies": {
    "prettydiff": "^101.0.0"
  },
  "scripts": {
    "diff": "prettydiff diff source:\"app.js\" diff:\"app.complete.js\""
  }
}

This does assume that you want this dependency tracked in package.json though, which isn't always the case, and I do think npx is a little more flexible in the end. Still a good trick though.

Tierney Cyren • Apr 22 '19

I never knew about this! I've seen many projects suggest using this, but was always incredibly frustrated because I could never quite get it to work and always ended up installing globally. My assumption was that the authors were making an assumption that I already have the module globally installed.

Thanks for sharing this, Bryan! 💖

dAVE Inden • Apr 22 '19

This is really cool! I didn't know about this. Being able to execute the package without having to download it is pretty nice. Thanks for sharing this!

dAVE Inden • Apr 22 '19

Great article! I love npx. I recently setup a new machine and as I was getting stuff installed I realized there are multiple CLIs I would install globally, but now don't have to and can just use npx when I need them. Now, I'm thinking about creating bash aliases to have the commands ready and waiting.

Tierney Cyren • Apr 23 '19

I've very much considered doing this as well, but I'm still a total bash newbie and every time I've ended up setting up aliases they somehow get blown away.

I've absolutely considered shipping an additional command to my bitandbang module that just creates a child process and does all the commands I want to run (specifically for project initialization, which actually ended up documenting for the workshop here!).

There's some npm-y things I learned about recently that are launching in the not too distant future that I'm super excited about and am going to use to do this instead.

dAVE Inden • Apr 23 '19

I'm not great with bash either. I have to search for everything whenever I need to make adjustments. I've been successful at adding aliases to my .bashrc file, but if it got blown away somehow I'm not sure I'd be able to debug that and would just have to recreate it. Thankfully that hasn't happened. It does remind me that I need to have a better place online to save things like this for later use.
That's awesome there is new features coming to npm to help your process. I'm excited to hear about them when they come out! I really like your workshop too. It's a very cool way to structure things for teaching folks.

Ben Sinclair • Apr 23 '19

I think npx is a handy wrapper for giving a one-off demo in your project's README, but it quickly becomes a chore - at a talk the other day someone kept using it instead of installing the apps locally and it added about an extra 30 seconds to every single time they ran the build command. In other words, if you're going to use the command on the regular, don't use npx.

Tierney Cyren • Apr 23 '19 • Edited

I definitely agree that heavier CLIs may not be perfect. Specifically, I've noticed this problem with a lot of "modern" CLIs that have absolutely massive dependency trees to accomplish scaffolding tasks (think create-react-app).

That said, npm still caches the modules so after the first run (outside of situations like CI, where you should be using npm ci) this realistically shouldn't be a problem and now I'm super curious about why it was taking so long to run those commands 🤔

Ben Sinclair • Apr 23 '19

I'm guessing that it was because the project was running BackstopJS (a visual regression testing tool) and in order to work cross platform, that runs in a docker container. If the npx command was running inside a container that didn't persist, then it would have to download everything every time.
TBH, I wasn't paying enough attention to that side of things to be able to give you an accurate description. I just assumed that it was npx that didn't cache anything!

Tierney Cyren • Apr 23 '19

Ah yeah, a Docker container would have absolutely stripped any benefits there. In that case, you're 100% right that a normal global install would be best.

Ideally, pre-baking that into the image would be the best case scenario... but if they couldn't have done that taking Bryan Hughes advice from his comment on this post and putting it into devDependencies. Combining that with npm ci usage would be great too, since npm ci just reads the predetermined dependencies in the package-lock.json and doesn't take compute time to resolve the dependency structure.