DEV Community

Discussion on: The node_modules problem

Collapse
 
jwerle profile image
Joseph Werle

I am so tired of seeing posts like this with no solutions. Yes yarn. Yes pnpm. Yes IMS. Yes . They exist. They are not a solution. They are an alternative.

Comparison to Java or equivalent development experiences is not fair nor should one really inform the other this way. Unfortunately, this post does just that.

This is yet another rant without an elixir. What is your actual issue? Too many files? Is your IO/poll/refresh too slow in your IDE?

I come from the world of node, c, and bash. I helped build a package manager for C called clib and one for bash called bpkg. The ideas have always been to allow for clean, small, and reusable code with the help of the package manager for removing duplicate code. Some have been successful in that more than others. I've recently started working with Kotlin/native. This comes with the baggage and tropes of the Java community (Scala too): Verbosity and complicated tooling, being married to IDE and JetBrains (quiet obvious how they monetize through OSS), and the general attitude of the community that they got things right over everyone else. This post is an example of that and the comments do not help.

Instead of having your editor open and bike shedding about the dependencies of a node project vs your Java project, audit what you install and use. Audit what it installs and uses, and so on. This may not be convention for the status quo here, but this is kind of what npm, node, OSS (FOSS too), etc expects implicitly.

We don't get shared objects, static archives, dynamic libraries, frameworks, jars, or any type of compiled object in node. There are ways to make them, but that is outside the scope of node. Instead, we get snapshots and wasm. But that's not useful to you.

What we need is reform in how packages are published and installed. We publish blindly, without whitelist, without security, without tests and we practically do the same for installation.

We (mostly) install source with npm that is built and/or executed on the host machine (ideally). This (hopefully) requires audit or knowledge of the source beyond what's advertised by the developers. We are not installing compatible ABI, we are installing JIT VM (except chakra et al) JavaScript that is not compiled, but evaluated. There are no symbols to deduplicate at compile time because there is no real compile time.

Being aware of these things when publishing and installing will not reduce your file and folder count, it may just help you understand it and why

Collapse
 
leoat12 profile image
Leonardo Teteo

I'm sorry If I sounded conceited or put Java and C# over JavaScript in any part of my text, it was not the intention, it was just that I had experience with these three languages mainly and it sounded to me a good idea to compare the various ideas around dependency management and, in the end, Java and C# indeed looks better for me in this front, but it is just my opinion and you can disagree with it. I think it is important to look around and see what we have already made regarding one problem and use the solution if we see fit and not create this environment of rivalry between languages and communities and reinvent the wheel just to do something different from language X or framework Y. That's what it sounds while reading your comment, sorry if it is not what you intended.

You say that we should audit what we use, but audit can only go so far in my point of view. I can audit the dependencies of my project, meaning that I will try to have as little dependencies as possible, pick only the ones that do just what I want them to do etc. But it is the job of the developer of the library to audit its own code. Downloading a dependency is a signal that we trust and believe that that library will do its job in an efficient way, without security risks. We assume that they are well maintained, if we have to audit the entire code of each library we use, it is better implement ourselves.