Hey everyone! I don't think I've seen this discussed on DEV yet. I'm not the most aware about the JS community (I'm a Pythonista at the core), but I thought it'd be interesting to have a chat together about this.
C J Silverio gave a talk at JSConfEU 2019 a few days ago:
The talk is super interesting and very well crafted — I highly recommend you watch it! But here's a modest tl;dr.
In this talk, Silverio discusses the reasons behind the Entropic project.
The first reason is that a part of the "JavaScript commons" (the package registry) is owned by a private company (npm Inc), whose sole purpose is to "take money and turn it into more money".
She makes the point that having another private company (Microsoft) run the Node package ecosystem isn't going to be the solution either.
(We previously discussed the GitHub Package Registry on DEV:)
GitHub announces GitHub Package Registry
Peter Kim Frank ・ May 10 '19
Plus, still according to her, building any other centralized package registry — even owned by a foundation like OpenJS — wouldn't work either, because the flow of package downloads in the Node.js ecosystem is humongous. It simply costs way too much to host and serve all packages from a central location.
With all the reasons in mind, at the very end of the talk Silverio introduces Entropic, a "federated package registry for anything, but mostly JavaScript". It comes with a CLI called ds (like "delta entropy" — a of bit of a physics joke).
From the words of C J Silverio herself, Entropic is still very young ("don't use it yet!"), but a handful of people are working on the Entropic project already. The main repository is here:
entropic-dev
/
entropic
🦝 📦 a package registry for anything, but mostly javascript 🦝 🦝 🦝
Entropic: a federated package registry for anything
A new package registry with a new CLI, designed to be easy to stand up inside your network. Entropic features an entirely new file-centric API and a content-addressable storage system that attempts to minimize the amount of data you must retrieve over a network. This file-centric approach also applies to the publication API. See the API section of the manifesto for more details about the API offered.
Entropic assumes many registries co-existing and interoperating as a part of your normal workflow. All Entropic packages are namespaced, and a full Entropic package spec also includes the hostname of its registry.
The legacy node package manager is treated as a read-only archive. You may install legacy packages through your Entropic home instance.
See docs/README.md for the manifesto.
Are you interested in contributing? Do you have some feedback to share? Come talk with us in our…
So, what are your reactions to this announcement?
Top comments (7)
From what I've understood, the idea behind Entropic is for packages to be distributed across many instances (each acting like a separate registry). To me, this sounds similar to how Mastodon tries to be a decentralized version of Twitter.
I think it's a very interesting idea. The "commons" of JavaScript should definitely be owned by the community, and having many instances sharing the load of running and serving packages sounds like a fairly sound idea.
But I also wonder:
I also think this is quite new, so it'd be a fair bit of experimentation and validation. Does anyone know of an existing (and successful) federated package registry model?
So I don't know if this counts as "federated", per se, but Perl's CPAN has had a bunch of mirrors for a very long time. Many companies who used Perl back in the day would mirror CPAN as a way of giving back to the community, and using a program called
minicpan, it was (and still is) easy to set up a mirror locally on your own harddrive. I've found this useful in the past for speeding up installs, browsing documentation for modules I don't have installed on an airplane, etc.Of course, that's only one part of it - publishing a module still needed to happen through a service called PAUSE. However, you can add whatever modules you want without PAUSE - private or otherwise - to a CPAN mirror you maintain; all you need to do is make sure your CPAN installer is pointed at the right mirror.
Institutions, foundations, companies. The important thing is that one could switch on a moments notice.
Yeah, there's always going to be a few "go to" servers, especially if they have a faster CDN. The trick is to make it transparent to the user. Even a public log where all the mirrors and modules are that a tool can parse and adjust at runtime it's a good start
I think Java's Maven offers an interesting model: the biggest public registry/repository is maintained by Sonatype, which afaik is also a private company, if an older and stabler one than npm. However, the Maven POM format makes including other and multiple registries extremely simple and there's no shortage of mirrors. Node package.json files don't allow for this: you're locked into one registry at a time. I got started with Node a year or two after npm became the dominant registry but I've often wondered how and why that turned out the way it did.
I'm glad it's happening! I think NPM could have avoided things coming to this, if they'd tried, but it might be for the better in the long run since their series of unforced errors seems to have lit a bit of a fire under the Node community's collective ass.
I'm curious to see if there's an Entropic-OpenRegistry collaboration in the cards.
Go's community, which doesn't have a registry yet (they rely on GitHub to pull in the code essentially), is trying to do the right thing and go decentralized from the start. Don't know how far along that is :D
I had just seen this video this is really awesome :)