Hello! Long time no see! Back in December, we decided to start working on our next major release, the 3.0. It took a bit of time to do everything we intended to do, but here we are! So let's talk a bit about what it changes, and what it brings. Note that these are only the highlights, the full changelog is much more comprehensive.
And if you just love to read our log posts, here are the past entries 👇
- Yarn 2.4 🎄🎁 Log Filters, Audits, Better Warnings, ...
- Yarn 2.3 🍦✨ Info Command, Detailed Options, Nohoist, ...
- Yarn 2.2 🚅🌟 Dedupe, Faster, Lighter, ...
- Yarn 2.1 🐱🏍 Git Workspaces, Focused Installs, Loose Mode, ...
Back when the project was started in 2017, we didn't took the time to establish a formal governance document. This is now addressed, and our team composition can be found here. It doesn't change anything in practice (this is how we worked for more than two years now), but we hope it may give you a better understanding as to how we work and reach consensus.
From 2017 to 2019 Yarn was mostly maintained by Facebook engineers. While it worked relatively well, the Yarn 1 -> 2 release also proved to be the right time to expand our active team to other horizons, and nowadays no two of our active contributors work at the same company - and none at Facebook.
Consequently, we've decided to setup an OpenCollective (or GitHub Sponsors) to give our supporters a way to both express their thanks to our team, and give us resources we can then inject back into the project.
While the migration from Yarn 1 to Yarn 2 brought some discomfort, the migration to Yarn 3 should prove easier - regardless of the version you come from. The user-facing breaking changes we made this season are mostly little details that may only affect you in very specific cases:
- Node 10 isn't supported anymore
- Plug'n'Play hooks are now called
- Virtual folders are now called
- The editor SDKs have been moved to
- Etc; full list here
Even for Yarn 1 users, migrating from 1 to 3 should be easier: we made it so that Yarn will detect when this situation arises to then automatically enable the
node-modules linker. That alone should address most of the problems you may have been hitting when attempting the upgrade - and for everything else, make sure to take a look at our Migration Guide which got significantly improved over the past year.
Support for the
When using Yarn 3 w/ PnP, the
exports field will be properly resolved regardless of your Node version. If you're not familiar with this field, you can see it as a way to:
- Replace the
- Soft-prevent accessing arbitrary files in the package
- Conditionally remap files depending on the context (bundlers, ...)
Various tweaks have been made to address some of the largest resource consumptions in Yarn. Installs have been improved (turning us faster than pnpm in some scenarios, which is quite a feat!), but not only: script execution tends to have a natural overhead, but bugs in 2.4 and prior caused this overhead to grow relative to the size of the project itself. This is no longer the case, and the overhead should now be constant.
As you may know, Yarn is built around a few interfaces. One of them is called a "linker", and tells Yarn how to install packages on disk. It's how we can support both PnP and node_modules installs without changing much code.
One advantage of this architecture is how it allows us to efficiently iterate on alternative install strategies. For this release, larixer implemented a new experimental
nmMode setting that can be used to instruct the linker to use a specific copy scheme:
hardlinks-localwill use hardlinks when the same package is found multiple times within the same project (but only if they have exactly the same version at the moment).
hardlinks-globalwill use hardlinks on identical files (even across different versions!), but will also make them point to a global content-addressable directory. This is similar to what pnpm does. Note that if the cache is corrupted (for example because you manually edited it), Yarn will automatically repair it on subsequent installs.
I myself have been playing with a pnpm-style linker. It hasn't shipped yet since I'm cautious about adding complexity that could end up unmaintained, but given how small it is there's a decent chance we could add it in a later release as an experimental install mode.
As you may know, given that system shells are rarely portable across Windows and Posix, Yarn no longer uses them to run your
scripts entries. Instead, we use our very own shell interpreter.
We're happy to report that this shell just got smarter, and now provides two additional syntaxes that you can reliably use on both Windows and Posix:
build-js & build-css & # Background jobs ls 2>/dev/null # File descriptor redirections
Additionally, background jobs have their output color-coded, so you can clearly identify their output, even interlaced.
We now use ESBuild to generate the Yarn bundles and as such worked to ensure good compatibility with Plug'n'Play installs. The result is the new
@yarnpkg/esbuild-plugin-pnp package which lets you transparently build your code using the default Yarn installation mode. It's still relatively young, so feel free to drop us an issue if you notice something strange!
While it won't change much for most end-users, the move to ESBuild also provided decent build speed improvements (around 6x faster), making it less frustrating to build Yarn from sources ✨
New plugin APIs
Yarn supports writing plugins that can inject themselves into various places and leverage some of the builtin modules provided by the core. While we didn't get the chance to make all the improvements we hoped, we've still been able to upgrade the command line framework to Clipanion 3, which lets you write intuitive type-checked commands with a minimal syntactic overhead.
According to DEV.to, those change notes take about 5 minutes to read - by contrast, the 2.0 release post was a whopping 15 minutes! Of course, this time around we didn't need to fully rewrite Yarn, hence a lower amount of "critical information" we need you to be aware of 😉 We expect that to be the norm from now on: majors won't have a lot of super impactful changes, mostly just some architecture cleaning and modernization, as new features will tend to land in minors.
As for our team, we're very happy of the work we've been doing! Working on the codebase still feels like a treat, and features are often constrained to a few identifiable files - proving that our initial redesign bet was right. Our stats suggest that the result are visible to our users as well, and while I remain cautious about popularity metrics it's certainly nice to see.
Finally, remember that Yarn now has an OpenCollective / GitHub Sponsors! If your company benefits from our work, or would like to see particular fixes land, sponsoring the project is a good way to engage with us 🙂
What's to come?
A few features initially slated for 3.0 have been pushed back to the next minor so that we have more time to properly incubate them. Some of the things we have in motion:
- Corepack integration
- ESM support under PnP mode
- Builtin CLI completion
- Changelog generation
- Improved performances
- pnpm-style linker
- And more...!
Of course that's only on the top of my head, so it's possible our objectives shift during the next weeks depending on our own priorities - and of course depending on whether you help us or not 😛
Top comments (18)
yarn up -R! It was my last blocker from migration from Yarn 1.
Thanks for the update. The new shell concurrent feature saved my day.
How do i upgrade from berry v2 to berry v3.
yarn set version latestbut it used the development branch something i don t want
I just tested this out upgrading from v2, dev.to/hydrosquall/comment/1gop5, hope this helps!
how to upgrade global yarn from berry v2 to berry v3 when using npm
I know ESM as well, but what does it mean ESM support as a package manager?
Been using Yarn 2 for several months now. PnP is fine except when 3rd party projects have stuff like references to node_modules hard-coded in their distributable. I don't see a workaround to this, and a HUGE chunk of 3rd party libraries do this. I'm going back to Yarn 1 or even NPM (if work on Yarn 1 halts).
Yarn 1 was cool, and moved NPM to get their act together a bit, but I hate when maintainers just assume the rest of the ecosystem will migrate with them. This is why Python 3 took a decade to be adopted, and that adoption only happened because people were absolutely forced to use Python whereas switching back to NPM is a breeze.
Almost every single day, for months, I've ran into problems with PnP...
You can enable the
node_modulesjust fine if that's what you prefer.
From my experience, very few 3rd-party libraries actually hardcode the n_m folder anymore, and I found them relatively easy to ignore (RN being the main contender). If large projects like Next or Gatsby manage to make things work without trouble, there's really no reason in 2021 for projects to still misslist their dependencies.
You're referring to
nodeLinker? Doesn't that just disable PnP project-wide? In which case, why use Yarn 2+ at all?
I agree it'd be nice if every maintainer fixed their projects to not reference n_m, but that's just not the reality of the Node ecosystem... github.com/search?q=%22node_module...
Yesterday my use case was trying to use Playwright (30k Stars, hundreds of thousands of downloads every week) with Yarn PnP. Nope.
The maintainer literally said "Wait what version of Yarn?" I said "2.4.2", he said "Ah yarn 2 my old enemy" github.com/microsoft/playwright/se...
I dunno man. I don't want to dog the project and I really appreciated what Yarn 1 did, and the goals of Yarn 2, but someone needs to say "Look, I've used this thing for several months on a decent sized project and it's painful. Every few days I run into something. I wish I would've just used Yarn 1" so others know what they're getting themselves into.
I'm not the only one with this opinion news.ycombinator.com/item?id=28122318
Until there is a really straightforward dev-friendly way to say "Okay, this package is fine if it's a zip and this package is probably never going to work with Yarn 2 and would just be easier to plop into
node_modules" people are just going to keep jumping ship back to npm.
What dev wants to have to study a ton of docs and do a bunch of workarounds to be able to use a package manager that used to be as simple as
yarn add foo?
It's just hard, whereas Yarn 1 wasn't, and npm isn't, and npm has workspaces now. 🤷♂️
Congrats, the release sounds really awesome! ❤️
Uhm, probably a dumb question but how do we update to yarn 3?
I used these two commands in sequence:
Thanks! 🙂 For me, just running
yarn set version berrywas good enough.
If I also ran
yarn set version 3.x, it upgraded me to a release candidate version.
yarn is good.
BUT I prefer npm 7 workspaces.
Upgrading yarn is a burden now.
npm is bundled with node, so easy.
Not sure what's a burdern about
yarn up -Rbut I respect your opinion :)
can you still use
&&or is there a new yarn scripts block? just wondering given the prolifity of
&&and I guess
||. not expecting full shell eval ;)
The day yarn support ESM I'll do the switch! I hope it will happen soon 🤞
Yarn supports ESM with node_modules install scheme. The PnP install scheme will support ESM when Node API let this happen
Yay! Happy to hear that there are fewer breaking changes this time around, and nice work! 🙂