Hi! My name's Kat, and I've been an FOSS dev for over a decade.
I had the opportunity of being, and eventually tech-leading, the NPM CLI team whi...
For further actions, you may consider blocking this person and/or reporting abuse
I use your software everyday! (and never knew who was behind it :) ) - so thanks!
I suppose I could ask: what are your feelings on yarn vs npm? I'm ashamed to say I've never bothered to dive deep into the differences between the two, so I just end up using whichever is more convenient at the time... is that something I should look more into?
Thanks for the ama!
They're both fine. I'm obviously biased towards the thing I worked on myself, even if my relationship with my former employer soured a bit.
Yarn is fine software and it works for a lot of people, and if you find that the way they do things works better for you, by all means, use it! That's also the policy we had on Yarn since its inception.
Once npm@5 came out, I think most differences disappeared, with the major exception of Workspaces, which I'm sure will eventually come to NPM as well. We had them on the roadmap, until they fired my team for trying to labor-organize, and then me and Rebecca (the OTHER former architect of the CLI and my long-time colleague) decided to leave. I have no idea about current plans anymore. :)
Yikes! I hadn't heard that story before; thanks for the link.
And thanks for the answer - makes me feel better about not knowing the difference, since there appears to not be much of a difference since npm@5 :)
Thank you for all the effort you have put into the npm and the ecosystem. I consider
npmone of the most important parts of today's web development ecosystem 😄Putting any work drama aside, and any budget/time constraints, if you could re-write part or all of
npmwhat would you change and why?The tree builder, because it would add Workspaces support and simplify some fairly complex code. I hope
I am usually doing backend programming on the JVM with gradle.org and when I do front-end development, the npm CLI seems to need a lot more of baby sitting. For example:
$ ./gradlew :myTaskwill auto-bootstrap it for me by downloading the right version ofgradle-xx.jar. Node and npm on the other hand will just fail on me with a weird error message$ ./gradlew :testwill detect it and install them.$ npm testwill fail on me because I didn't run$ npm installfirst.$ ./gradlew :myTaskwill complete almost immediatly with a messageTask UP-To-DATE, while npm will gladly waste my time by actually doing everything twice.Ever considered implementing at least some of those features?
I don't really work on NPM anymore, but I do work on a client for Entropic.
I think the first and third features are out of scope for how I generally think of package managers, but I might need to think more on that.
The second one, though, is something that
ds(the Entropic client) will actually take care of for you. We'll see how it goes! Thanks for sharing!For the first feature, I think the package manager should at least document which version of node is required and fail with an explicit error message if it's not present.
The third feature was already present in Makefiles - although obviously in a crude implementation compared to what Gradle is doing.
docs.gradle.org/current/userguide/...
Update: maybe my questions are more about Webpack than about npm/entropic
Thanks for that AMA! Use your software every day :)
Care to talk about the backgrounds that made you move to leave NPM?
I mean, I read some people were let go in a bad way, but I don't understand how a diverse powerhouse where all are biggest friends could lead to that outcome.
Basically, the CEO and COO are horrible human beings, and I strongly believe they fired my colleagues for trying to organize. There's a lot that happened internally that I can't talk about that completely eroded my faith in the company's mission to the community, as it was when I arrived.
I decided I didn't have to keep taking a pay cut for the sake of a culture that was no longer something I could stand behind, and was not what I had been promised.
Hey Kat,
I was wondering how you managed to become Tech Lead for NPM and how you got your job at Msft. Please don't get me wrong, I'm just asking you this because I've recently switched jobs but I'm constantly being unmotivated and I can't figure out why, also, I find that the impostor syndrome is always making me feel that I won't be as good as developers like yourself.
I got hired back in July 2015 the same way anyone would: applying and getting through the application process. From there, it was mostly a process of attrition. I was the second-oldest member of the team, and the oldest (and former architect), Rebecca, moved on to full-on management and product management, leaving an open spot for me once we started hiring more folks into our team. A big reason for giving me that tech lead role was that by then, I'd done major refactors, rewrites, and rearchitecting that reached into all parts of the CLI, so I was intimately familiar with the shape and function of the code. That took about 3 or 4 years to happen, though.
I think a lot of things are simply a matter of time.
As far as joining MSFT, I was introduced to the team through professional connections (both a colleague in TC39, and through twitter). The messages bounced around until I got in touch with someone on the NuGet team, and they got me started with the application process from there!
It's funny: I almost turned Microsoft down because I really really didn't want to do the interview gauntlet. I was deeply burnt out, depressed, and just not braining right after the chaos at NPM, and I didn't think I could perform well enough to get hired at a major company. I even cancelled my initial on-site interview, but one of the contacts messaged me a while later after finding out and talked me into giving it a shot anyway. I still wasn't sure I wanted to do all this, but I went for it, and after a couple of months, I ended up getting an offer!
MSFT wasn't the only place I applied to. I had a spreadsheet of about 40 different places, most of which I at least had first-contact with, and that resulted in about 3 final offers (most of them dropped off because of technicalities, constraints, or me simply deciding not to continue). Of all three, MSFT's ended up being the best combination of the requirements I was looking for in a new job (plus the compensation package), so that's what I picked.
As far as "being as good as developers like myself": I think it's important to note that I consider myself pretty mediocre for someone who's been doing this for 10 years, and any perception of ability is simply a function of experience. The rest is simply luck, and access to opportunity -- I moved to the SF Bay Area about 6 years ago, and the fact is the opportunities here were way more numerous (and lucrative) than anything I experienced living in several other places after I moved out of home.
I kinda take issue with the cult of personality that forms around people in visible position and so-called thoughtleaders on the internet. We're all actually fairly average, normal people who happened upon fame by sheer luck. Please don't evaluate your ability and competence based on where visible folks landed. So much of it has more to do with privilege and luck as opposed to what you or I can actually do, as developers. Keep at it!
Such a great and motivating answer for all of us. Thank you!
Was the v5 release of the npm difficult for you personally? Did the GitHub issues get to you ever? Was there a technical part of the project that was the biggest issue? Was improving the tool to such a great degree hard to do for other reasons (step on anyone's toes?)? #npm5 #theBehemoth #suchGood #wow #muchLove
Sorry if this is a bit long, but I guess it's story time?
npm@5was simultaneously one of the most satisfying and one of the most destructive things I've done in my career...So, flash back to late 2016, and Yarn comes out, putting an IMMENSE amount of pressure on my team. The main reason the CLI had seemed to stagnate was because our team had decided that stability and reliability and making sure what was there worked well were the primary goals for our project, so we'd spent the better part of that entire year fixing bugs, improving Windows support, and just generally making sure that the CLI was a nice, reliable tool.
Yarn came in and kind of flipped our table (and included a lot of FUD about it that I felt at the time, and still do, was very unfair), and our priorities had suddenly shifted: we were bleeding users, and we didn't have a plan to improve the CLI as much as it needed, on the timeline it needed to improve. The only clue we had that might do it was the so-called "cache rewrite" that had been in our backlog for literally years, but we didn't even have data on how much it would help. It was just clear that whatever it is we were doing was not actually what the community wanted from us, and we needed a new plan.
And then, my boss gets fired. Abruptly.
And my new boss tells me to go whole-hog on the cache rewrite.
Now, you have to keep in mind that working on the CLI was my day job, and I worked at a startup, so the emotional subjective experience of seeing Yarn take market share felt like these big Facebook bullies were putting my job and livelihood at risk. I was kinda panicking. I was genuinely scared and was ready to do whatever I needed to do to keep my job. My manager getting summarily executed (figuratively, I promise) heightened that sense of survival, I thought "I'm next", and such.
What followed was one of the most productive periods I've had in my entire career. I was working 12-14 hour days on npm@5, mostly on my own (while Rebecca took care of the stability of
npm@4and generally supported me in my work). I didn't really tell people this was what I was doing. I just did it because I needed to seem useful and that seemed like the most important thing. I needed to try to save the project that paid my bills.Out of that, frankly, herculean effort came libraries like cacache, and then ssri, and then make-fetch-happen and pacote for the network bits, which I worked on in complete isolation from the main CLI project, benchmarking and testing them thoroughly along the way to make sure they were the fastest and most stable things I could write. I essentially rewrote the entire networking and caching layer of the CLI, from scratch, mostly on my own, in about 6 months.
We weren't expecting to release as early as we did, but the Node Core project threw us a curve ball towards the end, saying that we needed to have a semver-major version of NPM ready by early May in order to get it into the new Node release, or we'd be SOL, and we really couldn't risk missing that deadline.
So, I integrated everything in a couple of weeks, and we threw together
package-lock.jsonin only a day or two, in a super-rush. We really didn't have time to test out the new semantics, the new format (which was largely based onnpm-shrinkwrap.json, or most of the integration (remember, the ENTIRE networking layer had been replaced, down to the low-level http client).So yeah, the
npm@5release was bumpy, but I was mostly numb by then from all the work I'd been doing the past several months, and I was relieved to get it out the door. I'm pretty proud of it, honestly! It turned out pretty damn good considering the constraints!You did so great! I loved
npm@5and was so impressed by how much work and love you poured into the project, and your story here just confirms a lot of my suspicion as a casual user of npm.Keep doing awesome stuff; and take care of yourself and do cool things at a sustainable pace.
You are an inspiration <3 👍
You can barely see the guillotine scars anymore!
Also big <3 for all the work you put into npm@5. I really wish you hadn't had to do it under the gun. I can't help but feel that casts a pall over what is really and truly an impressive achievement by pretty much any standards, and regardless of the material conditions of its creation.
No questions, just a huge thank you for what you have done! 🤘
I've been doing javascript for quite some time now and planning to learn Rust, so are you liking the Rust so far? also I would like to know what projects you working on in rust? also since JS abstracts a lot of internal things, does that make learning rust difficult?
I love Rust! I've never done systems programming, and I've just been really enjoying doing new things in it. The community is incredibly friendly, and there's a lot of People Like Me (queer, non-man, etc) actively participating and having a voice in the project, which makes me feel less alone!
Right now, I'm working on a new client for Entropic in Rust, and I've ported cacache, ssri, and srisum over to Rust, just to learn it, and also as backing code for the new Entropic client.
I think the fact that Rust is a systems language adds some necessary complexity, and having to deal with such an intricate type system, and the new borrow checker, means that there's plenty to learn for a plain old JS dev. Once you get over the initial bump, though, Rust gets super fun and interesting to work on! I think Rust is a surprisingly high-level language, considering its systems lang status, and I think it's a great way for JS devs to get into systems dev.
You are the real deal!
Thank you!
I hope you're having fun! That's what matters, in the end, for learning a new thing, I think.
Not yet, but I'm really looking forward to finding something I can wasm!
Do you think the deno project has competitive promise in
npm/nodes space?I don't think
denois at all trying to be useful for frontend web development (this is what I've heard folks say Ryan himself was saying), and I expect npm or, eventually, Entropic to continue to dominate in the tooling and frontend code distribution space.In retrospect what was your favorite open source project you worked on and why?
I would also be interested in how a normal work day looks like for you. :)
Probably sheeple, a full multiple-dispatching prototype object-oriented extension to Common Lisp. This was my first ever open source project and I worked on it for a while when I barely knew how to code at all, and I learned an immense amount from it! It's the reason JS was so easy for me to pick up years down the line.
It was also the first project I was really able to get some really good FOSS collaboration going on (with a friend called Adlai), and the positive feedback look from that was super energizing and made hacking on that project all sorts of fun!
There should be more "POOP" frameworks!
Why do you think the collaboration worked so well on this project?
I think Adlai and I had a particularly fervent enthusiasm for what we were working on, and every time ideas came up, it felt like we amplified them for each other.
I think a really key part of that collaboration was my openness (as the original dev/owner of the project) to let others' ideas in, even if I was uncomfortable with them at first. Working on Sheeple really taught me to trust others' competence when working on FOSS and that the end product is a function of a lot of great ideas and talents that end up being greater than the sum of their parts.
Thanks for the response. I think being more open about new ideas is an idea I should draw from for my personal projects.
Thanks for the opportunity to pick your brain! :) What do you think the future of JS package managers will look like in 5, 10 years?
My dream is to have package management built into the runtime itself. I'm continuing my Tink work now independent of NPM INC, so all the stuff Tink was supposed to do is something you should expect for ds, the package manager for Entropic. As part of this, I also hope the primary javascript registry stops being owned by a for-profit private corporation and is instead something owned by a Foundation, probably the JavaScript Foundation.
Basically, if things go well, we'll have transparent package management, ethically owned and managed by people who aren't union busters.
I guess I can expand on this a bit, because this isn't all that would happen in 5 years. 10 years is a lot harder, though.
So, aside from the built-into-runtime thing, and Entropic...
I think we'll reach a security/scale breaking point where we'll start re-evaluating the open-publishing policy where people push whatever code up, and there will start to be a push for better quality control on packages, now that the ecosystem is so large. I also imagine a market will open up for for-pay packages that people can download and use for a fee, at least for proprietary use. I believe this will come as part of a movement away from permissive licensing. I think a lot of this will be driven by major security events that will happen in that time that will make the events of the past few years look like a joke, and force everyone to take a long, hard look at the way we're doing things.
I also believe NPM INC will stop existing altogether in the next couple of years, and the NPM registry will go down with it (as opposed to being bought out by a major company). I think a bunch of competitors will pop up out of thin air and start vying for the position of being "the next main registry". Things will be in chaos for a while, but we'll finally settle on another monopoly, because that's the nature of the problem. Most likely, though, there will be smaller side-registries that see actual use.
I don't know exactly how, yet, but I think the rise of WASM, and eventually, WASI will have a profound impact on what it means to have a "JavaScript Package Manager", and we'll also need to adapt our tooling for all the various languages people are writing JavaScript-consumable WASM modules in. Lots of tool adaptation.
As far as 10 years from now -- I think it might be safe to assume JavaScript will be on its last legs as a primary development language and will be all but replaced by other languages compiling down to WASM, and using that to run on browsers.
Lots to think about here. Thanks Kat.
What is your take about yarn 2? do you see it as a competitor for Entropic?
Yarn 2 isn't a competitor to Entropic. And it's not a package manager I'd willingly design.
Thanks for the clarification!
And thank you for making JS tooling better for us all:)
I would like to say Hi! =)
Hi! 👋🏼
What is your favorite color? 😀
Green!💚 I've had green or partially green hair for years now!
🙂 cool!
I know it's been just a few months, but any reflections on open source development in node vs Microsoft so far?
I like it so far. Big companies (or, well, MSFT at least) are much better about work-life balance than your usual startup and, in my opinion, much less cult-like (this is up for debate, I'm sure).
And I've only been here a month + a week!
I do like working on NuGet, and I do like that MSFT, unlike a small company like NPM, actually has the resources to staff its open source development teams such that we have the headcount we need to achieve cool things, instead of making us scramble for dear life.
I'm glad you found a place to work on open source at a sustainable pace with resources+help.
Hope you stay @ MS for a while and make NuGet awesome! (Also... Maybe help out the f# peeps with their paket xD I'm legit confused about how I'm supposed to use F# community things that are both on paket and NuGet; are they the same?).
I'm pretty sure they're separate projects!
When do you think Entropic will be ready?
Will it compete with the GitHub registry or would the GHR just be one way of using it?
Will
dsbe an alternative toyarnandnpm?I don't know, the main maintainers all got slammed with work stuff, so work slowed down. I only just got back to working on client work this past weekend, because I had just finished my first month at my new job.
I guess, but I don't think GHR is a general-purpose solution to ecosystem hosting. I think GHR could most definitely implement the Entropic protocol, though, and they're free to do so!
Yes and no -- it's a completely different package manager with a different concept.
What do you think about ReasonML and functional programming?
My first programming language was Common Lisp, and I also hacked Erlang for a while (both of them professionally). I haven't done much in the way of statically-typed ML-family FP, though, but it seems fine if you're into that sort of thing.
ReasonML seems fine. I have no real opinion on it besides it being yet another web dev language/tool.
What are the differences of working for a corporate vs an open source company? Pros, cons?
How to go from your first PR to a full-time job for a opensource project