So yeah I am going to write about probably the most ignored file from our directories package-lock.json !!
package-lock.json is an extremely important file that is there to save you from a lot of boom boom bam bam 🔥 in your repositories.
So before we get into package-lock.json let's talk about semantic versioning and package.json.
1. Semantic Versioning
Semantic versioning or SemVer is the ideal way of versioning packages. They are usually written like 1.4.5
(major.minor.patch)
1a. Bug fix/patch version
Includes bug fixes/documentation spelling mistakes etc.
1b. Minor version
Includes additions of functions or API which does not break anything from the older versions So anything that runs on v1.1.0 should work on v1.9.0 as well.
1c. Major version
Includes version which breaks stuff. It can include removing APIs or changing names of functions so anything that works on v1.0.0 may not necessarily work on v2.0.0
2. Package.json
package.json is a file that contains information about your project (name, version, etc) and it lists the packages that your project is dependent on.
So as you can see in the picture above after every dependency listed under package.json there's a number something like ^2.20.0
which is the version of that package but before the version, there is ^
. So ^ this little guy can be a total destroyer for your project.
^ sign before the version tells npm that if someone clones the project and runs npm install in the directory then install the latest minor version of the package in his node_modules.
So lets say I am having express with ^2.20.0
in package.json and then express team releases version 2.24.0 and now when someone clone my repo and runs npm install
in that directory they will get the version 2.24.0 (You can also put ~
instead of ^
it will update to latest patch version)
However, this can be a huge issue if package developers break any of the functions on the minor version as it can make your application break down.
So npm later released a new file called package-lock.json to avoid such scenarios
3. package-lock.json
package-lock.json will simply avoid this general behavior of installing updated minor version so when someone clones your repo and run npm install in their machine. NPM will look into package-lock.json and install exact versions of the package as the owner has installed so it will ignore the ^
and ~
from package.json.
Also, it contains some other meta information which saves time of fetching that data from npm while you do npm install.
You can refer npm blog for some more information on package-lock.json.
Thank You for reading this!
I hope this was useful 🎉 :)
EDIT: So while reading the comments I thought I should also explain how package-lock.json changes so here's one of the replies that I wrote that I think everyone should go through
So I created a project named 'project' and did npm install --save vue-extra@1.0.0
and cloned it three times so there's 'projectclone1', 'projectclone2' and 'projectclone3'
projectclone1
In projectclone1 I have same package.json and package-lock.json as the original project (which means I did not change anything manually) and I run npm install
so it installed the same version as original that is v1.0.0 of vue-extra
projectclone2
In projectclone2 also I had the same package.json and package-lock.json but here instead of doing npm install
I did npm install --save vue-extra
which updated the package changing the package.json and package-lock.json so it installed the latest version that is v1.1.4 of vue-extra
projectclone3
In projectclone3 I opened package.json and manually changed vue-extra:"^1.0.0" to "^1.1.4" and did npm install
, Here since I updated package.json npm considered package.json as a matter of truth and installed v1.1.4 of vue-extra and it also updated package-lock.json to v1.1.4
So if your package.json is somehow changed or updated and the version in package.json does not match with the version in package-lock.json then it will install the version from package.json and will update the package-lock.json accordingly.
I hope this clears up everything
Thanks for reading and asking this question.
EDIT2 : Quoting from the comment of Kat Marchán
(https://dev.to/zkat/comment/epbj) She is the one who wrote npm ci
and added package-lock.json to NPM
Hi! I wrote npm ci
and I'm also the one who added package-lock.json
to NPM back in the day.
The story about package.json vs package-lock.json is tricky: npm install
does not ignore package.json
versions, nor does it ignore the package-lock.json
. What it does is verify that the package.json and package-lock.json correspond to each other. That is, if the semver versions described in package.json
fit with the locked versions in package-lock.json
, npm install
will use the latter completely, just like npm ci
would.
Now, ff you change package.json
such that the versions in package-lock.json
are no longer valid, your npm install
will be treated as if you'd done npm install some-pkg@x.y.z
, where x.y.z
is the new version in the package.json
for some-package
.
This was done intentionally because, after early feedback in npm@5
, we realized that one of the ways people edited their dependencies was by editing package.json
directly, and it became a bit of a usability nightmare to treat package-lock.json
as canonical in those cases. It was a trade-off between two competing worlds, and the current behavior won out.
This is why npm ci
was born: because the behavior for npm install
was actually what people wanted, in practice (when they actually ran into the behavior), and npm ci
had a nice ring to it anyway (it was eventually backronymed to clean-install
for this reason).
Hope this helps! Nice article! 👍🏼
</div>
Top comments (24)
Hi! I wrote
npm ci
and I'm also the one who addedpackage-lock.json
to NPM back in the day.The story about package.json vs package-lock.json is tricky:
npm install
does not ignorepackage.json
versions, nor does it ignore thepackage-lock.json
. What it does is verify that the package.json and package-lock.json correspond to each other. That is, if the semver versions described inpackage.json
fit with the locked versions inpackage-lock.json
,npm install
will use the latter completely, just likenpm ci
would.Now, ff you change
package.json
such that the versions inpackage-lock.json
are no longer valid, yournpm install
will be treated as if you'd donenpm install some-pkg@x.y.z
, wherex.y.z
is the new version in thepackage.json
forsome-package
.This was done intentionally because, after early feedback in
npm@5
, we realized that one of the ways people edited their dependencies was by editingpackage.json
directly, and it became a bit of a usability nightmare to treatpackage-lock.json
as canonical in those cases. It was a trade-off between two competing worlds, and the current behavior won out.This is why
npm ci
was born: because the behavior fornpm install
was actually what people wanted, in practice (when they actually ran into the behavior), andnpm ci
had a nice ring to it anyway (it was eventually backronymed toclean-install
for this reason).Hope this helps! Nice article! 👍🏼
Thank you so much for this comment and yes it explained everything about the
npm install
behavior.also thank you for reading this it's pretty cool to see the article reached the person who added package-lock 😭😭😭🌻🌻
Hello Kat 👋
I found this article and this comment after so much searching, and both have been a great help to my understanding of npm. Thanks everyone!
There are still somethings I am not clear on.
1) Does
npm install
with a package-lock present, with semver satisfied between it (the lockfile; having exact versions) and the package.json (with semver), cause the package-lock file to be updated? (maybe there's a new satisfiable version out there in the registry) -- OR does it just go about installing the modules since package.json and package-lock are in sync?If it does update even when satisfied, does
npm install some-new-package
also cause this update?2) Is it currently acceptable to manually update package.json?
3) You mentioned multiple
npm install
behaviors, so which are you referring to here:Thanks again! Any help is greatly appreciated!
Cheers 🤙
Robby
I'm not sure this is completely correct any longer with the latest version of NPM?
From my understanding, npm install command will still use the main packages file, and updates the lock file. This is why people constantly see the lock file being updated in commits.
To use the exact versions as specified in lock file, you can use npm ci. This is intended for builds/deploys so you never get newer versions than what someone developed on.
I may be misunderstanding though...
So I created a project named 'project' and did
npm install --save vue-extra@1.0.0
and cloned it three times so there's 'projectclone1', 'projectclone2' and 'projectclone3'projectclone1
In projectclone1 I have same package.json and package-lock.json as the original project (which means I did not change anything manually) and I run
npm install
so it installed the same version as original that is v1.0.0 of vue-extraprojectclone2
In projectclone2 also I had the same package.json and package-lock.json but here instead of doing
npm install
I didnpm install --save vue-extra
which updated the package changing the package.json and package-lock.json so it installed the latest version that is v1.1.4 of vue-extraprojectclone3
In projectclone3 I opened package.json and manually changed vue-extra:"^1.0.0" to "^1.1.4" and did
npm install
, Here since I updated package.json npm considered package.json as a matter of truth and installed v1.1.4 of vue-extra and it also updated package-lock.json to v1.1.4So if your package.json is somehow changed or updated and the version in package.json does not match with the version in package-lock.json then it will install the version from package.json and will update the package-lock.json accordingly.
I hope this clears up everything
Thanks for reading and asking this question.
Interesting, thanks for taking the time to run those tests!
Out of curiosity, what version of NPM are you using? I think some of the confusion is behavior changed at some point. So, depending on what version various team members are on, they see different actions.
See this S.O. post for an example of the confusion - stackoverflow.com/questions/450220...
I am using v6.11.2 and yes you are right the behavior had some issues and some changes during v5.x.x but now I guess almost all of them are fixed so v6 has been pretty stable about the behavior of package-lock.json
In the same stackoverflow answer I found this link of the issue github.com/npm/npm/issues/17979#is... which I found pretty useful.
This is how it works, author is incorrect.
Hi, You should checkout this reply : dev.to/saurabhdaware/comment/eoo4
and this comment from github: github.com/npm/npm/issues/17979#is...
So yes
^
from package.json and will install the version that is mentioned in package-lock.json.Thank you for reading and do correct me if I am wrong.
Hi Saurabh, my post was made on mobile so please excuse it's lack of detail. I've experienced package-lock.json updating itself in both development and CI environments and subsequently builds have failed when sub-dependencies have introduced bugs or breaking changes despite following semantic versioning.
npm ci
faithfully installs the correct package versions in the CI environment.As to why this happens, I'm not sure. I agree that what you've said should work yet in practice I've seen it to not always be the case.
The npm team seem to acknowledge this, here's a quote from their blog:
npm ci promises the most benefit to large teams. Giving developers the ability to "sign off" on a package lock promotes more efficient collaboration across large teams, and the ability to install exactly what is in a lockfile has the potential to save tens if not hundreds of developer hours a month, freeing teams up to spend more time building and shipping amazing things.
blog.npmjs.org/post/171556855892/i...
It seems hundreds of developers on SO are also confused about npm's lock behaviour: stackoverflow.com/questions/450220....
Yeah even ive ran into problems where 100s of lines were updated in my package-lock.json so i think it is because a lot combinations are possible like i mentioned above plus if you've seen dependabot commits, they update package-lock to bumb versions but along with that they also change the integrity hash so it doesn't end up creating conflicts.
also a lot of time we pull from other branches so if any of them updated your dependent package they may end up updating the tree in package-lock
so yeah a lot of permutations and combinations to think about :(
Hi Saurabh Thanks for the super easy explanation of Package-lock.json.
Right now I am learning React and every time I have to add new dependency in my app I use
npm install somePackageName
and the get the errornpm ERR! Cannot read property 'match' of undefined
so every time I delete package-lock.json file and again run the same command and this time with success result. This is what brought me here
That's a weird error. Maybe you should try having
--save-dev
or--save
flag on npm install. Not sure about what the default npm install behavior is but normall--save
or--save-dev
will add it to package.json dependency object and it will be installed locally.tried that too but that didn't solved the problem...! I had to delete package-lock.json and then execute the command again...
In order for npm to behave as described in the article (honoring versions from package-lock instead of just the latest that will satisfy package.json) you have to run
npm ci
instead ofnpm install
FYI. docs.npmjs.com/cli/ci.htmlnpm ci
will install from package-lock.json ONLY. So if you manually change the version from package.json it will throw an error.npm install
on the other hand will install from package-lock.json as long as package-lock.json and package.json are in sync. If in any case, package.json has changed then it will install version from package.json and will update the package-lock.json accordingly.Thanks for reading and pointing out
npm ci
. Do correct me if I am wrong.Please note that devs forget to add this important file to the version control system (read: git). So please devs, add this file. And update when needed (especially for security issues).
Bottom line: do NOT gitignore the package-lock.json file.
yeah exactly, thanks for pointing out :)
this is so far the best explanation i've read of what this file does. i'm assuming the reason why this file changes so much in our repo is because people are constantly upgrading npm packages? we've had many merge conflicts around that file, so it eventually got added to gitignore.
i didn't hear the whole conversation, but it sounded like one of our test deploys failed miserably because of npm package changes somewhere. could probably have been avoided then.
I think my this comment will explain a lot of extra things dev.to/saurabhdaware/comment/eoo4
So basically there are few cases when package-lock.json changes
npm install --save <package>
so it updates the package to absolute latest version changing the version inside package.json and package-lock.jsonAlso gitignoring package-lock is kind of risky unless you have any other workaround it
And thank you for reading! I am super happy to see that you found it useful :D
Amazing, I didn't know what this file did, till now..
Thank you so much! Do read the discussion as well, as it pretty much explains other parameters which may update package-lock.json
Semver should be banned
It's actually pretty useful just that package developers need to be aware of it