DEV Community

Discussion on: In-depth of tnpm rapid mode - how we managed to be 10 second faster than pnpm

Collapse
 
arcanis profile image
Maël Nison

It's a quite interesting experiment. I thought also about the idea of pre-computing dependency trees (or at least part of them) on the server, but the problem with that is that it relies on cooperation from a third-party we know we won't be able to influence. Similarly, a part of Yarn's overhead is caused by the tgz->zip conversion we run¹. It'd be much faster if npm could serve such archives natively, but we can't bet on that.

¹ We need that because, quite similar to what you do with Fuse, Yarn PnP loads files directly from zip archives, without decompression. However tgz doesn't support random file access, whereas zip does. Hence the conversion.

Collapse
 
atian25 profile image
TZ | 天猪 • Edited

Yes, we are an enterprise scenario, so it is relatively manageable.

Regarding the package format, we had considered converting the format to stargz + lz4 when syncing packages to cnpm registry at server-side, but we have found that the ROI is not high in our current practice.

Collapse
 
killagu profile image
killa

We are inspired by stargz introduced by crfs. It append the tar toc to the tail to make tar random access.

In our practice, we download the tgz file with stream, uncompress to tar, and read the tar entries to generate toc file.

  • Store the package tar file, gzip is too expensive.
  • Toc file store independently for the random access.