Rewrite nodejs cli using golang

Scenes

Since we use monorepo to manage all projects, the build time for lib module initialization is unacceptable, so for the past two days, we have been trying to use golang to rewrite the lib/cli build tool @liuli-util/cli to verify how fast golang can do it.

@liuli-util/cli is a zero-configuration build tool created by us for library and CLI application packaging, but also supports Some additional common features include generating template projects, syncing configurations, deploying, and more.

accomplish

saki npm, all tests are in liuli-tools Done, it's a monorepo with 44 modules, 28 of which need to be built.

The overall build command of liuli-cli is rewritten based on esbuild, supporting build lib/cli and custom targets. Here are some tests I did after the rewrite, proving that using golang can drastically improve lib build performance, it's so fast that it's possible to put it in postinstall.

Commands are repeated 10 times using time seq 10 | xargs -i <cmd>

saki build

Build lib or cli programs based on esbuild, which is very fast.

saki build lib # build lib
saki build cli # build cli

Performance Testing

The following two cli are based on the build implemented by esbuild, but the real runtime of the latter is very long, because nodejs itself also takes time (and is very slow) to load the code.

Download the binaries to install

$ saki build lib

real 0m0.647s
user 0m0.060s
sys 0m0.183s
# Average 64.7ms/time

$ saki build cli

real 0m2.831s
user 0m0.106s
sys 0m0.275s
# Average 283.1ms/time

npm install globally

$ saki build lib

real 0m2.724s
user 0m0.482s
sys 0m1.665s
# Average 272.4ms/time

$ saki build cli

real 0m4.622s
user 0m0.587s
sys 0m1.438s
# Average 462.2ms/time

liuli-cli

$ liuli-cli build lib

real 0m6.882s
user 0m0.648s
sys 0m1.364s
# Average 688.2ms/time

$ liuli-cli build cli

real 0m8.687s
user 0m0.587s
sys 0m1.333s
# Average 868.7ms/time

saki run

An alternative to pnpm --filter . run that tries to improve the efficiency of running commands in multiple threads.

saki run setup # run the setup command on all modules (if there is one)
saki run --filter libs/* setup # run the setup command in all modules matching libs/*
# use --filter array
saki run --filter libs/* --filter apps/* setup # or use , to split
saki run --filter libs/*,apps/* setup

Performance Testing

pnpm + liuli-cli + dts

$ pnpm --filter .run setup

real 4m49.648s
user 0m0.593s
sys 0m1.391s
# Average 1034.4ms/pc

pnpm + liuli-cli

$ pnpm --filter .run setup

real 1m3.847s
user 0m0.468s
sys 0m1.484s
# Average 228ms/pc

saki + liuli-cli

$ saki run setup

real 1m19.657s
user 0m0.497s
sys 0m1.494s
# Average 284.4ms/pc

pnpm + saki

$ pnpm --filter .run setup

real 0m16.168s
user 0m0.435s
sys 0m1.590s
# Average 57.7ms/pc

npm saki (almost unimaginable with a js toolchain)

$ saki run setup

real 0m13.742s
user 0m0.046s
sys 0m0.258s
# Average 49.0ms/pc

Use git bash --filter parameter under Windows, please wrap it with ""

in conclusion

At present, builder has improved significantly, while run is almost the same as node (sometimes even slower).

My generation may continue to try to use golang to rewrite more front-end basic tools. The performance of nodejs does not seem to be available at present.