When I started working with gRPC in Node.js, I followed Google's dynamically generated code base to create my servers and clients. And for basic development, this works really well. At least in a mono repository using relative paths to import the proto files. As we move closer to deployment though, a glaring problem has emerged.
A growing collection of proto files - many of which are sharing messages and models - that need to be imported into various gRPC servers and clients. These servers and clients will be hosted in various Kubernetes clusters.
My solution:
Create an npm package for each service cluster to house the needed proto files.
At first, I thought I would just write a bash script to copy the needed service and message/model files into an npm package and call it a day. However, I know that you can generate actual server and client from the proto files (in some languages), and I want to be ready for that when it comes to Node.js. Therefore, I decided that my npm packages would hold protoc
generated files.
To do this, I needed to have protoc
and grpc-tools
installed on my development machine. (Note: grpc-tools
needs to be installed globally for this to work).
I decided to make a bash script file to handle the actual generation. I figure we can use/modify this file later to use in a more automated process.
- I remove any existing files. For Node.js,
protoc
creates two files, both of which end in_pb.js
. So I used a wildcard to select and delete all files ending that way.
rm -f <path to target>/*_pb.js
- Then I actually generate the files:
grpc_tools_node_protoc --js_out=import_style=commonjs,binary:<path to export directory> --grpc_out=generate_package_definition:<path to export directory> <proto file>
Note: I found that the script needed to change directories and run in the directory housing the proto file. I was not able to figure out how to use relative paths to that directory, though I do believe there is a way to do it.
The --grpc_out=
flag supports two options:
grpc_js
: Generates code withrequire('@grpc/grpc-js')
instead ofrequire('grpc')
generate_package_definition
: Generates code that does not require any gRPC library, and instead generatesPackageDefinition
objects that can be passed to theloadPackageDefinition
function provided by both thegrpc
and@grpc/grpc-js
libraries.
I tried both flags and found that the generate_package_definition
worked best in both my servers and clients.
My export directory is the npm package I had created for the cluster that needed the proto file. I am now able to use the package - and therefore the files - in both my servers and clients.
I am very pleased with the results. The script files does need to be manually run with any proto file changes, but I can automate that at a future date.
Top comments (0)