This post was originally published in adityasridhar.com
This Article will give a basic introduction to GRPC and Protocol Buffers. Following that I will be showing how to use GRPC and Protocol Buffers in a NodeJS Application
What is GRPC
GRPC is a Open Source High Performance RPC Framework
So what exactly does a RPC do?
Take the following Example
function getEmployee() {
return "ABCD";
}
function greetEmployee()
{
let employee = getEmployee();
console.log("Greet",employee)
}
Here we have a function getEmployee which returns an Employee Name and another function greetEmployee which Calls getEmployee and gets the employee's Name and then prints a Greeting.
Here greetEmployee calling getEmployee is a regular function call.
Now if getEmployee and greetEmployee functions are located in different address spaces, or they are located in 2 different hosts which are separated by the network, then the function call is called a Remote Procedure Call. Here the System which has the getEmployee function acts like a Server and the System which has the greetEmployee function acts like a Client.
What is a Protocol Buffer
Protocol Buffer is the Interface Definition Language which is used by default in GRPC.
- It helps to define the various services offered by a server.
- It helps to define the Structure of the Payloads used in the System
- It helps to serialize the message ( To a Special Binary Format ) and send it across the wire between the server and client.
We will see how to use protocol buffers when we are working on the NodeJS application later in this article.
What are the Different Types of RPC Supported
Unary RPC
This is the Simplest RPC available. Here the Client sends a Request Message to the Server. The Server processes the request and then sends a Response message back to the Client.
In this Article this is the grpc we will be focussing on.
Server Streaming RPC
In this RPC, the client sends a request message to the server, and the server sends a sequence of messages back to the Client in a stream fashion.
Client Streaming RPC
In this RPC, the client sends a sequence of messages to the server in a stream fashion. The server then processes all these requests and then sends a response message back to the client.
Bidirectional Streaming RPC
In this RPC, the client sends a sequence of messages to the server in a stream fashion. The server then processes the request and then sends a sequence of messages back to the client in stream fashion.
How to use GRPC and Protocol Buffers in NodeJS
Create a folder called as grpc-nodejs-demo and Initialize nodejs in it using the following commands
mkdir grpc-nodejs-demo
cd grpc-nodejs-demo
npm init
This will create a package.json file. To know more about NodeJS you can read my other article here
Modify the package.json file
Replace the package.json file with the following
{
"name": "grpc-nodejs-demo",
"version": "1.0.0",
"description": "",
"main": "server.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"dependencies": {
"@grpc/proto-loader": "^0.1.0",
"grpc": "^1.11.0",
"lodash": "^4.6.1"
},
"author": "Name",
"license": "MIT"
}
Here we are adding 3 dependencies
- @grpc/proto_loader and grpc dependencies will help us use GRPC and Protocol Buffers in the Application.
- lodash is a general utility dependency. It will help simplify some code logic
Once the package.json file has been updated. Run the following command to install the dependencies
npm install
Defining the Protocol Buffer
In this Example we will build a service which will take employee ID as the input, and give the employee details as the output.
The Service Interface and payloads needed will be specified in a protocol buffer file. Protocol buffer files have an extension of .proto
Now let's create the .proto file.
Create a folder called as proto within the project. Inside the proto folder, create a file with the name employee.proto and add the following code to it
syntax = "proto3";
package employee;
service Employee {
rpc getDetails (EmployeeRequest) returns (EmployeeResponse) {}
}
message EmployeeRequest {
int32 id = 1;
}
message EmployeeResponse{
EmployeeDetails message = 1;
}
message EmployeeDetails {
int32 id = 1;
string email = 2;
string firstName = 3;
string lastName = 4;
}
So what exactly have we done here?.
syntax = "proto3"; indicates that we want to use Protocol Buffer version 3.
package employee; indicates that we are creating a package called as employee within which we will define our services
service Employee {
rpc getDetails (EmployeeRequest) returns (EmployeeResponse) {}
}
The Above script tells that We are creating a Service called as Employee. Within this service we are creating a function ( rpc ) called as getDetails which accepts input of type EmployeeRequest and provides response in the format EmployeeResponse
Next we need to define EmployeeRequest and EmployeeResponse. This is done in the following script
message EmployeeRequest {
int32 id = 1;
}
message EmployeeResponse{
EmployeeDetails message = 1;
}
message EmployeeDetails {
int32 id = 1;
string email = 2;
string firstName = 3;
string lastName = 4;
}
Here we see that the message EmployeeRequest has a single field of type int32 and name id. The number 1 assigned here is a field number and it helps during encoding and decoding of the message. Every field defined should have a unique field number
We also see that EmployeeResponse has a custom field of type EmployeeDetails and name message having field number of 1. This means that even EmployeeDetails has to be defined which is shown above as well.
EmployeeDetails has 4 fields comprising types int32 and string. All of them have unique field numbers
Field numbers between 1 -15 use 1 byte of space during encoding. and field numbers from 2 - 2047 uses 2 bytes for encoding and hence will take up more space. So try to design in such a way that the field numbers are between 1 - 15 as much as possible
Creating the GRPC Server
Create a file called as server.js
First Let us include all the libraries we need and also define the location where the .proto file is present
const PROTO_PATH = __dirname + '/proto/employee.proto';
const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
const _ = require('lodash');
Next we need to load the .proto file. This is done using the protoLoader libary loadSync method.
let packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
Next from the loaded proto file package Definition we need to get the package we need. This is done using the following script
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
here we are getting the employee package into employee_proto variable.
employee_proto will now have all the proto definitions.
Next we need to create some dummy employees data for the server to work with. Create a file called as data.js and add the following script into it
let employees = [{
id: 1,
email: "abcd@abcd.com",
firstName: "First1",
lastName: "Last1"
},
{
id: 2,
email: "xyz@xyz.com",
firstName: "First2",
lastName: "Last2"
},
{
id: 3,
email: "temp@temp.com",
firstName: "First3",
lastName: "Last3"
},
];
exports.employees = employees;
Next we need to import data.js into server.js. Add the following script in server.js for this
let {employees} = require('./data.js');
So employees will have the list of employees with their id, email, firstName and lastName
The next piece of script creates and starts the GRPC Server.
function main() {
let server = new grpc.Server();
server.addService(employee_proto.Employee.service, {getDetails: getDetails});
server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure());
server.start();
}
let server = new grpc.Server(); is the script which creates a new GRPC Server
In the .proto file we notice that we have a function called as getDetails inside Employee Service.
server.addService(employee_proto.Employee.service, {getDetails: getDetails}); is the script in which we add the Service implementation. This script says that we are adding the getDetails function in employee_proto.Employee Service. And Then We are adding this Service to the Server.
server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure()); is the script which tells that the server will start in port 4500 and have no Authentication
server.start(); is the script which actually starts the server.
The main thing pending now is to implement getDetails function. The below script shows the implementation
function getDetails(call, callback) {
callback(null,
{
message: _.find(employees, { id: call.request.id })
});
}
here call has the request parameters and callback is where we need to define the implementation.
Inside callback we have message: _.find(employees, { id: call.request.id }) which says the following
- Get the employee ID from Input - call.request.id
- search the employees list to find the Employee who has that Id
- Return that Employee Details
This completes the Server implementation. Here is the complete script for server.js
const PROTO_PATH = __dirname + '/proto/employee.proto';
const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
const _ = require('lodash');
let packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
let {employees} = require('./data.js');
function getDetails(call, callback) {
callback(null,
{
message: _.find(employees, { id: call.request.id })
});
}
function main() {
let server = new grpc.Server();
server.addService(employee_proto.Employee.service, {getDetails: getDetails});
server.bind('0.0.0.0:4500', grpc.ServerCredentials.createInsecure());
server.start();
}
main();
Creating the GRPC Client
Create a file called as client.js
Copy the following script into client.js
const PROTO_PATH = __dirname + '/proto/employee.proto';
const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
let packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
The Above script loads the employee package into employee_proto variable in the same way we saw in server.js
Next we need a way in which we can call the RPC. In this case we need to be able to call the getDetails function which is implemented in the Server.
For this we need to create a stub in the client. This is done using below script.
let client = new employee_proto.Employee('localhost:4500',
grpc.credentials.createInsecure());
This client Stub will help us call the getDetails function which is defined in Employee Service which runs on the Server. The Server in turn runs on port 4500. The Line of code also indicates that there is no Authentication used
finally we can call the getDetails function using the following script
let employeeId = 1;
client.getDetails({id: employeeId}, function(err, response) {
console.log('Employee Details for Employee Id:',employeeId,'\n' ,response.message);
});
As mentioned earlier the client stub helps us call the getDetails function in the Server like a normal function call. To this we pass the employeeId as the input.
Finally the Response comes in the response variable. We are then printing the response message.
The complete client.js code is given below
const PROTO_PATH = __dirname + '/proto/employee.proto';
const grpc = require('grpc');
const protoLoader = require('@grpc/proto-loader');
let packageDefinition = protoLoader.loadSync(
PROTO_PATH,
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
let employee_proto = grpc.loadPackageDefinition(packageDefinition).employee;
function main() {
let client = new employee_proto.Employee('localhost:4500',
grpc.credentials.createInsecure());
let employeeId;
if (process.argv.length >= 3) {
employeeId = process.argv[2];
} else {
employeeId = 1;
}
client.getDetails({id: employeeId}, function(err, response) {
console.log('Employee Details for Employee Id:',employeeId,'\n' ,response.message);
});
}
main();
Running The Server and Client
Running the Server
Open a command prompt and run the Server using the following command
node server.js
This will start the server
Open a new Command prompt and run the Client using the following command
node client.js
When we run the client. It will print the following output
Employee Details for Employee Id: 1
{ id: 1,
email: 'abcd@abcd.com',
firstName: 'First1',
lastName: 'Last1' }
So here the client has called the getDetails function in the server. The Client has passed the input for employeeId as 1. Then the server went over the data, found the employee with id as 1 and returned that Employee data back to the client.
In This example we have run the client and server in a single machine. But you can also test this out by having the server and client in different hosts.
Code
The Code discussed in this article can be found here
References
GRPC official Documentation : https://grpc.io/
Protocol Buffers Proto3 documentation : https://developers.google.com/protocol-buffers/docs/proto3
Congrats 😄
You now know what is GRPC And Protocol Buffers. You also know how to use them in a NodeJS Application
Happy Coding 😄
Top comments (0)