In this post, we will learn how to use Rust Tonic gRPC crate and implement CRUD with Postgresql database.
You can contact me with Telegram if you need to hire a full stack blockchain dev.
The purpose of it is to help you to have the working Rust Tonic code and start your own project immediately with it.
Prerequisite
- What is gRPC and protocol buffers
- Rust, Postgresql or other databases
- Official Tonic Guide
- gRPC Client
If you haven't installed Rust in your machine, read How to install Rust. We will use Rust Postgresql crate for this post. You should install Postgresql database first if you haven't yet.
I will assume that you are already familiar with gRPC, Rust and Postgresql or other databases. Otherwise, please read the documentations for them thoroughly before you read on.
You may not need gRPC Client at this point. But, I let it here before other contents because it is very useful and it takes long time to install it. Follow the instruction first to save your time later.
Table of Contents
- Project Setup
- Define the CRUD gRPC service
- Prepare Cargo.toml to install dependencies
- Make gRPC server with Tonic
- Implement gRPC Crud Service with Rust Postgresql
- Use gRPC client to test it
- Conclusion
I reused some parts of Official Tonic Guide to make the working flow of this post be similar to it. This will help you find the blog post better.
You can find gRPC relevant source code here.
1. Project Setup
We will first set up data before all others. I hope you already have any sql database installed in your machine. Refer to this sql commands.
Create database for whatever name you want.
CREATE DATABASE grpc OWNER you;
\c grpc;
Then, $psql users < users.sql or manually paste them to your psql console after you login to it.
-- users.sql
CREATE TABLE users(
id VARCHAR(255) PRIMARY KEY,
first_name VARCHAR(255) NOT NULL,
last_name VARCHAR(255) NOT NULL,
date_of_birth Date NOT NULL
);
INSERT INTO users VALUES
('steadylearner', 'steady', 'learner', 'yours');
INSERT INTO users VALUES
('mybirthdayisblackfriday', 'mybirthdayis', 'blackfriday', '2019-11-25');
INSERT INTO users VALUES
('mybirthdayisnotblackfriday', 'mybirthdayis', 'notblackfriday', '2019-11-26');
You can save those data from Postgresql with $pg_dump users > users.sql later.
The database setup is ready. Create a new Rust project to use the data and learn how to use Tonic.
$cargo new user
$cd user
Make .env file first in the folder to proetct your database login information. Refer to this command.
$echo DATABASE_URL=postgres://postgres:postgres@localhost/grpc > .env
2. Define the CRUD gRPC service
We prepared minimal set up for this blog post in the previous part. We will define the gRPC service with the method request and response types. We will use protocol buffers for the user data we made before with Postgresql.
We will make .proto files in proto folder. Use this commands.
$mkdir proto
$touch proto/user.proto
Then, first we will define our package name, which is what Tonic uses when including your protos in the client and server applications. It will be user.
syntax = "proto3";
package user;
Then, we wil define our Crud service. This service will contain the actual RPC calls. We will use them for the Rust Tonic CRUD example.
service Crud { // Use whatever name you want, this is for blog posts and not prouction files.
rpc GetUser (UserRequest) returns (UserReply) {} // becomes get_user in impl functions in Rust files
rpc ListUsers(Empty) returns (Users) {}
rpc CreateUser (CreateUserRequest) returns (CreateUserReply) {}
rpc UpdateUser (UpdateUserRequest) returns (UpdateUserReply) {}
rpc DeleteUser (UserRequest) returns (DeleteUserReply) {}
rpc DeleteUsers (Empty) returns (DeleteUserReply) {}
}
Nothing complicated here. It is a little bit verbose. But, it is to make them more explicit and easily write separate logics for them later.
If you have better options for them or expertise of gRPC, please contact me with Twitter or make a Github issue for this post etc.
Finally, we will make those types we used above in our Crud RPC method. RPC types are defined as messages which contain typed fields. They will be similar to this.
message Empty {}
message UserRequest {
string id = 1;
}
message UserReply {
string id = 1;
string first_name = 2;
string last_name = 3;
string date_of_birth = 4;
}
message CreateUserRequest {
string first_name = 1;
string last_name = 2;
string date_of_birth = 3;
}
message CreateUserReply {
string message = 1;
}
message UpdateUserRequest {
string id = 1;
string first_name = 2;
string last_name = 3;
string date_of_birth = 4;
}
message UpdateUserReply {
string message = 1;
}
message DeleteUserReply {
string message = 1;
}
message Users {
repeated UserReply users = 1;
}
You can see that I used string type for date_of_birth instead of date. I did that because I couldn't find the example to correctly type DATE in protobuf and make it also work with Tonic, protocol buffers and Rust type system.
If you are an expert with this and have a better way, you can help me to correct this.
The complete .proto file for our CRUD project will be similar to this.
// user.proto
syntax = "proto3";
package user;
service Crud {
rpc GetUser (UserRequest) returns (UserReply) {}
rpc ListUsers(Empty) returns (Users) {}
rpc CreateUser (CreateUserRequest) returns (CreateUserReply) {}
rpc UpdateUser (UpdateUserRequest) returns (UpdateUserReply) {}
rpc DeleteUser (UserRequest) returns (DeleteUserReply) {}
rpc DeleteUsers (Empty) returns (DeleteUserReply) {}
}
message Empty {}
message UserRequest {
string id = 1;
}
message UserReply {
string id = 1;
string first_name = 2;
string last_name = 3;
string date_of_birth = 4;
}
message CreateUserRequest {
string first_name = 1;
string last_name = 2;
string date_of_birth = 3;
}
message CreateUserReply {
string message = 1;
}
message UpdateUserRequest {
string id = 1;
string first_name = 2;
string last_name = 3;
string date_of_birth = 4;
}
message UpdateUserReply {
string message = 1;
}
message DeleteUserReply {
string message = 1;
}
message Users {
repeated UserReply users = 1;
}
Finding working Rust code is not easy and becomes even harder when you want to satisfy Rust compiler, protocol buffers type specification, Rust Postgresql at the same time.
If you want to make your own Rust Tonic project with other proto files later, compile
Tonic CRUD Example by Steadylearner first to collect binary files and then start by editing small parts of the Rust code and protobuf definitions in there.
3. Prepare Cargo.toml to install dependencies
We set up the proejct and made the protobuf file to use gRPC with Rust. Therefore, we can write Rust code for it with Tonic.
We will first prepare the dependencies for them with Cargo.toml.
[package]
name = "rust-tonic-crud-example"
version = "0.1.0"
authors = ["www.steadylearner.com"]
edition = "2018"
[dependencies]
tonic = { version = "0.1.0-alpha.4", features = ["rustls"] }
bytes = "0.4"
prost = "0.5"
prost-derive = "0.5"
prost-types = "0.5.0"
tokio = "=0.2.0-alpha.6"
futures-preview = { version = "=0.3.0-alpha.19", default-features = false, features = ["alloc"]}
async-stream = "0.1.2"
http = "0.1"
tower = "=0.3.0-alpha.2"
serde = "1.0.101"
serde_json = "1.0.41"
serde_derive = "1.0.101"
console = "0.9.0"
# Database(Postgresql)
postgres = { version = "0.15.2", features = ["with-chrono"] }
dotenv = "0.15.0"
chrono = "0.4.9"
uuid = { version = "0.8.1", features = ["serde", "v4"] }
# Help you use gRPC protobuf files in Rust.
[build-dependencies]
tonic-build = "0.1.0-alpha.4"
There are many dependencies for this simple project. But, it will be easy if you think this part
# Database(Postgresql)
postgres = { version = "0.15.2", features = ["with-chrono"] }
dotenv = "0.15.0"
chrono = "0.4.9"
uuid = { version = "0.8.1", features = ["serde", "v4"] }
is for the Rust Postgresql and others are for Tonic.
We include tonic-build to make our client and server side gRPC code.
If you haven't used gRPC or tonic it may confuse you. But, every languages that use gRPC integration have the similar process to use proto buffer definitions with them.
With Rust Tonic, we need to include it in build process of our application. We will setup this with build.rs at the root of your crate.
fn main() -> Result<(), Box<dyn std::error::Error>> {
tonic_build::compile_protos("proto/user.proto")?;
Ok(())
}
Nothing complicated here. You just need to modify the file name from other examples. This makes tonic-build to compile your protobufs files to work with your Rust gRPC projects.
It automatically builds some Rust modules you can use later with your Rust code depending on the protobuf definitions you used in your protobuf files. You can verify this with the Rust code you will read later.
If you want more details, please refer to tonic-build.
4. Make gRPC server with Tonic
The preparation process ended with the previous part. Finally, we will write our own Rust code. Start with building Rust Tonic gRPC server similar to this.
// main.rs
extern crate postgres;
extern crate dotenv;
extern crate chrono;
// 1.
pub mod user {
tonic::include_proto!("user");
}
use tonic::{transport::Server};
// 1.
use user::{
server::{CrudServer},
};
extern crate uuid;
extern crate console;
use console::Style;
mod db_connection; // 2.
mod service;
use crate::service::User;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "[::1]:50051".parse().unwrap(); // 3.
let user = User::default();
let blue = Style::new()
.blue();
println!("\nRust gRPC Server ready at {}", blue.apply_to(addr)); // 4.
Server::builder().serve(addr, CrudServer::new(user)).await?;
Ok(())
}
It will be the entry point when we start our Rust Tonic gRPC server. Most of them are to define dependencies, modules you will use.
The file is simple but there are some points you need to know.
1. This is where you can use auto generated codes from tonic_build::compile_protos("proto/user.proto")?; and tonic-build = "0.1.0-alpha.4".
You can use modules made from it similar to this and handlers we will build with models.rs.
use user::{
server::{CrudServer},
};
2. The contents of db_connection file will be similar to this.
use postgres::{Connection, TlsMode};
use dotenv::dotenv;
use std::env;
pub fn establish_connection() -> Connection {
dotenv().ok();
let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
Connection::connect(database_url, TlsMode::None).unwrap()
}
For it can be reused many times without modifications, we will separate this to function and can include it wherever we want.
3. This is code from the Tonic authors. You should use it without http prefix when you use CURL and gRPC client.
4. We use console::Style; with println! to easily visit the server with browsers and send CURL commands with it.
5. Implement gRPC Crud Service with Rust Postgresl
In this part, we will make some handlers in service.rs while we follow the definitions we made in user.proto file. The entire code will be similar to this.
The code snippet is long. This is to implement every gRPC CRUD operation with Postgresql database relevant logics. You may read only parts you want.
use chrono::*;
use uuid::Uuid;
use crate::db_connection::establish_connection;
use tonic::{Request, Response, Status};
// Compare it with user.proto file, imported from the main.rs file
use crate::user::{
server::Crud, CreateUserReply, CreateUserRequest, DeleteUserReply, Empty, UpdateUserReply,
UpdateUserRequest, UserReply, UserRequest, Users,
};
#[derive(Default)]
pub struct User {}
#[tonic::async_trait]
impl Crud for User {
// Compare it with the Crud service definition in user.proto file
// The method GetUser becomes get_user etc
async fn get_user(&self, request: Request<UserRequest>) -> Result<Response<UserReply>, Status> {
println!("Got a request: {:#?}", &request);
// request is private, so use this instead to get the data in it.
let UserRequest { id } = &request.into_inner();
let conn = establish_connection();
// 1.
let rows = &conn
.query("SELECT * FROM users WHERE id = $1", &[&id])
.unwrap();
// println!("{:#?}", rows);
// println!("{:#?}", rows.get(0));
// https://docs.rs/postgres/0.17.0-alpha.1/postgres/row/struct.Row.html
let row = rows.get(0);
println!("{:#?}", &row);
// 2.
let date_of_birth: NaiveDate = row.get(3);
let reply = UserReply {
id: row.get(0),
first_name: row.get(1),
last_name: row.get(2),
// 2.
date_of_birth: date_of_birth.to_string(),
};
Ok(Response::new(reply))
}
async fn list_users(&self, request: Request<Empty>) -> Result<Response<Users>, Status> {
println!("Got a request: {:#?}", &request);
let conn = establish_connection();
// 3.
let mut v: Vec<UserReply> = Vec::new();
for row in &conn.query("SELECT * FROM users", &[]).unwrap() {
let date_of_birth: NaiveDate = row.get(3);
let user = UserReply {
id: row.get(0),
first_name: row.get(1),
last_name: row.get(2),
date_of_birth: date_of_birth.to_string(),
};
v.push(user);
}
let reply = Users { users: v };
Ok(Response::new(reply))
}
// Test with create_users, Rust compiler shows errors to help you.
async fn create_user(
&self,
request: Request<CreateUserRequest>,
) -> Result<Response<CreateUserReply>, Status> {
println!("Got a request: {:#?}", &request);
// 4.
let user_id = Uuid::new_v4().to_hyphenated().to_string();
let CreateUserRequest {
first_name,
last_name,
date_of_birth,
} = &request.into_inner();
// 5.
let serialize_date_of_birth = NaiveDate::parse_from_str(date_of_birth, "%Y-%m-%d").unwrap(); // String to Date
let conn = establish_connection();
// 6.
let number_of_rows_affected = &conn.execute(
"INSERT INTO users (id, first_name, last_name, date_of_birth) VALUES ($1, $2, $3, $4)",
&[
&user_id,
&first_name,
&last_name,
&serialize_date_of_birth,
]
)
.unwrap();
let reply = if number_of_rows_affected == &(0 as u64) {
CreateUserReply {
message: format!(
"Fail to create user with id {}.",
&user_id
),
}
} else {
CreateUserReply {
message: format!(
"Create {} user with id {}.",
&number_of_rows_affected, &user_id
),
}
};
Ok(Response::new(reply))
}
async fn update_user(
&self,
request: Request<UpdateUserRequest>,
) -> Result<Response<UpdateUserReply>, Status> {
println!("Got a request: {:#?}", &request);
let UpdateUserRequest {
id,
first_name,
last_name,
date_of_birth,
} = &request.into_inner();
// 3.
let serialize_date_of_birth = NaiveDate::parse_from_str(date_of_birth, "%Y-%m-%d").unwrap(); // String to Date
let conn = establish_connection();
let number_of_rows_affected = &conn
.execute(
"UPDATE users SET first_name = $2, last_name = $3, date_of_birth = $4 WHERE id = $1",
&[
&id,
&first_name,
&last_name,
&serialize_date_of_birth,
]
)
.unwrap();
let reply = if number_of_rows_affected == &(0 as u64) {
UpdateUserReply {
message: format!("Fail to update the user with id {}.", id),
}
} else {
UpdateUserReply {
message: format!("Update {} user with id {}", &number_of_rows_affected, &id),
}
};
Ok(Response::new(reply))
}
async fn delete_user(
&self,
request: Request<UserRequest>,
) -> Result<Response<DeleteUserReply>, Status> {
println!("Got a request: {:#?}", &request);
let UserRequest { id } = &request.into_inner();
let conn = establish_connection();
let number_of_rows_affected = &conn
.execute("DELETE FROM users WHERE id = $1", &[&id])
.unwrap();
let reply = if number_of_rows_affected == &(0 as u64) {
DeleteUserReply {
message: format!("Fail to delete the user with id {}.", id),
}
} else {
DeleteUserReply {
message: format!("Remove the user with id {}.", id),
}
};
Ok(Response::new(reply))
}
async fn delete_users(
&self,
request: Request<Empty>,
) -> Result<Response<DeleteUserReply>, Status> {
println!("Got a request: {:#?}", &request);
let conn = establish_connection();
let rows = &conn.query("DELETE FROM users", &[]).unwrap();
let reply = DeleteUserReply {
message: format!("Remove {} user data from the database.", rows.len()),
};
Ok(Response::new(reply))
}
}
There were many lines of Rust code here. So it may be complicated to start with. I want you to test your project with get_user and list_users parts first. Then, you can improve it while you refer to these separate comments.
1. When you read Rust Postgresql documentation and examples, you can see that there are execute and query API to use Postgresql SQL commands. The difference is that execute retunrs the number of rows modified query returns data(returning the resulting rows"). You should find when to use them depending on your needs.
2. When you use Rust Postgresql, you may see the error message similar to this.
cannot infer type
the trait `postgres::types::FromSql` is not implemented for
Then, you should type the data with the definition that help your Rust code compatible with postgresql with chrono.
let date_of_birth: NaiveDate = row.get(3);
We used the string type for date_of_birth in user.proto file. We should turn date_of_birth to string similar to this.
date_of_birth: date_of_birth.to_string(),
This is the cost we have to pay to make it work with your Rust code, Postgresql database and Protobuf at the same time. You may find the better way.
3. We use imperative way to make the list of users following the documentation from the author. You should easily infer that repeated part of proto becomes vec in Rust.
message Users {
repeated UserReply users = 1;
}
Then, you also need to know that Rust postgresql crate requires you to include empty &[] when you have no values to pass in your sql commands.
&conn.query("SELECT * FROM users", &[])
4. We make random id for users with Rust uuid API. You should include v4 relevant parts in your cargo.toml to make it work.
uuid = { version = "0.8.1", features = ["serde", "v4"] }
5. We use API from chrono to make the string to DATE type to make it compatible with Postgresql.
6. We use let number_of_rows_affected = &conn.execute and its relevant logic to handle the database result from the gRPC client request. You can see that the similar logic is used in update_user, delete_user, delete_users.
I hope you read the entire code of the project and its relevant documentations. If you haven't tested the project, do it with cargo run --release and see the result similar to this.
Rust gRPC Server ready at [::1]:50051
Then, you can test it with CURL with $curl [::1]:50051.
If it worked well, it should show this.
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
Then, you can write Tonic gRPC client code similar to this.
pub mod user {
tonic::include_proto!("user");
}
use user::{client::CrudClient, UserRequest};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut client = CrudClient::connect("http://[::1]:50051")?;
let request = tonic::Request::new(UserRequest {
id: "steadylearner".into(),
});
let response = client.get_user(request).await?;
println!("RESPONSE={:?}", response);
let user_date_of_birth = &response.into_inner().date_of_birth;
println!("{}", user_date_of_birth);
Ok(())
}
It will be easy to find how this works if you read Official Tonic Guide already. Make it compile if you want and back to improve it later after you test all the gRPC end points with gRPC Client.
6. Use gRPC client to test it
<a href="https://raw.githubusercontent.com/uw-labs/bloomrpc/master/resources/blue/256x256.png" title="BloomRPC">
<img src="https://raw.githubusercontent.com/uw-labs/bloomrpc/master/resources/blue/256x256.png">
</a>
I want you already installed gRPC Client in your machine. You should have BloomRPC version.AppImage executable file in your release folder.
You may manually execute it or refer to the process similar to this with your editor.
Use pwd to find the location of the gRPC client and $vim ~/.bashrc to include alias similar to this.
# gRPC
alias grpc-client="grpc/bloomrpc/release/'BloomRPC version.AppImage'"
and use this command.
$source ~/.bashrc
Then, you can use it with $grpc-client whenever you want.
It will show the desktop application similar to this from the official repository.
If you used graphql before, you will find it is very similar to use graphiql to test its end points.
You will need to include your proto files first. For example, user.proto we made.
Then, it will automatically show the methods you can use.
When you click each methods, it will give the default value at the editor part. You can send requests with them or use your custom values.
For this example, you should have cautions when you define value for "date_of_birth" part. You should use correct DATE type string for Rust and Postgresql.
When the process pauses, you can stop it easily by clicking the same button you used to send gRPC request.
Refer to the request examples I let it here before you use it.
GetUser
{
"id": "steadylearner"
}
ListUsers
{}
CreateUser
{
"first_name": "steady",
"last_name": "learner",
"date_of_birth": "%Y-%m-%d"
}
UpdateUser
{
"id": "random-id",
"first_name": "steadylearner",
"last_name": "rust developer",
"date_of_birth": "use-numbers-instead"
}
DeleteUser
{
"id": "steadylearner"
}
DeleteUsers
{}
Test the gRPC server end points with your own code. Then, write more complicated Rust Tonic client in separate Rust files.
You can also use it with other Rust servers to make microservices.
7. Conclusion
I hope you made it work. You can edit the protobuf definition for your own project and write more Rust code to handle database relevant logics.
Rust and Tonic helped me to learn and write better gRPC codes. But, it was difficult to find the working examples with database integration and want this post be helpful for others.
If you want the latest contents, follow me here, Twitter or star Rust Full Stack.
If you need to hire a developer, you can contact me.
Thanks.
Top comments (4)
Unfortunately, this code doesn't work with more recent versions of the tonic and postgres libraries.
You will hit the error:
Because postgres driver executes
runtime.block_oninside theblock_oncoming from#[tokio:main]Thanks for letting the comment to help others. I will upgrade it later if I have to use Rust with gRPC again. Until then, it will be better to read this post only to learn the workflow to use gRPC with Rust and Tonic.
Hi, can you please share the source code with us? Thanks
You can find it here. I will update the link at the post also.
github.com/steadylearner/Rust-Full...
Not sure this is the same code used for this blog post because it had passed a few years. But, you will be able to test it with the dependencies in there.
It will be better to test with github.com/hyperium/tonic/tree/mas...