loading...

Moving to GRPC from REST

dmevanct profile image Evan Haston Updated on ・5 min read

Why move from GRPC to go?

Over the past few weeks I have decided to move from REST to GRPC. Why would I do that? Well to start GRPC is brand new and who doesn't want to try the cool new technology? If you are unaware of what GRPC is it's a remote procedural call framework. RPC differs from rest because the messages are sent in binary and not text like JSON. This means a much smaller network footprint for services especially at scale.

Scaling

Imagine going from 5 requests a second to 10,000 requests per second. With JSON you have a large network overhead because you have to send the JSON (text format) to the service 10,000 times. Because of the smaller size the RPC message you have a much smaller network footprint. I believe it's about 1/3 of the size of the same json request.

GRPC uses http2 which allows you to create one connection for multiple messages. This comes in handy when you are streaming a lot of data using the streaming feature of GRPC. I will get into streaming in a moment.

Streaming

One of the big benefits of GRPC is streaming. Streaming is the ability to send multiple messages as part of a request. GRPC has a few different ways you can stream

  • Server side streaming
  • Client side streaming
  • Bidirectional streaming

Server side streaming is when the client sends a single message and the server responds with many messages you will see that done in the example below.

Client side streaming is when the client sends multiple messages and the server sends back a single message. This comes in handy when you want the server to process multiple pieces of data and return a single piece maybe calculated from all the sent pieces.

Bidirectional streaming is when the client and the server both send multiple messages back and forth. I do not show this example as it gets a little complex as it uses goroutines and mutexes outside of the scope of basic golang.

You don't have to stream anything you can use GRPC to send one message and receive one message. But streaming comes in handy quite often.

Interoperability

In many environments people make use of different programing languages one team may code in golang while the other uses java or python. One of the big problems with traditional rest services is getting these clients to talk to each other effectively. With GRPC there is interoperability between many languages including JAVA, GO, C#, C++, PHP, and WEB (Node). You define the service with a simple service definition proto file and use it any service you want to establish intercommunication between.

Service Definitions Implementation

Services are easy to define and even easier to use. Instead of having to remember what to send via json the service tells you exactly what to send and what to expect back. This makes it much easier to write services to utilize other services especially in a microservice architecture.

Here is an example of a fair food service you request food by the foodtype ex: tai and by the fair name ex: Durham Fair and you get the food name price vendor and location back. This is streamed back to the client with a message for each food.

syntax = "proto3";


package FairService;

service FairFoodService {

    rpc GetFairFoodPrices(FoodType) returns (stream FairFoodPrice) {}
}


message FoodType {
    string foodtype = 1;
    string fairname = 2; 
}


message FairFoodPrice {
    string foodname = 1;
    string price = 2;
    string vendor = 3;
    string location = 4;


}

After you define the service you create the code file in the respective language

protoc -I FairService FairService/FoodService.proto  --go_out=plugins=grpc:. 

Foodservice.proto being the proto file create above. Once you create the go file it's pretty easy to implement.

Server fairfood.go:



func GetFairFood(fairname, foodtype string,  stream FairFoodService_GetFairFoodPricesServer) error {

    var foodname string
    var price string;
    var vendor string;
    var location string;

// STARR THE DB CONNECTION

    db_ip := Database.DatabaseByEnvironment("fairfood")
    log.Println(db_ip)

    db := Database.DatabaseInitAllHost("/etc/dm", "FairService", "FairService.username", "FairService.password", db_ip)

    foodfetch, err := db.Begin()
    if err != nil {
        log.Fatal("There was an error beginning", foodfetch)
    }

    defer foodfetch.Rollback()
// QUERY DATABASE FOR FOOD TYPE
    ftf, err := foodfetch.Query("SELECT foodname, price, vendor, location FROM fairvendors.tbl_food where foodtype = " + foodtype + "fairname = " + fairname)
    if err != nil {
        log.Println("There was an error with the prepare statement")
    }
// DEFER CLOSING DATABASE CONNECTION UNTIL AFTER QUERY IS COMPLETE
    defer ftf.Close()
// ITERATE OVER ROWS IN DATABASE TO RETURN MULTIPLE VALUES
    for ftf.Next() {
        err := ftf.Scan(&foodname, &price, &vendor, &location)
        if err != nil {
            log.Println("There was an error with the query")
        }
// STREAM EACH OF THE VALUES BACK TO THE CLIENT
        err = stream.Send(&FairFoodPriceResp{
            Foodname:             foodname,
            Price:                price,
            Vendor:               vendor,
            Location:             location,
            XXX_NoUnkeyedLiteral: struct{}{},
            XXX_unrecognized:     nil,
            XXX_sizecache:        0,
        })
        if err != nil {
            log.Printf("There was an error sending the stream: %v", err)
        }

    }
    return nil
}

Server main.go:

package main

import (
    "github.com/DMEvanCT/FairVendorService/FairFood"
    "google.golang.org/grpc"
    "google.golang.org/grpc/reflection"
    "log"
    "net"
)

// SET THE VARIABLE PORT = 50051
const (
    port = ":50051"
)

type server struct {}


func (*server) GetFairFood(req *FairFood.FoodTypeReq, stream  FairFood.FairFoodService_GetFairFoodPricesServer) error {
    fairname := req.GetFairname()
    foodtype := req.GetFoodtype()
    FairFood.GetFairFood(fairname, foodtype, stream)

    return nil
}

// Start the server under main function
func main() {
    lis, err := net.Listen("tcp", port)
    if err != nil {
        log.Fatalf("failed to listen %v", err )
    }
    s := grpc.NewServer()
    pb.RegisterFilevisServiceServer(s, &server{})
    reflection.Register(s)
    if err := s.Serve(lis); err != nil {
        log.Printf("Failed to serve %v", err)
    }


}

Client fairfoodclient.go:

This example receives the stream and prints out the output in the terminal. Obviously you don't need to print the output but it's good for demonstration.

package FairFood

import (
    "context"
    "fmt"
    "io"
    "log"
)

func  GetRunningFiles(fairname, foodtype string, c FairFoodServiceClient) {
// SET THE request variable = fairname and foodtype to send to server
    request := &FoodTypeReq{
        Fairname:          fairname,
        Foodtype:          foodtype,
        XXX_NoUnkeyedLiteral: struct{}{},
        XXX_unrecognized:     nil,
        XXX_sizecache:        0,
    }

// Send the Request to the server
    resStream, err := c.GetFairFoodPrices(context.Background(), request)
    if err != nil {
        log.Fatalf("There was a problem with the request: %v", err )
    }

// Get the response from the server 
    for {
        msg, err := resStream.Recv()
        if err == io.EOF {
            // reached the end of messages
            break
        }
        if err != nil {
            log.Fatalf("There was an error rec request: %v", err)
        }
        foodname := msg.GetFoodname()
        foodprice := msg.GetPrice()
        vendor := msg.GetVendor()
        location := msg.GetLocation()


        fmt.Println( "FoodName: " + foodname + " | Food Price: "  + foodprice + " | Vendor: " + vendor + " | Location: " + location )
    }


}

Client grpcimp.go

I use this file to start the grpc client and run the code above.

func InitGRPCGetFood(fairname, fairfoodtype string) {
    // Intermidiate Cert used for cert that fronts with nginx.
    sslcert := "/etc/dm/CACert.crt"
    creds, sslErr := credentials.NewClientTLSFromFile(sslcert, "")
    if sslErr != nil {
        log.Fatalf("There was an error reading ssl cert: %v", sslErr)
        return
    }
// Set the credentials
    opts := grpc.WithTransportCredentials(creds)
/Start the connection to grpc
    conn, err := grpc.Dial(address, opts)
    if err != nil {
        log.Fatalf("There was an error connecting to grpc stream: %v", err)
    }

// Wait to close the connection until after the function ends
    defer conn.Close()
// Set up the client
    cinit := NewFilevisServiceClient(conn)
//Run the GetFairFood function with client


    FairFood.GetFairFood(fairname, fairfoodtype, cinit)

}

Client main.go:

Finally to run the code in the main function.


package main

//Run the GRPC client with "Durham Fair" as the fair and "TAI" as the food type. 

func main() {
   Fairfood.InitGRPCGetFood("Durham Fair", "Tai")

}

I hope you have learned something from this and will go and check out grpc! If you have any suggestions or improvements I can make to this article please let me know!

If you want to learn more about grpc go to https://grpc.io

Discussion

pic
Editor guide