Protoc Plugins in Go: gRPC-REST Gateway From Scratch

gRPC-REST gateway is one of the most popular projects in the gRPC-Ecosystem. Since I really like to understand how things work under the hood and how they are implemented, I decided to try and build a minimal gRPC-REST Gateway from scratch. If you are interested as well, make sure to read through the article as I’ve learned a bunch while trying to code it up.

I tend to make lots of mistakes on a daily basis, so if you’re one of the many that encountered one, I’d be grateful if you would correct me.

Introduction

Choosing a gRPC-powered backend is not an uncommon thing nowadays, considering all the benefits it might provide if used in the proper environment, especially in a micro-service architecture. But what if you’re planning to migrate your currently REST backend services to gRPC? You’re probably going to face some issues. Since gRPC relies on HTTP trailers to operate, it’s a bit tricky to call gRPC backend services from a JS frontend client. Feel free to read further about the curious, yet questionable use of HTTP Trailers in the design of gRPC in Carl’s Blog, as well as Akshayshah’s. You probably don’t have many options. The best one might be the gRPC-Web project which enables you to proxy an HTTP/1.1 request (made by gRPC-Web) to an HTTP/2-gRPC backend through an Envoy proxy (more on this in the gRPC-Web official documentation). But for various reasons (e.g. migration, development and maintenance costs) you might not have the luxury to do so. What are you options then? Introducing gRPC-REST Gateway.

gRPC-REST Gateway

What if you could deploy an HTTP/1.1 server, accepting REST requests from the frontend, converting them to gRPC requests, calling the gRPC services and returning their responses in a format that the REST clients will expect? It looks pretty similar to the gRPC-Web request flow, but now your frontend is completely oblivious about anything gRPC-related happening behind the curtains. Removing the need for an Envoy proxy or any proxy for that matter, it will arguably reduce the related costs as well.

In the incoming sections, we’re going to implement a minimal version of the well-known gRPC-REST Gateway. We will be using a top-down approach as I personally understand details and reasonings easier and better when I know the high-level concepts and requirements.

Implementation

The first thing that we might ask ourselves, is what behaviour do we expect from our mini gRPC-REST gateway? Let’s say there is a gRPC backend running on port 50051 (this is easily generalizable to a service running in a Kubernetes cluster). Here’s a .proto file as well as the code needed for a simple gRPC server:

    syntax = "proto3";

    package post;

    option go_package = "github.com/HomayoonAlimohammadi/mini-grpc-gateway/pb/post";

    message Empty {}

    message GetPostResponse {
        string title = 1;
        string description = 2;
    }

    service PostService {
      rpc GetPost(Empty) returns (GetPostResponse);
    }

First of all, yes, I know there is a google.protobuf.Empty message and I don’t need a custom Empty{} message, but it will result in an easier implementation later on. Our service description has nothing speciall. Just a GetPost RPC without any input (which is not realistic at all, but again, let’s keep it simple) and a simple response. Let’s briefly look at the Go implementation of this service as well:

    package main

    import (
     "context"
     "fmt"
     "log"
     "net"

     "google.golang.org/grpc"
     "google.golang.org/grpc/reflection"

     "github.com/HomayoonAlimohammadi/mini-grpc-gateway/pb/post"
    )

    type serverImpl struct {
     post.UnimplementedPostServiceServer
    }

    func main() {
     lis, err := net.Listen("tcp", ":50051")
     if err != nil {
      log.Fatalf("failed to listen: %v", err)
     }

     grpcServer := grpc.NewServer()
     post.RegisterPostServiceServer(grpcServer, newServer())
     reflection.Register(grpcServer)

     fmt.Println("running server on :50051")
     log.Fatal(grpcServer.Serve(lis))
    }

    func newServer() *serverImpl {
     return &serverImpl{}
    }

    func (s *serverImpl) GetPost(ctx context.Context, _ *post.Empty) (*post.GetPostResponse, error) {
     return &post.GetPostResponse{Title: "some title", Description: "some description"}, nil
    }

Again, nothing special here, just a repetitive response for any given request.

What we might expect is being able to make an http/1.1 call to an endpoint and receive a response from this gRPC service:

    curl localhost:8000/api/post

    >>> {"title": "some title", "description": "some description"}

But what is powering this behaviour? Let’s have a sneak peek on the final implementation of our Protoc plugin:

    func main() {
     r := mux.NewRouter()

     PostServiceHandler, err := GetPostServiceHandler()
     if err != nil {
      log.Fatalf("failed to get PostService handler: %v", err)
     }

     r.HandleFunc("/api/post", PostServiceHandler.GetPostHandler)

     srv := &http.Server{
      Handler:      r,
      Addr:         ":8000",
      WriteTimeout: 15 * time.Second,
      ReadTimeout:  15 * time.Second,
     }

     log.Print("serving on :8000")
     log.Fatal(srv.ListenAndServe())
    }

Let’s see what’s going on here:

• A new router is created (or we could just use the http.HandleFunc and skip this step).

• A handler struct is initialized for a given gRPC service (since we’re better off having the same underlying gRPC client for each service, more on this later).

• A new route with a matcher for the URL path is registered.

• Server is configured and started.

All the magic is happening in the GetPostHandler:

    func (sh *postService) GetPostHandler(w http.ResponseWriter, r *http.Request) {
     ctx := r.Context()
     in := makeGetPostRequest()

     resp, err := sh.client.GetPost(ctx, in)
     if err != nil {
      writeError(
       w, fmt.Errorf("failed to call GetPost: %w", err),
       http.StatusInternalServerError,
      )
      return
     }

     b, err := protojson.Marshal(resp)
     if err != nil {
      writeError(
       w, fmt.Errorf("failed to convert response to json: %w", err),
       http.StatusInternalServerError,
      )
      return
     }

     w.Write(b)
    }

    func makeGetPostRequest() *post.Empty {
     return &post.Empty{}
    }

    func writeError(w http.ResponseWriter, err error, status int) {
     w.WriteHeader(status)
     w.Write([]byte(err.Error()))
    }

Let’s review what happens in the code snippet above:

• gRPC Request Proto message and context is prepared and the request is made.

• The Response is marshaled into JSON and sent back to the REST client.

It’s probably a good idea to have a single gRPC client for any given gRPC backend service (or a client pool if faced with an immense load or complex load balancing). So let’s initialize our service handler like below:

    type postService struct {
     client post.PostServiceClient
    }

    func GetPostServiceHandler() (*postService, error) {
     conn, err := grpc.Dial(":50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
     if err != nil {
      return nil, fmt.Errorf("failed to dial 50051: %w", err)
     }

     return &postService{
      client: post.NewPostServiceClient(conn),
     }, nil
    }

It seems like we have everything we’ll ever need for a proper gRPC-REST gateway. There is just one huge problem: Are we going to write and maintain it all manually? What if a service is removed? What if one is added? What happens if an endpoint changes? Or when a service port is changed?

The solution is obvious: code generation. But how? Well, a Protoc Plugin might the best fit since it is both self-documenting and easy to maintain. Again, with a top-down approach, let’s see what our desired .proto file might look like:

    import "google/protobuf/descriptor.proto";

    service PostService {
      rpc GetPost(Empty) returns (GetPostResponse) {
        option (mini_grpc_gateway_options) = {
            url: "/api/post"
        };
      }
    }

    message MiniGRPCGatewayOptions {
        string url = 1;
    }

    extend google.protobuf.MethodOptions {
        MiniGRPCGatewayOptions mini_grpc_gateway_options = 1234562;
    }

Notice that there is only a minor difference in the service definition here. A custom mini_grpc_gateway_options is added to the only RPC, denoting a url that is likely the endpoint that we’re going to use to fetch a GetPost response. In order to create this custom option, we need to extend the google.protobuf.MethodOptions message and add our custom option as a new field. The very large field tag is there to ensure our custom option does not overwrite any other fields in the original message.

Next, we need to parse and extract the needed information from the .proto file in order to generate the output. Achieving it through Protoc is pretty straight forward. First we need to build the executable binary for our plugin and then put it somewhere in our $PATH.

    func main() {
     var flags flag.FlagSet

     protogen.Options{
      ParamFunc: flags.Set,
     }.Run(func(gen *protogen.Plugin) error {
      conf, err := config.ExtractServiceConfig(gen, config.ServicesConfigJSON)
      if err != nil {
       panic(err)
      }

      export.Export(gen, conf)

      return nil
     })
    }

    go build -o $(GOPATH)/bin/protoc-gen-mini-grpc-gateway main.go

Now we can use our custom plugin just like any other Protoc plugin (e.g. protoc-gen-go):

    protoc --go_out=./pb/post --go_opt=paths=source_relative \
     --go-grpc_out=./pb/post --go-grpc_opt=paths=source_relative \
     --mini-grpc-gateway_out=./pb/post service.proto 

I’m not going to walk through the rest of the generation process as I’ve covered the intricacies of writing a custom Protoc plugin in one of my previous blogs: Protoc Plugins in Go. Feel free to check it out as well. The source code also contains the complete plugin as well.

The main idea is to extract what ever we need from the .proto files as well as a service configuration source (e.g. a JSON file or maybe a centralized config, controlled and maintained via a control plane in a Kubernetes cluster) for our gRPC clients to be configured correctly. Given that information, we should be able to generate handlers and clients for every method of every service annotated with our custom MiniGRPCGatewayOptions method option.

Note that for the sake of simplicity, I skipped arguably pretty important features. Feel free to contribute to the project with any of the following or your own idea:

• Treating URL params as input for our backend gRPC call

• Using parametric path in order to enrich backend gRPC call input (e.g. /api/post/{token})

• Limiting the JSON response to only specific gRPC response fields.

Conclusion

gRPC-REST Gateway is an interesting, yet pretty intuitive piece of software. I really enjoyed implementing a simple version of it. I wish you’ve enjoyed and hopefully learned something as well. Feel free to contact me in any means (the comments, my LinkedIn and Twitter).

References

• Source Code (Github Repository): https://github.com/HomayoonAlimohammadi/mini-grpc-gateway

• Carl Mastrangelo — Why does gRPC insist on Trailers: https://carlmastrangelo.com/blog/why-does-grpc-insist-on-trailers

• Akshayshah — gRPC Doesn’t Need Trailers: gRPC Doesn’t Need Trailers: https://akshayshah.org/grpc-doesnt-need-trailers

• gRPC-Web: https://github.com/grpc/grpc-web