DEV Community

Elton Minetto
Elton Minetto

Posted on

Microservices in Go using the Go kit

In one of the chapters of the book Microservice Patterns: With examples in Java the author mentions the "Microservice chassis" pattern:

Build your microservices using a microservice chassis framework, which handles cross-cutting concerns, such as exception tracking, logging, health checks, externalized configuration, and distributed tracing.

He goes further and gives some examples of frameworks that implement these concepts in Java and Go:

After some research I chose the Go kit as it is one of the most popular, it is being updated at a constant speed and I liked the architecture that it proposes.

Architecture

Service

service

Services are where all of the business logic is implemented. In Go kit, services are typically modeled as interfaces, and implementations of those interfaces contain the business logic. Go kit services should strive to abide the Clean Architecture or the Hexagonal Architecture. That is, the business logic should not know of transport-domain concepts: your service shouldn’t know anything about HTTP headers, or gRPC error codes.

Endpoint

endpoint

An endpoint is like an action/handler on a controller; it’s where safety and antifragile logic lives. If you implement two transports (HTTP and gRPC), you might have two methods of sending requests to the same endpoint.

Transport

transport

The transport domain is bound to concrete transports like HTTP or gRPC. In a world where microservices may support one or more transports, this is very powerful; you can support a legacy HTTP API and a newer RPC service, all in a single microservice.

Example

Let's create an example of a microservice using this architecture. The directory structure looks like this:

example

Service

The service layer code in this example is very simple:

package user

import (
    "auth/security"
    "context"
    "errors"
)

type Service interface {
    ValidateUser(ctx context.Context, mail, password string) (string, error)
    ValidateToken(ctx context.Context, token string) (string, error)
}

var (
    ErrInvalidUser  = errors.New("Invalid user")
    ErrInvalidToken = errors.New("Invalid token")
)

type service struct{}

func NewService() *service {
    return &service{}
}

func (s *service) ValidateUser(ctx context.Context, email, password string) (string, error) {
    //@TODO create validation rules, using databases or something else
    if email == "eminetto@gmail.com" && password != "1234567" {
        return "nil", ErrInvalidUser
    }
    token, err := security.NewToken(email)
    if err != nil {
        return "", err
    }
    return token, nil
}

func (s *service) ValidateToken(ctx context.Context, token string) (string, error) {
    t, err := security.ParseToken(token)
    if err != nil {
        return "", ErrInvalidToken
    }
    tData, err := security.GetClaims(t)
    if err != nil {
        return "", ErrInvalidToken
    }
    return tData["email"].(string), nil
}
Enter fullscreen mode Exit fullscreen mode

As the Go kit documentation recommends, the first step is to create an interface for our service, which will be implemented with our business logic. Soon, this decision to create an interface will prove useful when we include logging and monitoring metrics in the application.

Because it only has business rules, the service layer test is also very simple:

package user_test

import (
    "auth/user"
    "context"
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestValidateUser(t *testing.T) {
    service := user.NewService()
    t.Run("invalid user", func(t *testing.T) {
        _, err := service.ValidateUser(context.Background(), "eminetto@gmail.com", "invalid")
        assert.NotNil(t, err)
        assert.Equal(t, "Invalid user", err.Error())
    })
    t.Run("valid user", func(t *testing.T) {
        token, err := service.ValidateUser(context.Background(), "eminetto@gmail.com", "1234567")
        assert.Nil(t, err)
        assert.NotEmpty(t, token)
    })
}

Enter fullscreen mode Exit fullscreen mode

Endpoint

We will now expose our functions to the outside world. In this example the two functions will be able to be accessed externally, so we will create two endpoints. But this is not always true. Depending on the scenario you can expose only a few functions and keep the others accessible only within the service layer.

package user

import (
    "context"

    "github.com/go-kit/kit/endpoint"
)

//definition of endpoint input and output structures 
type validateUserRequest struct {
    Email    string `json:"email"`
    Password string `json:"password"`
}

type validateUserResponse struct {
    Token string `json:"token,omitempty"`
    Err   string `json:"err,omitempty"` // errors don't JSON-marshal, so we use a string
}

//the endpoint will receive a request, convert to the desired
//format, invoke the service and return the response structure 
func makeValidateUserEndpoint(svc Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(validateUserRequest)
        token, err := svc.ValidateUser(ctx, req.Email, req.Password)
        if err != nil {
            return validateUserResponse{"", err.Error()}, err
        }
        return validateUserResponse{token, ""}, err
    }
}

//definition of endpoint input and output structures 
type validateTokenRequest struct {
    Token string `json:"token"`
}

type validateTokenResponse struct {
    Email string `json:"email,omitempty"`
    Err   string `json:"err,omitempty"`
}

//the endpoint will receive a request, convert to the desired
//format, invoke the service and return the response structure 
func makeValidateTokenEndpoint(svc Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(validateTokenRequest)
        email, err := svc.ValidateToken(ctx, req.Token)
        if err != nil {
            return validateTokenResponse{"", err.Error()}, err
        }
        return validateTokenResponse{email, ""}, err
    }
}

Enter fullscreen mode Exit fullscreen mode

The role of the endpoint is to receive a request, convert it to the expected struct, invoke the service layer, and return another struct. The endpoint layer does not know anything about the upper layer, because it makes no difference whether the endpoint is being invoked via HTTP, gRPC, or another form of transport.

Because of its simplicity, testing this layer is equally easy to implement:

package user

import (
    "context"
    "testing"
)

func TestMakeValidateUserEndpoint(t *testing.T) {
    s := NewService()
    endpoint := makeValidateUserEndpoint(s)
    t.Run("valid user", func(t *testing.T) {
        req := validateUserRequest{
            Email:    "eminetto@gmail.com",
            Password: "1234567",
        }
        _, err := endpoint(context.Background(), req)
        if err != nil {
            t.Errorf("expected %v received %v", nil, err)
        }
    })
    t.Run("invalid user", func(t *testing.T) {
        req := validateUserRequest{
            Email:    "eminetto@gmail.com",
            Password: "123456",
        }
        _, err := endpoint(context.Background(), req)
        if err == nil {
            t.Errorf("expected %v received %v", ErrInvalidUser, err)
        }
    })
}

Enter fullscreen mode Exit fullscreen mode

This test could be improved by replacing the use of the service with a mock that implements the same Service interface, making the tests more efficient.

Transport

In this layer, we can have several implementations like HTTP, gRPC, AMPQ, NATS, etc. In this example, we are going to expose our endpoints in the form of an HTTP API. So, we will create the file transpor_http.go:

package user

import (
    "context"
    "encoding/json"
    "net/http"

    "github.com/go-kit/kit/log"
    httptransport "github.com/go-kit/kit/transport/http"
    "github.com/gorilla/mux"
)

func NewHttpServer(svc Service, logger log.Logger) *mux.Router {
    //options provided by the Go kit to facilitate error control 
    options := []httptransport.ServerOption{
        httptransport.ServerErrorLogger(logger),
        httptransport.ServerErrorEncoder(encodeErrorResponse),
    }
    //definition of a handler 
    validateUserHandler := httptransport.NewServer(
        makeValidateUserEndpoint(svc), //use the endpoint
        decodeValidateUserRequest, //converts the parameters received via the request body into the struct expected by the endpoint 
        encodeResponse, //converts the struct returned by the endpoint to a json response 
        options...,
    )

    validateTokenHandler := httptransport.NewServer(
        makeValidateTokenEndpoint(svc),
        decodeValidateTokenRequest,
        encodeResponse,
        options...,
    )
    r := mux.NewRouter() //I'm using Gorilla Mux, but it could be any other library, or even the stdlib 
    r.Methods("POST").Path("/v1/auth").Handler(validateUserHandler)
    r.Methods("POST").Path("/v1/validate-token").Handler(validateTokenHandler)
    return r
}

func encodeErrorResponse(_ context.Context, err error, w http.ResponseWriter) {
    if err == nil {
        panic("encodeError with nil error")
    }
    w.Header().Set("Content-Type", "application/json; charset=utf-8")
    w.WriteHeader(codeFrom(err))
    json.NewEncoder(w).Encode(map[string]interface{}{
        "error": err.Error(),
    })
}

func codeFrom(err error) int {
    switch err {
    case ErrInvalidUser:
        return http.StatusNotFound
    case ErrInvalidToken:
        return http.StatusUnauthorized
    default:
        return http.StatusInternalServerError
    }
}

//converts the parameters received via the request body into the struct expected by the endpoint 
func decodeValidateUserRequest(ctx context.Context, r *http.Request) (interface{}, error) {
    var request validateUserRequest
    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
        return nil, err
    }
    return request, nil
}

//converts the parameters received via the request body into the struct expected by the endpoint 
func decodeValidateTokenRequest(ctx context.Context, r *http.Request) (interface{}, error) {
    var request validateTokenRequest
    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
        return nil, err
    }
    return request, nil
}

//converts the struct returned by the endpoint to a json response
func encodeResponse(ctx context.Context, w http.ResponseWriter, response interface{}) error {
    return json.NewEncoder(w).Encode(response)
}

Enter fullscreen mode Exit fullscreen mode

The code looks like a series of settings, indicating which endpoint will be used at each API address. I tried to describe the behavior in the code comments. And the test of this layer looked like this:

package user

import (
    "net/http"
    "net/http/httptest"
    "os"
    "strings"
    "testing"

    "github.com/go-kit/kit/log"
)

func TestHTTP(t *testing.T) {
    var logger log.Logger
    logger = log.NewLogfmtLogger(os.Stderr)
    logger = log.With(logger, "listen", "8081", "caller", log.DefaultCaller)
    s := NewService()
    r := NewHttpServer(s, logger)
    srv := httptest.NewServer(r)

    for _, testcase := range []struct {
        method, url, body string
        want              int
    }{
        {"POST", srv.URL + "/v1/auth", `{"email": "eminetto@gmail.com", "password":"1234567"}`, http.StatusOK},
        {"GET", srv.URL + "/v1/auth", `{"email": "eminetto@gmail.com", "password":"1234567"}`, http.StatusMethodNotAllowed},
        {"POST", srv.URL + "/v1/auth", `{"email": "eminetto@gmail.com", "password":"invalid"}`, http.StatusNotFound},
        {"POST", srv.URL + "/v1/validate-token", `{"token": "invalid"}`, http.StatusUnauthorized},
    } {
        req, _ := http.NewRequest(testcase.method, testcase.url, strings.NewReader(testcase.body))
        resp, _ := http.DefaultClient.Do(req)
        if testcase.want != resp.StatusCode {
            t.Errorf("%s %s %s: want %d have %d", testcase.method, testcase.url, testcase.body, testcase.want, resp.StatusCode)
        }

    }
}

Enter fullscreen mode Exit fullscreen mode

Just like testing the endpoint layer, we could improve this test using a mock of the service.

Main

In the main.go file we are going to use all the layers:

package main

import (
    "auth/user"
    "net/http"
    "os"

    "github.com/go-kit/kit/log"
)

func main() {

    var logger log.Logger
    logger = log.NewLogfmtLogger(os.Stderr)
    logger = log.With(logger, "listen", "8081", "caller", log.DefaultCaller)

    svc := user.NewLoggingMiddleware(logger, user.NewService())
    r := user.NewHttpServer(svc, logger)
    logger.Log("msg", "HTTP", "addr", "8081")
    logger.Log("err", http.ListenAndServe(":8081", r))
}
Enter fullscreen mode Exit fullscreen mode

Here we can see another advantage in having created an interface for our service. The user.NewHttpServer function expects as a first parameter something that implements the Service interface. The user.NewLoggingMiddleware function creates a struct that implements this interface and has our original service inside it. The code for the logging.go file looks like this:

package user

import (
    "context"
    "time"

    "github.com/go-kit/kit/log"
)

func NewLoggingMiddleware(logger log.Logger, next Service) logmw {
    return logmw{logger, next}
}

type logmw struct {
    logger log.Logger
    Service
}

func (mw logmw) ValidateUser(ctx context.Context, email, password string) (token string, err error) {
    defer func(begin time.Time) {
        _ = mw.logger.Log(
            "method", "validateUser",
            "input", email,
            "err", err,
            "took", time.Since(begin),
        )
    }(time.Now())

    token, err = mw.Service.ValidateUser(ctx, email, password)
    return
}

func (mw logmw) ValidateToken(ctx context.Context, token string) (email string, err error) {
    defer func(begin time.Time) {
        _ = mw.logger.Log(
            "method", "validateToken",
            "input", token,
            "err", err,
            "took", time.Since(begin),
        )
    }(time.Now())

    email, err = mw.Service.ValidateToken(ctx, token)
    return
}

Enter fullscreen mode Exit fullscreen mode

It implements all the functions of the interface, adding the functionality of logging each function call, before invoking the code of the real service. The same can be used to implement metrics, limit access to API, etc. In the official tutorial, we have some examples of this.

If our microservice needs to deliver the logic in more formats, such as gRPC or NATS, we would only need to implement these codes in the transport layer indicating which endpoints will be used. This gives a lot of flexibility for the growth of functionalities without increasing complexity.

In this post, I focused more on the architecture provided by the Go kit, but in the official documentation, you can see the other chassis features that it provides as authentication, circuit breaker, log, metrics, rate limit, service discovery, tracing, etc.

I liked the architecture and features it provides and I believe it can be useful to create services in a fast, clean and efficient way.

The codes for this example are in this repository.

Discussion (0)