I've searched a lot on the Internet (I consider my google-fu quite good) for some well written tutorial about writing simple "RESTful" microservices in Go with Go kit.
I was not able to find any...
Examples from the Go kit repository are good, but documentation is dull, IMHO.
I then decided to buy this book, called Go Programming Blueprints, 2nd Edition which is quite good, but has only two chapters dedicated to Go kit (one for actual development of a microservice and one for the actual deploy) and I don't really care for now about gRPC, which examples on chapter 10 of the book are also implementing. To much scaffolding code if you ask me :P
Sooo, I decided to give back something to the community and write a tutorial, in order to "learn by doing". This tutorial will be heavily inspired by the book mentioned above, and probably could be improved in many ways.
Feel free to give a feedback!
You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com
What is Go kit?
From the Go kit README.md:
Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and application architecture so you can focus on delivering business value
[...]
Go is a great general-purpose language, but microservices require a certain amount of specialized support. RPC safety, system observability, infrastructure integration, even program design — Go kit fills in the gaps left by the standard library, and makes Go a first-class language for writing microservices in any organization.
I don't want to discuss much about it: I'm too new to Go to have a strong opinion. The community is of course divided in the ones that like it and the ones that don't. You can also find a good article about differences in frameworks for Go microservices here.
What are we going to do?
We are going to create a very basic microservice that will return and validate a date... The goal is understand how Go kit works, nothing more than that. You can easily replicate all the logic without Go kit, but I'm here to learn, and so...
I hope you'll have a glimpse and a good starting point for your next projects!
Our microservice will have some endpoints.
- One
GETendpoint at/statusthat will return a simple answer that will confirm that the microservice is up and running - One
GETendpoint at/getthat will return today's date - One
POSTendpoint at/validatethat will receive a date string in thedd/mm/yyyy(the only existing date format if you ask me, take this USA!) format and validate it according to a simple regex...
Again, you will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com
Let's start!!!
Prerequisites
You should have Golang installed & working on your machine. I've found the official download package working better (I had some problems with the env. vars) than the Homebrew installation on my Macbook.
Plus, you should know the Go language, I'm not going to explain what a struct is, for example.
The napodate microservice
Ok, let's start by creating a new folder in our $GOPATH folder called napodate. This will also be the name of our package.
Put a service.go file there. Let's add our service interface on top of the file.
package napodate
import "context"
// Service provides some "date capabilities" to your application
type Service interface {
Status(ctx context.Context) (string, error)
Get(ctx context.Context) (string, error)
Validate(ctx context.Context, date string) (bool, error)
}
Here we're defining the "blueprint" for our service: in Go kit, you have to model a service as an interface. As described above, we will need three endpoints that will be mapped, going forward, to this interface.
Why are we using the context package? Read https://blog.golang.org/context:
At Google, we developed a context package that makes it easy to pass request-scoped values, cancelation signals, and deadlines across API boundaries to all the goroutines involved in handling a request
Basically, this is needed because our microservice should be made from the beginning to handle concurrent requests and a context for every request is mandatory.
You don't want to mix up stuff. More on this later in the tutorial. We are not using that much now, but get used to it! :P We now have our microservice interface.
Implementing our service
As you probably know, an interface is nothing without an implementation, so let's implement our service. Let's add some more code to service.go.
type dateService struct{}
// NewService makes a new Service.
func NewService() Service {
return dateService{}
}
// Status only tell us that our service is ok!
func (dateService) Status(ctx context.Context) (string, error) {
return "ok", nil
}
// Get will return today's date
func (dateService) Get(ctx context.Context) (string, error) {
now := time.Now()
return now.Format("02/01/2006"), nil
}
// Validate will check if the date today's date
func (dateService) Validate(ctx context.Context, date string) (bool, error) {
_, err := time.Parse("02/01/2006", date)
if err != nil {
return false, err
}
return true, nil
}
The newly defined type dateService (an empty struct) is how we are going to group together the methods of our service, while "hiding" somehow the implementation to the rest of the world.
See NewService() as a constructor for our "object". This is what we'll call to get an instance of our service, all while masking the internal logic as good programmers should do.
Let's write a test
A good example of how NewService() is used can be seen in the test for our service. Go on and create a service_test.go file.
package napodate
import (
"context"
"testing"
"time"
)
func TestStatus(t *testing.T) {
srv, ctx := setup()
s, err := srv.Status(ctx)
if err != nil {
t.Errorf("Error: %s", err)
}
// testing status
ok := s == "ok"
if !ok {
t.Errorf("expected service to be ok")
}
}
func TestGet(t *testing.T) {
srv, ctx := setup()
d, err := srv.Get(ctx)
if err != nil {
t.Errorf("Error: %s", err)
}
time := time.Now()
today := time.Format("02/01/2006")
// testing today's date
ok := today == d
if !ok {
t.Errorf("expected dates to be equal")
}
}
func TestValidate(t *testing.T) {
srv, ctx := setup()
b, err := srv.Validate(ctx, "31/12/2019")
if err != nil {
t.Errorf("Error: %s", err)
}
// testing that the date is valid
if !b {
t.Errorf("date should be valid")
}
// testing an invalid date
b, err = srv.Validate(ctx, "31/31/2019")
if b {
t.Errorf("date should be invalid")
}
// testing a USA date date
b, err = srv.Validate(ctx, "12/31/2019")
if b {
t.Errorf("USA date should be invalid")
}
}
func setup() (srv Service, ctx context.Context) {
return NewService(), context.Background()
}
I made the tests more readable, but you should really write them with Subtests, for a more up-to-date syntax.
Tests are green (!) but focus for a second on the setup() method. For every test we return an instance of our service using NewService() and the context.
Transports
Our service will be exposed using HTTP. We are now going to model the accepted HTTP requests and responses. Go and create a file called transport.go in the same folder of service.go.
package napodate
import (
"context"
"encoding/json"
"net/http"
)
// In the first part of the file we are mapping requests and responses to their JSON payload.
type getRequest struct{}
type getResponse struct {
Date string `json:"date"`
Err string `json:"err,omitempty"`
}
type validateRequest struct {
Date string `json:"date"`
}
type validateResponse struct {
Valid bool `json:"valid"`
Err string `json:"err,omitempty"`
}
type statusRequest struct{}
type statusResponse struct {
Status string `json:"status"`
}
// In the second part we will write "decoders" for our incoming requests
func decodeGetRequest(ctx context.Context, r *http.Request) (interface{}, error) {
var req getRequest
return req, nil
}
func decodeValidateRequest(ctx context.Context, r *http.Request) (interface{}, error) {
var req validateRequest
err := json.NewDecoder(r.Body).Decode(&req)
if err != nil {
return nil, err
}
return req, nil
}
func decodeStatusRequest(ctx context.Context, r *http.Request) (interface{}, error) {
var req statusRequest
return req, nil
}
// Last but not least, we have the encoder for the response output
func encodeResponse(ctx context.Context, w http.ResponseWriter, response interface{}) error {
return json.NewEncoder(w).Encode(response)
}
You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com
A bit of code if you ask me, but you'll find comments in the transport.go in the repo file that will help you navigate it.
In the first part of the file we are mapping requests and responses to their JSON payload. For statusRequest and getRequest we don't need much, since no payload is sent to the server. For validateRequest we are going to pass a date to be validated, so here is the date field.
Responses are pretty straightforward too.
In the second part we will write "decoders" for our incoming requests, telling the service how he should translate requests and map them to the correct request struct. get and status are empty, I know, but they're there for the sake of completeness. Remember, I'm learning by doing...
Last but not least, we have the encoder for the response output, which is a simple JSON encoder: given an object, we'll return a JSON object from it.
That's it for transports, let's create our endpoints!
Endpoints
Let's create a new file endpoint.go. This file will contain our endpoints that will map the request coming from the client to our internal service
package napodate
import (
"context"
"errors"
"github.com/go-kit/kit/endpoint"
)
// Endpoints are exposed
type Endpoints struct {
GetEndpoint endpoint.Endpoint
StatusEndpoint endpoint.Endpoint
ValidateEndpoint endpoint.Endpoint
}
// MakeGetEndpoint returns the response from our service "get"
func MakeGetEndpoint(srv Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
_ = request.(getRequest) // we really just need the request, we don't use any value from it
d, err := srv.Get(ctx)
if err != nil {
return getResponse{d, err.Error()}, nil
}
return getResponse{d, ""}, nil
}
}
// MakeStatusEndpoint returns the response from our service "status"
func MakeStatusEndpoint(srv Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
_ = request.(statusRequest) // we really just need the request, we don't use any value from it
s, err := srv.Status(ctx)
if err != nil {
return statusResponse{s}, err
}
return statusResponse{s}, nil
}
}
// MakeValidateEndpoint returns the response from our service "validate"
func MakeValidateEndpoint(srv Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(validateRequest)
b, err := srv.Validate(ctx, req.Date)
if err != nil {
return validateResponse{b, err.Error()}, nil
}
return validateResponse{b, ""}, nil
}
}
// Get endpoint mapping
func (e Endpoints) Get(ctx context.Context) (string, error) {
req := getRequest{}
resp, err := e.GetEndpoint(ctx, req)
if err != nil {
return "", err
}
getResp := resp.(getResponse)
if getResp.Err != "" {
return "", errors.New(getResp.Err)
}
return getResp.Date, nil
}
// Status endpoint mapping
func (e Endpoints) Status(ctx context.Context) (string, error) {
req := statusRequest{}
resp, err := e.StatusEndpoint(ctx, req)
if err != nil {
return "", err
}
statusResp := resp.(statusResponse)
return statusResp.Status, nil
}
// Validate endpoint mapping
func (e Endpoints) Validate(ctx context.Context, date string) (bool, error) {
req := validateRequest{Date: date}
resp, err := e.ValidateEndpoint(ctx, req)
if err != nil {
return false, err
}
validateResp := resp.(validateResponse)
if validateResp.Err != "" {
return false, errors.New(validateResp.Err)
}
return validateResp.Valid, nil
}
Let's dig a bit into this... In order to expose all our service methods Get(), Status() and Validate() as endpoints we are going to write functions that will handle the incoming requests, call the corresponding service method and depending on the response will build and return a proper object.
These methods are the Make... ones. They will receive the servuce as argument, whe then use a type assertion to "force" the request type to a specific one and use it to call the service method for it.
After these Make... methods, that will be used in the main.go file, we will write the endpoints to comply with the service interface
type Endpoints struct {
GetEndpoint endpoint.Endpoint
StatusEndpoint endpoint.Endpoint
ValidateEndpoint endpoint.Endpoint
}
Let's make an example:
// Status endpoint mapping
func (e Endpoints) Status(ctx context.Context) (string, error) {
req := statusRequest{}
resp, err := e.StatusEndpoint(ctx, req)
if err != nil {
return "", err
}
statusResp := resp.(statusResponse)
return statusResp.Status, nil
}
This method will allow us to use the endpoints as Go methods.
The HTTP server
For our microservice we need an HTTP server. Go is pretty helpful with this, but I chose https://github.com/gorilla/mux for our routes since its syntax looks so clean and concise, so let's create a little cute HTTP server with mappings to our endpoints.
Create a new file called server.go in your project.
package napodate
import (
"context"
"net/http"
httptransport "github.com/go-kit/kit/transport/http"
"github.com/gorilla/mux"
)
// NewHTTPServer is a good little server
func NewHTTPServer(ctx context.Context, endpoints Endpoints) http.Handler {
r := mux.NewRouter()
r.Use(commonMiddleware) // @see https://stackoverflow.com/a/51456342
r.Methods("GET").Path("/status").Handler(httptransport.NewServer(
endpoints.StatusEndpoint,
decodeStatusRequest,
encodeResponse,
))
r.Methods("GET").Path("/get").Handler(httptransport.NewServer(
endpoints.GetEndpoint,
decodeGetRequest,
encodeResponse,
))
r.Methods("POST").Path("/validate").Handler(httptransport.NewServer(
endpoints.ValidateEndpoint,
decodeValidateRequest,
encodeResponse,
))
return r
}
func commonMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Add("Content-Type", "application/json")
next.ServeHTTP(w, r)
})
}
The endpoints will be passed to the server from the main.go file, and commonMiddleware(), not exposed will take care to add specific headers to every response.
And finally, our main.go file
Let's wrap up! We have a service with endpoints. We have an HTTP server, we just need a place where we can wrap up everything and of course it's our main.go file. Put it into a new folder, let's call it cmd.
package main
import (
"context"
"flag"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"napodate"
)
func main() {
var (
httpAddr = flag.String("http", ":8080", "http listen address")
)
flag.Parse()
ctx := context.Background()
// our napodate service
srv := napodate.NewService()
errChan := make(chan error)
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
errChan <- fmt.Errorf("%s", <-c)
}()
// mapping endpoints
endpoints := napodate.Endpoints{
GetEndpoint: napodate.MakeGetEndpoint(srv),
StatusEndpoint: napodate.MakeStatusEndpoint(srv),
ValidateEndpoint: napodate.MakeValidateEndpoint(srv),
}
// HTTP transport
go func() {
log.Println("napodate is listening on port:", *httpAddr)
handler := napodate.NewHTTPServer(ctx, endpoints)
errChan <- http.ListenAndServe(*httpAddr, handler)
}()
log.Fatalln(<-errChan)
}
Let's analyze this file together. We declare the main package and import what we need.
We use a flag to make the listening port configurable, the default port for our service will be the classic 8080 but we can lunch it with whatever port we wont thanks to the flag.
What follows is the setup of our server: we create a context (see above for an explanation of what a context is) and we get our service. An error channel is also setup.
Channels are the pipes that connect concurrent goroutines. You can send values into channels from one goroutine and receive those values into another goroutine.
We then create two goroutines. One to stop the server when we press CTRL+C and one that will actually listen for incoming requests.
Look at handler := napodate.NewHTTPServer(ctx, endpoints) this handler will map our service endpoints (do you remember the Make... methods from above?) and return the correct answer.
Where did you see NewHTTPServer() before?
As soon as the channel receives an error message, the server will stop and die.
Let's the service!
If you did everything correctly, by running
go run cmd/main.go
from your project folder, you should be able to curl your microservice!
curl http://localhost:8080/get
{"date":"14/04/2019"}
curl http://localhost:8080/status
{"status":"ok"}
curl -XPOST -d '{"date":"32/12/2020"}' http://localhost:8080/validate
{"valid":false,"err":"parsing time \"32/12/2020\": day out of range"}
curl -XPOST -d '{"date":"12/12/2021"}' http://localhost:8080/validate
{"valid":true}
Wrapping up
We created a new microservice from scratch, even if it's super simple, it's a good way to start using Go kit with the Go programming language.
Hope you enjoyed this tutorial as much as I did!
You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com
Top comments (10)
This was a great tutorial, very insightful indeed. However, I am stuck with the endpoints part and would request you to emphasize better on that part alone. Cheers! :)
How can I help you on this? Any specific question?
It seems that all the code under "/ Get endpoint mapping" is never executed. I've removed all the "Endpoint Mappings" and it still works the same. Not sure what those mappings were supposed to map
Great effort Francesco.
I was thinking how can we create multiple services and spin them up with docker swarm?
Like I have some microservices written in GO (any microservice framework) and then I want to define them separately in yml file with different configurations.
Any suggestion or link about this please :)
Thanks!
I am used to C# and for the sake of comparison, creating a service in go with this approach seems too verbose. I am wandering if I need to write so many lines to get a http service up and running. Nonetheless I appreciate this how-to article
This is a very opinionated article using GoKit. I wrote it because I had to learn GoKit in some days, it worked for me.
You can easily create an http micro with other frameworks, like GoMicro micro.mu/docs/framework.html
Or by simply answering http requests from a go program.
Read this. medium.com/statuscode/how-i-write-... This is very ispirational!
How to handle error io.EOF error in request?
It's a great and simple tutorial! Thanks!
It was my first go micro-service with go kit.
Hello, thanks for your post! But i have a question. As microservices, it also can deploy, update code independent. So how can I do that with go-kit.
Hi,
Go kit is it production support?