Build Your First Rest API with GO

There is three part to this workshop.

1. API
2. Rest API
3. Rest API with GO

API

If you have been around a computer for long enough you probably heard of this thing. What is this API?

API stands for Application Program Interface. Like most thing in computer science the abbreviation doesn't help much.

What it actually means is it exposes functionality without exposing internals. If you program in a language that supports writing functions or methods (pretty much all programming languages) you would totally understand what I am talking about.

func addNumber(a, b int) int {
    // DO AMAZING MATH HERE
    // and return the result
}

Even if you are super new to go, you can tell this function is about adding two numbers and returning the result.

For the user of the function you just call the function and never worry about how the function is doing what it does (don't trust every function).

Thats all an API is. An API could be a function you wrote, or a function from a library or method from a framework, or a http endpoint.

Rest API

Most APIs written this days are web apis. Don't quote me on that one because I didn't do any research to get a proper number. 😁 But given the number of web services and web application I don't think I am too far off.

What is REST

REST is acronym for REpresentational State Transfer. It is architectural style for distributed hypermedia systems and was first presented by Roy Fielding in 2000 in his famous dissertation.

Like any other architectural style, REST also does have it’s own 6 guiding constraints which must be satisfied if an interface needs to be referred as RESTful. These principles are listed below.

Guiding Principles of REST

1. Client–server – By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components.
2. Stateless – Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.
3. Cacheable – Cache constraints require that the data within a response to a request be implicitly or explicitly labeled as cacheable or non-cacheable. If a response is cacheable, then a client cache is given the right to reuse that response data for later, equivalent requests.
4. Uniform interface – By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. In order to obtain a uniform interface, multiple architectural constraints are needed to guide the behavior of components. REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.
5. Layered system – The layered system style allows an architecture to be composed of hierarchical layers by constraining component behavior such that each component cannot “see” beyond the immediate layer with which they are interacting.
6. Code on demand (optional) – REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts. This simplifies clients by reducing the number of features required to be pre-implemented.

To see an example of a REST API we can use

HTTP Verbs

These are some conventions HTTP apis follow. These are actually not part of Rest specification. But we need to understand these to fully utilize Rest API.

HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. Although they can also be nouns, these request methods are sometimes referred as HTTP verbs. Each of them implements a different semantic, but some common features are shared by a group of them: e.g. a request method can be safe, idempotent, or cacheable.

GETThe GET method requests a representation of the specified resource. Requests using GETshould only retrieve data.

HEADThe HEAD method asks for a response identical to that of a GET request, but without the response body.

POSTThe POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server.

PUTThe PUT method replaces all current representations of the target resource with the request payload.

DELETEThe DELETE method deletes the specified resource.

CONNECTThe CONNECT method establishes a tunnel to the server identified by the target resource.

OPTIONSThe OPTIONS method is used to describe the communication options for the target resource.

TRACEThe TRACE method performs a message loop-back test along the path to the target resource.

PATCHThe PATCH method is used to apply partial modifications to a resource.

THESE ARE ALL LIES.

Status Codes

1xx Information

• 100 Continue
• 101 Switching Protocols
• 102 Processing

2xx Success

• 200 OK
• 201 Created
• 202 Accepted
• 203 Non-authoritative Information
• 204 No Content
• 205 Reset Content
• 206 Partial Content
• 207 Multi-Status
• 208 Already Reported
• 226 IM Used

3xx Redirects

• 300 Multiple Choices
• 301 Moved Permanently
• 302 Found
• 303 See Other
• 304 Not Modified
• 305 Use Proxy
• 307 Temporary Redirect
• 308 Permanent Redirect

4xx Client Error

• 400 Bad Request
• 401 Unauthorized
• 402 Payment Required
• 403 Forbidden
• 404 Not Found
• 405 Method Not Allowed
• 406 Not Acceptable
• 407 Proxy Authentication Required
• 408 Request Timeout
• 409 Conflict
• 410 Gone
• 411 Length Required
• 412 Precondition Failed
• 413 Payload Too Large
• 414 Request-URI Too Long
• 415 Unsupported Media Type
• 416 Requested Range Not Satisfiable
• 417 Expectation Failed
• 418 I'm a teapot
• 421 Misdirected Request
• 422 Unprocessable Entity
• 423 Locked
• 424 Failed Dependency
• 426 Upgrade Required
• 428 Precondition Required
• 429 Too Many Requests
• 431 Request Header Fields Too Large
• 444 Connection Closed Without Response
• 451 Unavailable For Legal Reasons
• 499 Client Closed Request

5xx Server Error

• 500 Internal Server Error
• 501 Not Implemented
• 502 Bad Gateway
• 503 Service Unavailable
• 504 Gateway Timeout
• 505 HTTP Version Not Supported
• 506 Variant Also Negotiates
• 507 Insufficient Storage
• 508 Loop Detected
• 510 Not Extended
• 511 Network Authentication Required
• 599 Network Connect Timeout Error

This also has no actual meaning.

Terminologies

The following are the most important terms related to REST APIs

• Resource is an object or representation of something, which has some associated data with it and there can be set of methods to operate on it. E.g. Animals, schools and employees are resources and delete, add, update are the operations to be performed on these resources.
• Collections are set of resources, e.g Companies is the collection of Company resource.
• URL (Uniform Resource Locator) is a path through which a resource can be located and some actions can be performed on it.

API Endpoint

This is what a API endpoint looks like.

https://www.github.com/golang/go/search?q=http&type=Commits

This URL can be broken into these parts

Protocol

How the browser or client should communicate with the server.

Subdomain

Sub Division of the main domain

Domain

Unique reference to identify web site on the internet

Port

Port on the server the application is running on. By default its 80. So most cases we don't see it

Path

Path parameters in a Rest API represents resources.

https://jsonplaceholder.typicode.com/posts/1/comments

posts/1/comments

This path is representing the 1st posts resource'c comments

Basic structure is

top-level-resource/<some-identifier>/secondary-resource/<some-identifier>/...

Query

Queries are key value pairs of information, used mostly for filtering purposes.

https://jsonplaceholder.typicode.com/posts?userId=1

Parts after the ? is the query parameters. We have only one query here. userId=1

Headers

This was not part of the URL itself but header is a part of network component sent by the client or the server. Based on who sends it. There are two kinds of header

1. Request Header (client -> server)
2. Response Header (server -> client)

Body

You can add extra information to both the request to the server and to the response from the server.

Response Type

Usually JSON or XML.

Now a days it's mostly JSON.

Rest API with GO

This is why you are here. Or I hope this is why you are here.

Language Design in the Service of Software Engineering

This above article came out in 2012. But still pretty relevant to learn the ideology behind go.

If you are writing Rest API why should you choose go?

1. It's compiled. So you get small binaries.
2. It's fast. (slower than c/c++ or rust) but faster than most other web programming languages.
3. It's simple to understand.
4. It works really well in the microservices world for reason no 1.

net/http

The standard library in go comes with the net/http package, which is an excellent starting point for building RestAPIs. And most other libraries the adds some additional feature are also interoperable with the net/http package so understanding the net/http package is crucial to using golang for RestAPIs.

net/http

We probably don't need to know everything in the net/http package. But there are a few things we should know to get started.

The Handler Interface

I am never a proposer of memorizing something but as Todd Mcleod in his course mentions over and over. We need to memorize the Handler interface.

type Handler interface {
        ServeHTTP(ResponseWriter, *Request)
}

And here it is.

It has one method and one method only.

A struct or object will be Handler if it has one method ServeHTTP which takes ResponseWriter and pointer to Request.

With all our knowledge now we are ready to do some damage.

Let's Begin

I think now we are ready to get started.

That was a lot of theory. I promised you will build you first RestAPI.

Simple Rest API

So let's jump right into it.

In a folder where you want to write your go code

go mod init api-test

Create a new file, you can name it whatever you want.

I am calling mine main.go

package main

import (
    "log"
    "net/http"
)

type server struct{}

func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "hello world"}`))
}

func main() {
    s := &server{}
    http.Handle("/", s)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Lets break down this code.

At the top we have our package main all go executable need a main package.

We have our imports. log for logging some error if it happens. net/http because we are writing a rest api.

Then we have a struct called server. It has no fields. We will add a method to this server ServeHTTP and that will satisfy the Handler interface. One thing you will notice in go we don't have to explicitly say the interface we are implementing. The compiler is smart enough to figure that out. In the ServeHTTP method we set httpStatus 200 to denote its the request was a success. We se the content type to application/json so the client understands when we send back json as payload. Finally we write

{"message": "hello world"}

To the response.

Lets run our server

go run main.go

If you had installed postman before, let's test our app with postman real quick.

Get returns us our message.

Great work!

But Wait.

Lets see what other HTTP verbs our application serves.

In postman we can change the Type of request we make. Click on the dropdown and select something else. Lets say we do post.

Now if we run the request, we get back the same result.

Well its not really a bug per se. But in most cases we probably want to do different things based on the request types.

Lets see how we can do that.

We will modify our ServeHTTP method with the following.

func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

If our server is already running lets stop it with ctrl-c

Run it again.

go run main.go

Test it with postman or curl again.

One thing you may have noticed is that we are using our server struct literally for attaching a method to.

The go team knew this was an inconvenience and gave us HandleFunc Its a method on the http package that allows us to pass a function that has the same signature as the ServeHTTP and can serve a route.

We can clean up our code a little bit with this

package main

import (
    "log"
    "net/http"
)

func home(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

func main() {
    http.HandleFunc("/", home)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Functionality should be exactly the same.

Gorilla Mux

net/http built in methods are great. We can write a server with no external libraries. But net/http has its limitations. There is no direct way to handle path parameters. Just like request methods we have to handle path and query parameters manually.

Gorilla Mux is a very popular library that works really well to net/http package and helps us do a few things that makes api building a breeze.

Using Gorilla Mux

To install a module we can use go get

Go get uses git under the hood.

In the same folder you have your go.mod and main.go file run

go get github.com/gorilla/mux

We change our code to this

package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func home(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    switch r.Method {
    case "GET":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "get called"}`))
    case "POST":
        w.WriteHeader(http.StatusCreated)
        w.Write([]byte(`{"message": "post called"}`))
    case "PUT":
        w.WriteHeader(http.StatusAccepted)
        w.Write([]byte(`{"message": "put called"}`))
    case "DELETE":
        w.WriteHeader(http.StatusOK)
        w.Write([]byte(`{"message": "delete called"}`))
    default:
        w.WriteHeader(http.StatusNotFound)
        w.Write([]byte(`{"message": "not found"}`))
    }
}

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", home)
    log.Fatal(http.ListenAndServe(":8080", r))
}

Looks like nothing really changed except for a new import and line 32.

HandleFunc HTTP Methods

But now we can do a little bit more with our HandleFunc Like making each function handle a specific HTTP Method.

It looks something like this

package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"
)

func get(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "get called"}`))
}

func post(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    w.Write([]byte(`{"message": "post called"}`))
}

func put(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusAccepted)
    w.Write([]byte(`{"message": "put called"}`))
}

func delete(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "delete called"}`))
}

func notFound(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusNotFound)
    w.Write([]byte(`{"message": "not found"}`))
}

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", get).Methods(http.MethodGet)
    r.HandleFunc("/", post).Methods(http.MethodPost)
    r.HandleFunc("/", put).Methods(http.MethodPut)
    r.HandleFunc("/", delete).Methods(http.MethodDelete)
    r.HandleFunc("/", notFound)
    log.Fatal(http.ListenAndServe(":8080", r))
}

If you run this it should still do the exact same thing.

At this point you might be wondering how is doing the same thing with more lines of code a good thing?

But think of it this way. Our code became much cleaner and much more readable.

Clear is Better than Clever

Rob Pike

Subrouter

func main() {
    r := mux.NewRouter()
    api := r.PathPrefix("/api/v1").Subrouter()
    api.HandleFunc("", get).Methods(http.MethodGet)
    api.HandleFunc("", post).Methods(http.MethodPost)
    api.HandleFunc("", put).Methods(http.MethodPut)
    api.HandleFunc("", delete).Methods(http.MethodDelete)
    api.HandleFunc("", notFound)
    log.Fatal(http.ListenAndServe(":8080", r))
}

Everything else stays the same except we are creating something called a sub-router. Sub-router are really useful when we want to support multiple resources. Helps us group the content as well as save us from retyping the same path prefix.

We move our api to api/v1 . This way we can create v2 of our api if need be..

Path and Query Parameter

package main

import (
    "fmt"
    "log"
    "net/http"
    "strconv"

    "github.com/gorilla/mux"
)

func get(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "get called"}`))
}

func post(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    w.Write([]byte(`{"message": "post called"}`))
}

func put(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusAccepted)
    w.Write([]byte(`{"message": "put called"}`))
}

func delete(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"message": "delete called"}`))
}

func params(w http.ResponseWriter, r *http.Request) {
    pathParams := mux.Vars(r)
    w.Header().Set("Content-Type", "application/json")

    userID := -1
    var err error
    if val, ok := pathParams["userID"]; ok {
        userID, err = strconv.Atoi(val)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{"message": "need a number"}`))
            return
        }
    }

    commentID := -1
    if val, ok := pathParams["commentID"]; ok {
        commentID, err = strconv.Atoi(val)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{"message": "need a number"}`))
            return
        }
    }

    query := r.URL.Query()
    location := query.Get("location")

    w.Write([]byte(fmt.Sprintf(`{"userID": %d, "commentID": %d, "location": "%s" }`, userID, commentID, location)))
}

func main() {
    r := mux.NewRouter()

    api := r.PathPrefix("/api/v1").Subrouter()
    api.HandleFunc("", get).Methods(http.MethodGet)
    api.HandleFunc("", post).Methods(http.MethodPost)
    api.HandleFunc("", put).Methods(http.MethodPut)
    api.HandleFunc("", delete).Methods(http.MethodDelete)

    api.HandleFunc("/user/{userID}/comment/{commentID}", params).Methods(http.MethodGet)

    log.Fatal(http.ListenAndServe(":8080", r))
}

Lets look at the params functions on line 36. We handle both path param and query params.

With this you now know enough to be dangerous.

Bookdata API

In Kaggle there is a dataset for bookdata. Its a csv file with about 13000 books. We will use it to make our own bookdata api.

books.csv

You can look at the file ☝🏼 there.

Clone Repo

Let's get started.

In a separate folder

git clone https://github.com/moficodes/bookdata-api.git

Tour of the Code

There are two packages inside the code. One is called datastore, one is called loader.

Loader deals with converting the csv data into an array of bookdata objects.

Datastore deals with how we access the data. It's mainly an interface that has some methods.

Run app

From the root of the repo

run

go run .

EndPoints

The App has a few Endpoints

All api endpoints are prefixed with /api/v1

To reach any endpoint use baseurl:8080/api/v1/{endpoint}

Get Books by Author
"/books/authors/{author}" 
Optional query parameter for ratingAbove ratingBelow limit and skip

Get Books by BookName
"/books/book-name/{bookName}"
Optional query parameter for ratingAbove ratingBelow limit and skip


Get Book by ISBN
"/book/isbn/{isbn}"

Delete Book by ISBN
"/book/isbn/{isbn}"

Create New Book
"/book"

Deploy app to cloud

This step is completely optional. But if you want to run you go app in the cloud somewhere IBM Cloud has an excellent paas solution using Cloud Foundry.

If you want to follow along

• Click Here to Get Started
• Install IBM Cloud CLI

Once you have an IBM Cloud account and the IBM Cloud CLI installed,

From terminal

ibmcloud login

ibmcloud target --cf

Open the manifest.yaml in the root of the cloned repository.

Change the app name to something you like. After the change the file should look something like this

---
applications:
- name: <your-app-name>
  random-route: true
  memory: 256M
  env:
    GOVERSION: go1.12
    GOPACKAGENAME: bookdata-api
  buildpack: https://github.com/cloudfoundry/go-buildpack.git

Then from the root of the repo run

ibmcloud cf push

Wait a few minutes and voila! It should be running.

To find your app url

ibmcloud cf apps

You should see your app as running and also have the URL there.

From that url you can test out all the endpoints still work.

Test

You can test my running app mofi-golang-api-demo-appreciative-antelope.mybluemix.net (There is no front-end here, try a endpoint)

If you want to see all the books written by JK Rowling

https://mofi-golang-api-demo-appreciative-antelope.mybluemix.net/api/v1/books/authors/rowling

If you have any question, feel free to ping me @moficodes.