DEV Community

Viacheslav Poturaev
Viacheslav Poturaev

Posted on • Edited on

Tutorial: Developing a RESTful API with Go, JSON Schema validation and OpenAPI docs

#go #openapi #tutorial #rest

This tutorial continues Developing a RESTful API with Go and Gin featured in Go documentation. Please check it first.

TL;DR We're going to replace gin-gonic/gin with swaggest/rest to obtain type-safe OpenAPI spec with Swagger UI and JSON Schema request validation.

Providing reliable and accurate documentation becomes increasingly important thanks to growing integrations between the services. Whether those integrations are between your own microservices, or you are serving an API to 3rd party.

OpenAPI v3 is currently a dominating standard to describe REST API in machine-readable format. There is a whole ecosystem of tools for variety of platforms and languages that help automating integrations and documentation using OpenAPI schema. For example, 3rd party can generate SDK from schema to use your API.

Prerequisites

Let's start with the result of previous tutorial.



package main

import (
    "net/http"

    "github.com/gin-gonic/gin"
)

// album represents data about a record album.
type album struct {
    ID     string  `json:"id"`
    Title  string  `json:"title"`
    Artist string  `json:"artist"`
    Price  float64 `json:"price"`
}

// albums slice to seed record album data.
var albums = []album{
    {ID: "1", Title: "Blue Train", Artist: "John Coltrane", Price: 56.99},
    {ID: "2", Title: "Jeru", Artist: "Gerry Mulligan", Price: 17.99},
    {ID: "3", Title: "Sarah Vaughan and Clifford Brown", Artist: "Sarah Vaughan", Price: 39.99},
}

func main() {
    router := gin.Default()
    router.GET("/albums", getAlbums)
    router.GET("/albums/:id", getAlbumByID)
    router.POST("/albums", postAlbums)

    router.Run("localhost:8080")
}

// getAlbums responds with the list of all albums as JSON.
func getAlbums(c *gin.Context) {
    c.JSON(http.StatusOK, albums)
}

// postAlbums adds an album from JSON received in the request body.
func postAlbums(c *gin.Context) {
    var newAlbum album

    // Call BindJSON to bind the received JSON to
    // newAlbum.
    if err := c.BindJSON(&newAlbum); err != nil {
        return
    }

    // Add the new album to the slice.
    albums = append(albums, newAlbum)
    c.JSON(http.StatusCreated, newAlbum)
}

// getAlbumByID locates the album whose ID value matches the id
// parameter sent by the client, then returns that album as a response.
func getAlbumByID(c *gin.Context) {
    id := c.Param("id")

    // Loop through the list of albums, looking for
    // an album whose ID value matches the parameter.
    for _, a := range albums {
        if a.ID == id {
            c.JSON(http.StatusOK, a)
            return
        }
    }
    c.JSON(http.StatusNotFound, gin.H{"message": "album not found"})
}
Enter fullscreen mode Exit fullscreen mode

Initialize web service

Let's update main function to use a web service for our use case interactors, it will be capable of collecting automated documentation and applying request validation.

Also we can provide basic information about our API using type-safe OpenAPI bindings.



func main() {
    service := web.DefaultService()

    service.OpenAPI.Info.Title = "Albums API"
    service.OpenAPI.Info.WithDescription("This service provides API to manage albums.")
    service.OpenAPI.Info.Version = "v1.0.0"
Enter fullscreen mode Exit fullscreen mode

Add web to imports.



    "github.com/swaggest/rest/web"
Enter fullscreen mode Exit fullscreen mode

Upgrade a handler to return all items

In order to express more information about our http handler, we need refactor it to a use case interactor.

The constructor usecase.NewInteractor takes a generic interact function that should be called for input and prepare data at output pointer.

When web service receives request it will determine correct use case based on route and will prepare instances of input and output for further interaction (call of a function).

Input instance will be filled with data from http request, output instance will be created as a pointer to new output value.

In this case we don't need any request parameters, so input type can be struct{}.

This action will provide a list of albums. So the output type would be a pointer to slice of albums *[]album.



func getAlbums() usecase.Interactor {
    u := usecase.NewInteractor(func(ctx context.Context, _ struct{}, output *[]album) error {
        *output = albums
        return nil
    })
    u.SetTags("Album")

    return u
}
Enter fullscreen mode Exit fullscreen mode

Input and output types are most important for automated http mapping and documentation generation. You can provide more information for the use case, for example tags to group multiple use cases together.

Now we can add upgraded use case to web service (in main function).



    service.Get("/albums", getAlbums())
Enter fullscreen mode Exit fullscreen mode

Upgrade a handler to create new item

In this case we receive input as a JSON payload of album, so input type would be album.

We also return received album in response, so the output would be *album.

Output instance is provided as a placeholder for data, so it has to be a pointer. In contrast, input is not used after interact function is invoked, so it can be a non-pointer value.



func postAlbums() usecase.Interactor {
    u := usecase.NewInteractor(func(ctx context.Context, input album, output *album) error {
        // Add the new album to the slice.
        albums = append(albums, input)

        *output = input
        return nil
    })
    u.SetTags("Album")

    return u
}
Enter fullscreen mode Exit fullscreen mode

Let's implement additional logic in this use case, to restrict id duplicates in the albums. In such case we can return conflict error.



func postAlbums() usecase.Interactor {
    u := usecase.NewInteractor(func(ctx context.Context, input album, output *album) error {
        // Check if id is unique.
        for _, a := range albums {
            if a.ID == input.ID {
                return status.AlreadyExists
            }
        }

        // Add the new album to the slice.
        albums = append(albums, input)

        *output = input
        return nil
    })
    u.SetTags("Album")
    u.SetExpectedErrors(status.AlreadyExists)

    return u
}
Enter fullscreen mode Exit fullscreen mode

As you can see, we've also added u.SetExpectedErrors(status.AlreadyExists) to inform documentation collector that this use case may fail in a particular way.

Now we can add the upgraded use case to web service (in main function).



    service.Post("/albums", postAlbums(), nethttp.SuccessStatus(http.StatusCreated))
Enter fullscreen mode Exit fullscreen mode

Mind the additional option that changes successful status from default http.StatusOK to http.StatusCreated. This fine control is left outside of use case definition because it is specific to http, use case interactor can potentially be used with other transports (see Clean Architecture for more details on this concept).

Add validation to album structure

Now let's add some validation rules to our album structure.



// album represents data about a record album.
type album struct {
    ID     string  `json:"id" required:"true" minLength:"1" description:"ID is a unique string that determines album."`
    Title  string  `json:"title" required:"true" minLength:"1" description:"Title of the album."`
    Artist string  `json:"artist,omitempty" description:"Album author, can be empty for multi-artist compilations."`
    Price  float64 `json:"price" minimum:"0" description:"Price in USD."`
}
Enter fullscreen mode Exit fullscreen mode

Validation rules can be added with field tags (or special interfaces). Along with validation rules you can supply brief descriptions of field values.

  • ID is a required field that can not be empty,
  • Title as well,
  • Artist is an optional field,
  • Price can't be negative.

Validation is powered by JSON Schema.

Upgrade a handler to return specific item

In this case we need to read request parameter from URL path. For that our input structure should contain a field with path tag to enable data mapping.

Given this use case can end up with Not Found status, we add the status to expected errors for documentation. We also wrap the error in use case body to have a correct http status.



func getAlbumByID() usecase.Interactor {
    type getAlbumByIDInput struct {
        ID string `path:"id"`
    }

    u := usecase.NewInteractor(func(ctx context.Context, input getAlbumByIDInput, output *album) error {
        for _, album := range albums {
            if album.ID == input.ID {
                *output = album
                return nil
            }
        }
        return status.Wrap(errors.New("album not found"), status.NotFound)
    })
    u.SetTags("Album")
    u.SetExpectedErrors(status.NotFound)

    return u
}
Enter fullscreen mode Exit fullscreen mode

Now we can add this use case to web service (in main function).



    service.Get("/albums/{id}", getAlbumByID())
Enter fullscreen mode Exit fullscreen mode

Mind the path placeholder has changed from :id to {id} to comply with OpenAPI standard.

Mount Swagger UI

You can add a web interface to the API with Swagger UI.



    service.Docs("/docs", v4emb.New)
Enter fullscreen mode Exit fullscreen mode

Add v4emb to imports.



    "github.com/swaggest/swgui/v4emb"
Enter fullscreen mode Exit fullscreen mode

Then documentation will be served at http://localhost:8080/docs.

Resulting program 



package main

import (
    "context"
    "errors"
    "log"
    "net/http"

    "github.com/swaggest/rest/nethttp"
    "github.com/swaggest/rest/web"
    "github.com/swaggest/swgui/v4emb"
    "github.com/swaggest/usecase"
    "github.com/swaggest/usecase/status"
)

// album represents data about a record album.
type album struct {
    ID     string  `json:"id" required:"true" minLength:"1" description:"ID is a unique string that determines album."`
    Title  string  `json:"title" required:"true" description:"Title of the album."`
    Artist string  `json:"artist,omitempty" description:"Album author, can be empty for multi-artist compilations."`
    Price  float64 `json:"price" minimum:"0" description:"Price in USD."`
}

// albums slice to seed record album data.
var albums = []album{
    {ID: "1", Title: "Blue Train", Artist: "John Coltrane", Price: 56.99},
    {ID: "2", Title: "Jeru", Artist: "Gerry Mulligan", Price: 17.99},
    {ID: "3", Title: "Sarah Vaughan and Clifford Brown", Artist: "Sarah Vaughan", Price: 39.99},
}

func main() {
    service := web.DefaultService()

    service.OpenAPI.Info.Title = "Albums API"
    service.OpenAPI.Info.WithDescription("This service provides API to manage albums.")
    service.OpenAPI.Info.Version = "v1.0.0"

    service.Get("/albums", getAlbums())
    service.Get("/albums/{id}", getAlbumByID())
    service.Post("/albums", postAlbums(), nethttp.SuccessStatus(http.StatusCreated))

    service.Docs("/docs", v4emb.New)

    log.Println("Starting service")
    if err := http.ListenAndServe("localhost:8080", service); err != nil {
        log.Fatal(err)
    }
}

func getAlbums() usecase.Interactor {
    u := usecase.NewInteractor(func(ctx context.Context, _ struct{}, output *[]album) error {
        *output = albums
        return nil
    })
    u.SetTags("Album")

    return u
}

func postAlbums() usecase.Interactor {
    u := usecase.NewInteractor(func(ctx context.Context, input album, output *album) error {
        // Check if id is unique.
        for _, a := range albums {
            if a.ID == input.ID {
                return status.AlreadyExists
            }
        }

        // Add the new album to the slice.
        albums = append(albums, input)

        *output = input
        return nil
    })
    u.SetTags("Album")
    u.SetExpectedErrors(status.AlreadyExists)

    return u
}

func getAlbumByID() usecase.Interactor {
    type getAlbumByIDInput struct {
        ID string `path:"id"`
    }

    u := usecase.NewInteractor(func(ctx context.Context, input getAlbumByIDInput, output *album) error {
        for _, album := range albums {
            if album.ID == input.ID {
                *output = album
                return nil
            }
        }
        return status.Wrap(errors.New("album not found"), status.NotFound)
    })
    u.SetTags("Album")
    u.SetExpectedErrors(status.NotFound)

    return u
}
Enter fullscreen mode Exit fullscreen mode

Tidy modules and start the app!

In order to download necessary modules run go mod tidy in the directory of your module.

Then run the app with go run main.go and open http://localhost:8080/docs.

Swagger UI Screenshot

OpenAPI schema will be available at http://localhost:8080/docs/openapi.json.

openapi.json 


{
 "openapi": "3.0.3",
 "info": {
  "title": "Albums API",
  "description": "This service provides API to manage albums.",
  "version": "v1.0.0"
 },
 "paths": {
  "/albums": {
   "get": {
    "tags": [
     "Album"
    ],
    "summary": "Get Albums",
    "description": "",
    "operationId": "getAlbums",
    "responses": {
     "200": {
      "description": "OK",
      "content": {
       "application/json": {
        "schema": {
         "type": "array",
         "items": {
          "$ref": "#/components/schemas/Album"
         }
        }
       }
      }
     }
    }
   },
   "post": {
    "tags": [
     "Album"
    ],
    "summary": "Post Albums",
    "description": "",
    "operationId": "postAlbums",
    "requestBody": {
     "content": {
      "application/json": {
       "schema": {
        "$ref": "#/components/schemas/Album"
       }
      }
     }
    },
    "responses": {
     "201": {
      "description": "Created",
      "content": {
       "application/json": {
        "schema": {
         "$ref": "#/components/schemas/Album"
        }
       }
      }
     },
     "409": {
      "description": "Conflict",
      "content": {
       "application/json": {
        "schema": {
         "$ref": "#/components/schemas/RestErrResponse"
        }
       }
      }
     }
    }
   }
  },
  "/albums/{id}": {
   "get": {
    "tags": [
     "Album"
    ],
    "summary": "Get Album By ID",
    "description": "",
    "operationId": "getAlbumByID",
    "parameters": [
     {
      "name": "id",
      "in": "path",
      "required": true,
      "schema": {
       "type": "string"
      }
     }
    ],
    "responses": {
     "200": {
      "description": "OK",
      "content": {
       "application/json": {
        "schema": {
         "$ref": "#/components/schemas/Album"
        }
       }
      }
     },
     "404": {
      "description": "Not Found",
      "content": {
       "application/json": {
        "schema": {
         "$ref": "#/components/schemas/RestErrResponse"
        }
       }
      }
     }
    }
   }
  }
 },
 "components": {
  "schemas": {
   "Album": {
    "required": [
     "id",
     "title"
    ],
    "type": "object",
    "properties": {
     "artist": {
      "type": "string",
      "description": "Album author, can be empty for multi-artist compilations."
     },
     "id": {
      "minLength": 1,
      "type": "string",
      "description": "ID is a unique string that determines album."
     },
     "price": {
      "minimum": 0,
      "type": "number",
      "description": "Price in USD."
     },
     "title": {
      "type": "string",
      "description": "Title of the album."
     }
    }
   },
   "RestErrResponse": {
    "type": "object",
    "properties": {
     "code": {
      "type": "integer",
      "description": "Application-specific error code."
     },
     "context": {
      "type": "object",
      "additionalProperties": {},
      "description": "Application context."
     },
     "error": {
      "type": "string",
      "description": "Error message."
     },
     "status": {
      "type": "string",
      "description": "Status text."
     }
    }
   }
  }
 }
}
Enter fullscreen mode Exit fullscreen mode

Source code.

P.S. Original tutorial has a race condition on albums, this is not fixed here as out of scope of the article, but please be aware. :)

Top comments (13)

Collapse
 
servernoj profile image
servernoj •

Is it possible to return a JSON object for error? For example, on the client side, we need to differentiate various kinds of 400(bad request errors), but we don't want to parse the returned string. So we created a simple struct type

type ErrorResponse struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
}
Enter fullscreen mode Exit fullscreen mode

which, in the case of Gin, would be returned by using c.JSON(HTTP.StatusBadRequest, errorResponse). I cannot wrap my head completely into using usecase interactions, so that's the only thing that stops me to migrate away from using Gin (which I don't like for their approach to validation)

Thanks

PS: Great tutorial

Collapse
 
vearutop profile image
Viacheslav Poturaev •

This can be done on a handler processor level, with MakeErrResp configuration.

Please check an example:

github.com/swaggest/rest/blob/v0.2...

    r := openapi31.NewReflector()
    s := web.NewService(r)
.........
    s.Wrap(
        // Example middleware to set up custom error responses and disable response validation for particular handlers.
        func(handler http.Handler) http.Handler {
            var h *nethttp.Handler
            if nethttp.HandlerAs(handler, &h) {
                h.MakeErrResp = func(ctx context.Context, err error) (int, interface{}) {
                    code, er := rest.Err(err)

                    var ae anotherErr

                    if errors.As(err, &ae) {
                        return http.StatusBadRequest, ae
                    }

                    return code, customErr{
                        Message: er.ErrorText,
                        Details: er.Context,
                    }
                }
            }

            return handler
        },
Enter fullscreen mode Exit fullscreen mode

Probably it would be good to have a special interface that could be implemented on a custom error instance to simplify the configuration. 🤔

Collapse
 
servernoj profile image
servernoj •

Thank you for your reply. I saw this example and even (before seeing it) came up with a solution to add a handler (for error logic response generation) as an extra argument to service.Post(). But after doing all that, I started thinking of this problem in terms of supporting multiple kinds of 400 (and other) errors such that my implemented "use cases" would embed all needed details into the returned error.

In this case, the responsibility of the service-wise wrapper would be to identify if the error is of a special kind, and if so, then return the aforementioned struct with 2 fields directly taken from the prepared error object

I think this behavior partially matches the implemented one, but I need to update it to exactly meet the expectation, and here is my main issue:

I find it challenging that the need to build a custom logic requires extensively studying the internals of the library implementation. Not only the examples but the sense of how these examples work. The http.Handler and nethttp.Handler are often confused after a brief look through the sources and require a few extra moments to adjust the thinking.

Lastly, the tight incorporation of the chi router into the implementation in the sense that many internal structs have chi structs embedded, leaves no option but to get rid of the currently used router (gin in my case) and adopt chi instead. I have no objection to using chi, but the lack of flexibility in the early phase of the library adoption doesn't make it an easy walk.

Thread Thread
 
vearutop profile image
Viacheslav Poturaev •

Interesting, now I'm a bit puzzled about your current setup. Could you make a small example app that matches your current approach (with a "misbehaving" error structure)?

The library is built around chi, because router is an essential part of a web application. But I'd be happy to make it easier to use with other cases. As far as I understood, you use gin with response encoder from swaggest/rest. There is an example of integration with gin, but it only focuses on documentation, leaving request decoding and response writing to gin's standard facilities.

Thread Thread
 
servernoj profile image
servernoj •

We first define a bunch of constants like this

const (
    Err404_UnknownError ErrorCode = Err404_Shift + iota + 1
    Err404_ChatGroupNotFound
    Err404_ChannelNotFound
)
Enter fullscreen mode Exit fullscreen mode

to address various scenarios of 404 (and other kinds of) errors. Each constant is of type ErrorCode int (used to generate separate documentation for all errors as illustrated in app.poc.quible.tech/api/v1/docs/er... to be used by the client). The big goal is to introduce "numbers" instead of "strings" for the client to identify the error reason.

We then define a map to associate numerical codes and short descriptions (as shown in the page above) and use that map when a certain gin handler needs to respond with a specific error. For example, SendError(c, http.StatusBadRequest, Err400_InvalidRequestBody)

I am looking for a plug&play replacement of SendError that can be fed with const like Err400_InvalidRequestBody to produce an error returned from the usecase, such that, that error would be handled elsewhere to result in responding with JSON objects

type ErrorResponse struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
}
Enter fullscreen mode Exit fullscreen mode

to clients, given that code and message are those, linked in the error map.

Collapse
 
tonyhsuclrous profile image
Tepisu Cloroc •

Great work!
One of a kind.

Collapse
 
reyesdiego profile image
Diego Reyes •

I don't want to abuse, you helped me the past week. I need to set a cookie in a NewIterator implementation, but I don't get how to use the responseWriter to do

http.setCookie(w, cookie)

Could you please tell me some tips.

u := usecase.NewIOI(models.LoginInput{}, models.LoginOutput{}, func(ctx context.Context, input, output interface{}) error {
        in := input.(models.LoginInput)
        out := output.(*models.LoginOutput)

    cookie := http.Cookie{Domain: "http://local.io:4321", Name: "token", Value: token, Path: "/", Expires: time.Now().Add(1 * time.Hour), MaxAge: 86400}

 http.SetCookie(w, cookie)
 // I don't know from where I get the w

        *out = models.LoginOutput{
            Token: cookie.Value,
        }
        return err
    })
Enter fullscreen mode Exit fullscreen mode
Collapse
 
reyesdiego profile image
Diego Reyes •

Ok.. I've done this solution, but I don't get proper documentation on swagger using .mount :(
By the way, as much I use your your repo I could say it is very very complete...

service.Mount("/login", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        var loginInput models.LoginInput

        err := json.NewDecoder(r.Body).Decode(&loginInput)

        cookie, err := auth.SetCookie(loginInput)
        if err != nil {
            fmt.Println(err)
        }
        http.SetCookie(w, &cookie)
    }))
Enter fullscreen mode Exit fullscreen mode
Collapse
 
reyesdiego profile image
Diego Reyes • • Edited

Very nice work!!.. I would like to be able to return (when paginating) an array in the response body and the total count of the query in a header response X-Total-Count.
But If I set a struct for the response like for instance
// Declare output port type.
type helloOutput struct {
TotalCount int64 header:"X-Total-Count" json:"-"
Data []Vehicles json:"vehicles"
}
I need the response to be
[{
vehicle:1
},{
vehicle:2}
]
and NOT
{
data: [{
vehicle:1
},{
vehicle:2}
]
}

Is there any suggestion to make it possible with the usecase callback function?

Collapse
 
vearutop profile image
Viacheslav Poturaev •

Replied at github.com/swaggest/rest/issues/95....

Collapse
 
alexkapustin profile image
Oleksandr •

Well done! Nice and clean! :)
I would recommend to support openapi.yaml as well.

Collapse
 
vearutop profile image
Viacheslav Poturaev •

Thank you!

Serving openapi.yaml is relatively simple, you just need to marshal spec as YAML and attach http handler to service.

    service.Method(http.MethodGet, "/docs/openapi.yaml", http.HandlerFunc(func(rw http.ResponseWriter, _ *http.Request) {
        document, err := service.OpenAPICollector.Reflector().Spec.MarshalYAML()
        if err != nil {
            http.Error(rw, err.Error(), http.StatusInternalServerError)
        }

        rw.Header().Set("Content-Type", "text/yaml; charset=utf8")

        _, err = rw.Write(document)
        if err != nil {
            http.Error(rw, err.Error(), http.StatusInternalServerError)
        }
    }))
Enter fullscreen mode Exit fullscreen mode
Collapse
 
bethmage profile image
bethmage • • Edited

Very Nice, I have done this solution and Works. Thank you.

View full discussion (13 comments)

Read next

shivamkatare profile image

Create Beautiful Scroll Animations Using Framer Motion

Shivam Katare -

vitor_melo212 profile image

Comandos GIT

Vitor Melo -

moh_moh701 profile image

C# Advanced:Operator Overloading

mohamed Tayel -

furkangozukara profile image

Huge FLUX LoRA vs Fine Tuning / DreamBooth Experiments Completed, Moreover Batch Size 1 vs 7

Furkan Gözükara -