In part 1 of this series, we looked at the basics of writing tests in Go with the testing.T
type, and in part 2, we looked at how with just the testing.T
type, you can organize your tests with its Run
, Skip
, and Cleanup
methods. Even with a good grasp of just testing.T
, you're ready to write professional test coverage in your Go codebase.
But testing.T
and isn't all that the Go standard library provides for writing Go tests! One of the most popular uses of Go is building the server code for web apps, using its detailed net/http package. So the Go Team provided us with a sweet additional package for testing web apps in addition to the main testing
package: net/http/httptest
!
In this tutorial we'll look at:
- 📋 how to test an HTTP handler with httptest
- 💻 how to use httptest to test against a real server
This tutorial is for you if you're interested in Go web development or do webdev already, you're familiar with the basics of Go testing and structs, and you have some familiarity with the HTTP protocol's concepts of requests, responses, headers. If you've used the net/http
package, that will help you follow along, but if you're new to net/http
, this tutorial does have an overview of some Go web app concepts.
📶 A quick overview of Go HTTP handlers
If you already are familiar with Go HTTP handlers, feel free to read the code sample and then skip ahead to the next section. If you're new to Go web development or just want a recap, read on!
Let's start with a recap of one of its core concepts: when you get an HTTP request, you process it using a handler.
Handlers look something like the handleSlothfulMessage
function in this code sample (if you're following along, save this to a file titled server.go
):
package main
import (
"net/http"
)
func handleSlothfulMessage(w http.ResponseWriter, r *http.Request) {
w.Header().Add("Content-Type", "application/json")
w.Write([]byte(`{"message": "Stay slothful!"}`))
}
func appRouter() http.Handler {
rt := http.NewServeMux()
rt.HandleFunc("/sloth", handleSlothfulMessage)
return rt
}
func main() { http.ListenAndServe(":1123", appRouter()) }
All Go HTTP handlers take in an implementation of the ResponseWriter
interface, and a Request
struct. Those objects have the following responsibilities:
- The
Request
contains the data of the HTTP request that the HTTP client (ex a browser or cURL), sends to your web server, like:- The URL that was requested, like the path and query parameters.
-
Request headers, for example a
User-Agent
header saying whether the request comes from Firefox, Chrome, a command-line client, or a punched card someone delivered to a card reader. - The request body for POST and PUT requests.
- The
ResponseWriter
is in charge of formulating the HTTP response, so it handles things like:- Writing status codes, like 200 for a successful request, or the familiar "404 file not found" websites make corny webpages for.
- Writing Response headers, such as the
Content-Type
to say what format our response is in, like HTML or JSON. - Writing the response body, which can be any response format, like an HTML webpage, a JSON object, or picture of a sloth 🌺!
In our handleSlothfulMessage
function, we first add a header to indicate that our response is JSON with the line:
w.Header().Add("Content-Type", "application/json")
And then we write out the bytes of a JSON message that has a header saying "Stay slothful!" with the line:
w.Write([]byte(`{"message": "Stay slothful!"}`))
Note that because we didn't call w.WriteHeader
to explicitly select a status code for our HTTP response, the response's code will be 200/OK. If we had some kind of error scenario we'd need to handle in one of our web app's handlers, for example issues talking to a database, then in the handler function we might have code like this to give a 500/internal server error response:
if errorScenarioOccurs {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte(`{"error_message": "description of the error"}`))
return
}
// rest of HTTP handler
In the appRouter
function, we make a router so requests to our /sloth
endpoint are handled with the handleSlothfulMessage
function:
func appRouter() http.Handler {
rt := http.NewServeMux()
rt.HandleFunc("/sloth", handleSlothfulMessage)
return rt
}
Finally, in the main
function, we start an HTTP server on port 1123 with http.ListenAndServe
, causing all requests to localhost:1123
to be handled by the HTTP router we made in appRouter
.
Now if you start this Go program and then go to localhost:1123/sloth
in your browser or send a request to it via a client like cURL or Postman, you can see that we got back a simple JSON object!
📋 Testing your handler with httptest
As you can see, you can start using your HTTP handler in a real web server without a lot of code. When you're running a net/http
server with http.ListenAndServe
, Go does the work behind the scenes for you of making http.Request
and ResponseWriter
objects when a request comes in from a client like your browser.
But that does raise the question, where do we get a ResponseWriter
and Request
in our Go code when we're testing our HTTP handlers inside go test
?
Luckily, the standard library has convenient code for testing that in the net/http/httptest package to facilitate writing test coverage for that, with functions and types like:
- The
NewRequest
function for making the*http.Request
. - a
ResponseRecorder
type that both implements thehttp.ResponseWriter
interface and lets you replay the HTTP response. - a
Server
type for testing Go code that sends HTTP requests by setting up a real HTTP server to send them to.
To see this httptest in action, let's see how we would test handleSlothfulMessage
. If you're following along, save this code to server_test.go
:
package main
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestHandleSlothfulMessage(t *testing.T) {
wr := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/sloth", nil)
handleSlothfulMessage(wr, req)
if wr.Code != http.StatusOK {
t.Errorf("got HTTP status code %d, expected 200", wr.Code)
}
if !strings.Contains(wr.Body.String(), "Stay slothful!") {
t.Errorf(
`response body "%s" does not contain "Stay slothful!"`,
wr.Body.String(),
)
}
}
Run go test -v
, and you should see a passing test!
Let's take a look at what happened, and how we're using httptest in our code:
First, there's making the ResponseWriter and Request:
wr := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/sloth", nil)
We make our net/http ResponseWriter
implementation with NewRecorder
, and a Request
object pointed at our /sloth
endpoint using NewRequest
.
Then, we run our HTTP handler:
handleSlothfulMessage(wr, req)
Remember that Go HTTP handler functions are just regular old Go functions that happen to take in specialized net/http
objects. That means we can run a handler without any actual HTTP server if we have a ResponseWriter and Request to pass into it. So we run our handler by passing wr
and req
into a call to our handleSlothfulMessage
function. Or if we wanted to test our web app's entire router rather than just one endpoint, we could even run appRouter().ServeHTTP(wr, req)
!
Then, in the next piece of code, we check out the results of running handleSlothfulMessage
:
if wr.Code != http.StatusOK {
t.Errorf("got HTTP status code %d, expected 200", wr.Code)
}
An httptest ResponseRecorder
implements the ResponseWriter
interface, but that's not all it gives us! It also has struct fields we can use for examining the response we get back from our HTTP request. One of them is Code
; we expect our response to be a 200, so we have an assertion comparing our status code to http.StatusOK
.
Additionally, a ResponseRecorder
makes it easy to look at the body of our response. It gives us a bytes.Buffer
field titled Body
that recorded the bytes of the response body. So we can test that our HTTP response contains the string "Stay slothful!", having our test fail if it does not.
if !strings.Contains(wr.Body.String(), "Stay slothful!") {
t.Errorf(
`response body "%s" does not contain "Stay slothful!"`,
wr.Body.String(),
)
}
By the way, this technique also works with POST requests. If we had an endpoint that took in a POST request with an encoded JSON body, then sending the request in the test for that endpoint would look something like this:
var b bytes.Buffer
err := json.NewEncoder(b).Encode(objectToSerializeToJSON)
if err != nil {
t.Fatal(err)
}
wr := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/post-endpoint", b)
handlePostEndpointRequest(wr, req)
First we set up a bytes.Buffer
to use as our POST request's body. This is because a net/http Request's body needs to be an implementation of the io.Reader interface. bytes.Buffer
conveniently has the Read
method, so it implements that interface. We then use json.NewEncoder(b).Encode to convert a Go struct into bytes of JSON that get stored in the buffer.
We make our POST request by passing MethodPost
, rather than MethodGet
, into httptest.NewRequest
. Our bytes.Buffer
is passed in as the last argument to NewRequest
as the request body. Finally, just like before, we call our HTTP request handler using our ResponseRecorder and Request.
💻 The httptest server
Not only does httptest
provide us a way to test our handlers with requests and responses, it even provides ways to test your code with a real HTTP server!
A couple scenarios where this is useful are:
- When you're doing functional tests or integration tests for a web app. For example, testing that communication goes as expected between different microservices.
- If you are implementing clients to other web servers, you can define an httptest server giving back a response in order to test that your client can handle both sending the correct request and processing the server's response correctly.
Let's try out the latter of these scenarios, by making a client struct to send an HTTP request to our /sloth
endpoint, and deserialize the response into a struct.
First, import fmt
and encoding/json
(and net/http
if you're putting the client code in its own file) and then write this code for the client. If you're newer to JSON deserialization, no worries if the code doesn't click 100% for you. The main things you need to know are:
- The client we're making has a
GetSlothfulMessage
that sends an HTTP request to the/sloth
of itsbaseURL
. - Using Go's awesome
encoding/json
package, the HTTP response body is converted to aSlothfulMessage
struct, which is returned if the request and JSON deserialization are successful. We are using json.NewDecoder(res.Body).Decode for reading the response body into ourSlothfulMessage
struct. - If we get a non-200 HTTP status code sending the request, or there's a problem deserializing the JSON response, then
GetSlothfulMessage
instead returns an error.
type Client struct {
httpClient *http.Client
baseURL string
}
type SlothfulMessage struct {
Message string `json:"message"`
}
func NewClient(httpClient *http.Client, baseURL string) Client {
return Client{
httpClient: httpClient,
baseURL: baseURL,
}
}
func (c *Client) GetSlothfulMessage() (*SlothfulMessage, error) {
res, err := c.httpClient.Get(c.baseURL + "/sloths")
if err != nil {
return nil, err
}
if res.StatusCode != http.StatusOK {
return nil, fmt.Errorf(
"got status code %d", res.StatusCode,
)
}
var m SlothfulMessage
if err := json.NewDecoder(res.Body).Decode(&m); err != nil {
return nil, err
}
return &m, nil
}
We've got our client, so let's see how we can test it with an httptest.Server
:
func TestGetSlothfulMessage(t *testing.T) {
router := http.NewServeMux()
router.HandleFunc("/sloth", handleSlothfulMessage)
svr := httptest.NewServer(router)
defer svr.Close()
c := NewClient(http.DefaultClient, svr.URL)
m, err := c.GetSlothfulMessage()
if err != nil {
t.Fatalf("error in GetSlothfulMessage: %v", err)
}
if m.Message != "Stay slothful!" {
t.Errorf(
`message %s should contain string "Sloth"`,
m.Message,
)
}
}
Here's what happens:
First, we start our server:
svr := httptest.NewServer(appRouter())
defer svr.Close()
We pass in our web app's HTTP router as a request handler for the server. When we run NewServer
, a server is set up to run on your localhost, on a randomized port. In fact, if you had your test run time.Sleep
to pause for a while, you could actually go to that server in your own browser!
Now that we've got our server, we set up our client and have it test an HTTP roundtrip to our /sloth
endpoint:
c := NewClient(http.DefaultClient, svr.URL)
m, err := c.GetSlothfulMessage()
The base URL we give to the Client
, is the URL of the server, which is the randomized port I mentioned earlier. So a request might go out to somewhere like "localhost:1123/sloths", or "localhost:5813/sloths". It all depends on which port httptest.NewServer
picks!
Finally, we check that we didn't get an error, and that the response is what we expected. If we run go test -v, we'll get:
=== RUN TestHandleSlothfulMessage
-------- PASS: TestHandleSlothfulMessage (0.00s)
=== RUN TestGetSlothfulMessage
webapp_test.go:37: error in GetSlothfulMessage: got status code 404
-------- FAIL: TestGetSlothfulMessage (0.00s)
A failing test, because we got a 404 response, not the 200 we expected. So that means there's a bug in our client.
The part of GetSlothfulMessage
that was for sending our HTTP request was:
func (c *Client) GetSlothfulMessage() (*SlothfulMessage, error) {
res, err := c.httpClient.Get(c.baseURL + "/sloths")
if err != nil {
return nil, err
}
As you can see, we're sending the request to c.baseURL + "/sloths"
. We wanted to send it to /sloth
, not /sloths
. So fix that code, run go test -v
, and now...
=== RUN TestHandleSlothfulMessage
-------- PASS: TestHandleSlothfulMessage (0.00s)
=== RUN TestGetSlothfulMessage
-------- PASS: TestGetSlothfulMessage (0.00s)
Your test should pass!
As you can see, with the httptest
package's ResponseRecorder
and Server
objects, we've got the ability to take the concepts we were already working with for writing tests using the testing
package, and then start using functionality to test both receiving and sending HTTP requests. Definitely a must-know package in a Go web developer's toolbelt!
Thank you to my coworker Aaron Taylor for peer-reviewing this blog post!
Top comments (2)
[[Pingback]]
Curated as a part of #18th Issue of Software Testing Notes newsletter.
softwaretestingnotes.substack.com/...
thanks! this really helped me understand how to write simple tests for my handlers