Testing your API client in Go - a method

Testing is unquestionably an art. Between deciding what to test, how and making your code testable in the first place, the road is not straightforward. Well, not at first.

"Pragmatic" Dave (Thomas) famously doesn't test. His explanation makes (a sort of) sense in that given his (very) vast experience, he's been writing testable code to such an extent that problems become apparent right off the bat and so going the last mile to write tests is superfluous.

I wish I had that kind of mastery and experience, but alas I am but a plebeian relegated to actually writing tests.

Testing in Go

Go (often written as Golang) is one of those rare languages that packs everything you'd ever need, including for testing. The testing package provides you with some basics and if you're writing your own HTTP server, then httptest is what you're looking for to mock requests and assert responses.

The Problem

Here's the gist of the application to test. We're talking a service that acts as a proxy between some services and an external third party. You have one main and several additional services that need to interact with a third party (sometimes synchronously but also asynchronously). The application exposes some HTTP endpoints to receive requests, but also processes a queue and in the end makes requests to the third party, processes the results for simplification and returns data.

Structure

There's obviously an API client struct that holds a HTTP client as well as credentials. It has some functions attached for each type of call it performs, that satisfy an interface. Each function takes a request DTO built from received HTTP requests (or queue messages) and returns a response DTO that's ready to be serialised.

The API client methods are called by a service layer where each relevant function takes an API client, does the processing of the result and returns the response that our API needs to provide.

What to test? (take 1)

Since our client satisfies an interface, we could simply create a mock (test) client that returns some predefined response DTOs.

That's good enough to test the service layer and will definitely catch the bulk of service logic issues as well as some data issues (like serializing responses - probably)

Caveat between reading the response from the third party and returning the DTO there's still an important (albeit mundane) process: deserializing the (JSON) response from the third party and the construction of the DTO. Any data structure changes on our side (we should unmarshal json to a struct - directly into a DTO or some intermediary) would affect our ability to process the response (resulting in unmarshaling errors or missing data).

What to test2 (take 2)

OK, so mocking the API client isn't good enough. We should ... mock the HTTP client? Go doesn't provide that. Well, not the mocking of the client, per se.

However, when creating a client, we can specify a Transport. The Transport needs to satisfy the RoundTripper interface, that is, to have a RoundTrip method that takes a pointer to a http.Request and returns a pointer to http.Response.

// RoundTripFunc .
type RoundTripFunc func(req *http.Request) *http.Response

// RoundTrip .
func (f RoundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
    return f(req), nil
}

//NewTestClient returns *http.Client with Transport replaced to avoid making real calls
func NewTestClient(fn RoundTripFunc) MyProxyClient {
    return MyProxyClient{
        HTTPClient: &http.Client{
            Transport: RoundTripFunc(fn),
        },
        TPUsername: "user",
        TPPassword: "pass",
    }
}

There you have it. A Function that creates a test client. In a test, you will need to (of course!) provide the actual RountTripFunc implementation, which will result in a predefined response return to the function you want to test.

func TestGetToken(t *testing.T) {
    client := NewTestClient(func(req *http.Request) *http.Response {
        // Test request parameters
        assert.Equal(t, req.URL.String(), "https://thirdparty.url.com/api/token/CreateToken")
        return &http.Response{
            StatusCode: 200,
            // Send response to be tested
            Body: ioutil.NopCloser(strings.NewReader("{\"token\":\"testToken\"}")),
            // Must be set to non-nil value or it panics
            Header: make(http.Header),
        }
    })

    token, err := GetToken(client, NewTestEnvironment())
    assert.Nil(t, err)

    assert.Equal(t, token.Token, "testToken")
}

Here I am testing a function called GetToken which calls the third party to get a token to use for other requests. Not pictured: a function NewTestEnvironment that provides some extra context (also to be mocked in my case)

What I'm checking? I'm checking that this function calls a certain URL (that URL is based off a root that comes from that NewTestEnvironment). My RoundTrip returns a JSON string, of which a certain value (of the token) I expect to have in the DTO produced (if all goes well) and of course: no errors.

Conclusion

API client testing can become trickier than API server testing, in that you're only limited to which cases you want to cover. You can cover marshalling errors, missing data pieces, unexpected data and so on.

There's another method, where you actually spin up an HTTP server to which you can make calls to and that will return predefined responses (perhaps from some hardcoded files/templates).

This comes with some extra overhead. Personally I feel providing a mock Transport through a mock Client provides more control and sidesteps any performance issue that could come from starting an HTTP server (which may not be possible, depending on where you end up running the tests).