Andy Haskell for Salesforce Engineering

Posted on Oct 24, 2022

Using Testify Mock in web clients

#go #testing #webdev #codenewbie

In the last tutorial, we looked at:

How Testify Mock helps us make nondeterministic code testable
How to implement an interface using an embedded Testify Mock
What the general flow of a Testify Mock test looks like

We demonstrated that on random numbers first, since that's the simplest nondeterministic code to be working with.

However, the use case of Testify Mock that ultimately got me learning about that package, was to use it in code that builds on top of clients to web APIs.

The reason code using web API clients is nondeterministic to test is, you don't have total control over what you get back from a web server for a given input, for reasons such as:

✍️ The data you get back for a given server endpoint might change, either slightly or dramatically.
🚧 The server might be down altogether.
🐆 You might be being rate-limited from sending too many requests to the server.
🐛 You'll want test coverage so that your code does the right thing if the server you're talking to is giving back 500s. But since a 500 by definition is an internal server error, it's not necessarily possible to reliably get 500s, especially in the long term.

So for tests where you want the same output every time you run the same API client call in your test, that's a great use case for Testify Mock. So for this tutorial, we'll see how Testify Mock can be used as one of your tools to write test coverage when your Go code's got some Internet to do!

📚 Setting up a fake API to test against

The code we're going to test uses a client for an API that the zoo set up to give facts about animals.

The endpoint our code will be talking to is GET /animal-facts?species={species}&page-token={page-token}, and it returns a JSON object containing a page worth of animal facts, plus a token if there's another page.

If you're following along, copy the code ahead into a file named something like client.go. The main takeaway is that we have a ZooHTTPClient for talking to the zoo's API, and we call the /animal-facts endpoint with the method ListAnimalFacts()

package main

import (
    "fmt"
    "net/http"
)

type ZooHTTPClient struct {
    baseURL string
    client *http.Client
}

// Serialization of error response from zoo API service
type ErrorResponse struct {
    StatusCode int    `json:"status_code"`
    Message    string `json:"message"`
}

func (res *ErrorResponse) Error() string {
    return fmt.Sprintf("got %d error: %s", res.StatusCode, res.Message)
}

type AnimalFactsQuery struct {
    AnimalName string
    PageToken  string
}

func (c *ZooHTTPClient) ListAnimalFacts(q AnimalFactsQuery) (*AnimalFactsResponse, error) {
    // HTTP implementation here; returns an
    // AnimalFactsResponse if the HTTP request succeeds,
    // or an error, of type ErrorResponse, if the request
    // gets a non-2xx HTTP status code.

    // returning nil, nil just so the code compiles
    return nil, nil
}

type AnimalFactsResponse struct {
    Facts         []string `json:"facts"`
    AreThereMore  bool     `json:"are_there_more"`
    NextPageToken string   `json:"next_page_token"`
}

And here is the code we want to test, which we'll copy to a file like main.go. In that code we pass the API client into getSlothsFavoriteSnack so we know what snack to donate a whole bunch of to the zoo. The function returns that snack as a string on success, or if that's not mentioned in any of the facts from the API, then we return errFactNotFound and dust off that library card in order to find that information.

⚠️ NOTE: There is a bug in this code. Don't spend a bunch of time looking for it now, though, we'll fix it when we find that bug using our tests!

package main

import (
    "errors"
    "regexp"
)

var errFactNotFound = errors.New("fact not found")

var favSnackMatcher = regexp.MustCompile(
    "favorite snack is (.*)",
)

func getSlothsFavoriteSnack(c *ZooHTTPClient) (string, error) {
    // Until we're at the last page of facts, call ListAnimalFacts
    // with the current page token to paginate through the list,
    // exiting when either the response's AreThereMore field is
    // false, or we find out what sloths' favorite snack is.
    var pageToken string
    for {
        res, err := c.ListAnimalFacts(AnimalFactsQuery{
            AnimalName: "sloth",
            PageToken:  pageToken,
        })
        if err != nil {
            return "", err
        }

        // check if any facts match the "favorite snack is"
        // regex and if so, return the match
        for _, f := range res.Facts {
            match := favSnackMatcher.FindStringSubmatch(f)
            if len(match) < 2 {
                continue
            }
            return match[1], nil
        }

        // check the response to see if there are any more
        // pages of facts about sloths
        pageToken = res.NextPageToken
    }

    // otherwise if the fact about sloths' favorite snack
    // isn't in the zoo API, return errFactNotFound.
    return "", errFactNotFound
}

Since ListAnimalFacts talks to the server for the zoo's API, the code is nondeterministic. So testing it in a repeatable way is a great use case for Testify Mock.

🥸 Basic test with mock

If you recall from the last tutorial, the steps to doing a test with Testify Mock are:

Take the nondeterministic piece of functionality and wrap it in a Go interface type.
Write an implementation of the interface that uses Testify Mock.
In the tests, use the mock implementation to select deterministic results of the functions your interface calls.

So to start, let's make an interface for our API client.

Wrapping our client in an interface

type ZooClient interface {
    ListAnimalFacts(q AnimalFactsQuery) (*AnimalFactsResponse, error)
}

If we had additional client methods used in our code, such as for other endpoints on the API, we would also add those methods to the ZooClient type as well, but for this tutorial we're only testing code that uses ListAnimalFacts.

Now, to enable using a mock in getSlothsFavoriteSnack, we change that function's signature to take in a ZooClient rather than a *ZooHTTPClient.

- func getSlothsFavoriteSnack(c *ZooHTTPClient) (string, error) {
+ func getSlothsFavoriteSnack(c ZooClient) (string, error) {

Writing a mock implementation of our client

And now just like with the random number generator in the last tutorial, we can use the mock.Mock type to write a mock implementation of our client.

package main

import (
    "github.com/stretchr/testify/mock"
)

type mockClient struct { mock.Mock }

func newMockClient() *mockClient { return &mockClient{} }

func (c *mockClient) ListAnimalFacts(
    q AnimalFactsQuery,
) (*AnimalFactsResponse, error) {
    args := c.Called(q)
    if args.Get(0) == nil {
        return nil, args.Error(1)
    }
    return args.Get(0).(*AnimalFactsResponse), args.Error(1)
}

Just like in the last tutorial, we start by making a mock client that embeds a mock.Mock object, and we retrieve the return values we assigned to the passed-in parameters using Mock.Called.

Unlike in the random number generator example, though, the object we're returning is an *AnimalFactsResponse rather than a primitive Go type like an int. So to get our response as the type we want, we call args.Get(0) which returns an interface{}/any that we then can convert to the type we want with .(*AnimalFactsResponse).

Using our client in a test

Now, let's use our client in a test!

import (
    "testing"
)

var firstPageAPIReq = AnimalFactsQuery{
    AnimalName: "sloth",
    PageToken:  "",
}

func TestGetSlothsFavoriteSnack(t *testing.T) {
    c := newMockClient()
    c.On(
        "ListAnimalFacts", firstPageAPIReq,
    ).Return(&AnimalFactsResponse{
        Facts: []string{
            "Sloths' slowness is actually used as a form of camouflage",
            "Baby sloths make the cutest li'l squeak 🥰",
            "Sloths' favorite snack is hibiscus flowers",
        },
        AreThereMore: false,
        NextPageToken: "",
    }, nil)

    favSnack, err := getSlothsFavoriteSnack(c)
    if err != nil {
        t.Fatalf("got error getting sloths' favorite snack: %v", err)
    }

    if favSnack != "hibiscus flowers" {
        t.Errorf(
            "expected favorite snack to be hibiscus flowers, got %s",
            favSnack,
        )
    }
}

First, we pass in the AnimalFactsQuery we're going to use to c.On() and assign it a mock API response with c.Return().

Now our mock client is ready to use; we call getSlothsFavoriteSnack, and it uses our mock client; when it calls ZooClient.ListAnimalFacts, it gets back our API response, loops through the Facts in the API response, and when it reaches the one titled Sloths' favorite snack is hibiscus flowers, that matches the regular expression, so getSlothsFavoriteSnack returns "hibiscus flowers", nil.

Finally, back in our test, we verify that favSnack is hibiscus flowers and err is nil, just like we would if this test was using a client pointed at and authenticated to a real zoo API.

Run the test, and you'll see that it passed and that sloths love hibiscus flowers! 🌺

Just like that, we've got a Go test on what happens when our code gets back a certain response from an API, and we can run it any time we want with the same result from our mock API; if the zoo's real API server is down, then this test will still run successfully since we're not talking to the real API!

Speaking of when the server is down, we don't want to test just the happy path; we should have some test coverage for what happens if we get an error calling getSlothsFavoriteSnack too.

⛔️ Handling error cases

There's a couple error cases from the API we should have test coverage for:

The server is giving back 500's, which as I mentioned at the beginning of this post, is hard to re-create with a real API.
The API doesn't have the animal fact we want.

Testing that our code handles internal server errors correctly

Let's start with testing the 500 internal server error case.

func TestGetSlothsFavoriteSnack500Error(t *testing.T) {
    c := newMockClient()
    c.On("ListAnimalFacts", firstPageAPIReq).Return(
        (*AnimalFactsResponse)(nil), &ErrorResponse{
            StatusCode: 500,
            Message: "server error",
        },
    )

    _, err := getSlothsFavoriteSnack(c)
    if err == nil {
        t.Fatal("got nil error from getSlothsFavoriteSnack")
    }

    errRes, ok := err.(*ErrorResponse)
    if !ok {
        t.Fatalf("expected error to be ErrorResponse, got %T", err)
    }
    if status := errRes.StatusCode; status != 500 {
        t.Errorf("expected 500, got %d status code", status)
    }
}

Like before, we set up a mock response for firstPageAPIReq in c.On().Return(). But this time, rather than passing in a successful response, we pass in a nil AnimalFactsResponse and an ErrorResponse as our error.

Then, we call getSlothsFavoriteSnack and validate that we got an error, it's of type ErrorResponse, and that its status code is 500. So when we run the test, once again it passes.

Testing the error case where the API doesn't have the sloth fact we want

Now that we've got a test for our 500 error, let's test the case where we don't have the sloth fact we want. In that case we expect that getSlothsFavoriteSnack returns errFactNotFound.

func TestGetSlothsFavoriteSnackNotFound(t *testing.T) {
    c := newMockClient()
    c.On(
        "ListAnimalFacts", firstPageAPIReq,
    ).Return(&AnimalFactsResponse{
        Facts: []string{
            "Sloths' slowness is actually used as a form of camouflage",
            "Baby sloths make the cutest li'l squeak 🥰",
            "Sloths need their beauty sleep, plz send noise-cancelling headphones",
        },
        AreThereMore: false,
        NextPageToken: "",
    }, nil)

    if _, err := getSlothsFavoriteSnack(c); err != errFactNotFound {
        t.Fatalf("should have gotten errFactNotFound, got %v", err)
    }
}

We start with a similar test setup to the success case, except this time around, the response we set up doesn't include information about what sloths like to eat. So at the bottom of the test, we assert that the error we get back is errFactNotFound and that we should head to the library.

Run this test though, and you should get an infinite loop; if you wait about 10 minutes the test will fail because getSlothsFavoriteSnack never returns, so the test times out.

In other words, we found a bug in getSlothsFavoriteSnack. If you recall, this was the for loop in there:

for {
    res, err := c.ListAnimalFacts(AnimalFactQuery{
        AnimalName: "sloth",
        PageToken:  pageToken,
    })
    if err != nil {
        return "", err
    }

    for _, f := range res.Facts {
        match := favSnackMatcher.FindStringSubmatch(f)
        if len(match) < 2 {
            continue
        }
        return match[1], nil
    }

    pageToken = res.NextPageToken
}

We use a for loop because this API is paginated so that if there's more than one page worth of sloth facts, we can check the next page. The bug is that we never break out of the for loop if we're at the end of the list of sloth facts, as specified by the AreThereMore field on the response. If you've tricked out your editor, that might have told you that the line returning errFactNotFound was unreachable, and that's why.

So let's fix the bug!

    for _, f := range res.Facts {
        match := favSnackMatcher.FindStringSubmatch(f)
        if len(match) < 2 {
            continue
        }
        return match[1], nil
    }

    pageToken = res.NextPageToken
+   if !res.AreThereMore {
+       break 
+   }

Run the test again and now our test should pass!

📖 Pagination with MatchedBy, Once, and AssertNumberOfCalls

As one last thing to test, since the API is paginated, we also should have some test coverage for when the information about sloths' favorite snack isn't on page 1.

In this API we're mocking, requests and responses include a PageToken/NextPageToken field, to tell the API which page of sloth facts to retrieve. So a test for the scenario where the search results are on the second page might look like this pseudocode:

Add a mock for page 1, where the AnimalName is "sloth" and PageToken is a blank string.
1. Have that mock API call return our search results, AreThereMore=true, and NextPageToken=NEXT.
Add a mock for page 2, where AnimalName is "sloth" and PageToken is "NEXT".
1. Have that mock API call return a second page of search results; AreThereMore=false and page 2 contains the information that sloths love hibiscus flowers.
Run getSlothsFavoriteSnack and check that "hibiscus flowers" is still retrieved.

We could pass each individual query into Mock.On(), and that wouldn't be too difficult to write in this example code. However, if this API was more complicated and we were testing more-complicated fields, it can be a pain to ensure that each call to Mock.On has exactly the right fields on the structs we're passing in. In particular, you might have a more complicated function than getSlothsFavoriteSnack you want to test, where it takes a lot of staring at your code in order to figure out the exact arguments each individual API call has.

In this test we're running, we're not that worried about the exact content of the PageToken field, since API page tokens tend to be pretty opaque. Whether the page token says something like "page 2", "some more facts on the next page", or a UUID. Essentially, we're not the ones who care about the page token's formst; the zoo's API team is in charge of that!

What we're really testing in a TestGetSlothsFavoriteSnackResultIsOnTheSecondPage (that's a mouthful!) test is that if the information we want is on the second page, we're still able to retrieve it with another query for sloth facts. And there's a handful of functions that can help us test that, to a reasonable "just the important stuff" level of detail, which are:

MatchedBy; instead of passing in an argument to Mock.On() that the argument in the test has to exactly match, you pass in a function that defines the criteria for whether an argument in the test matches the argument to Mock.On().
Once; specify arguments and return values with Mock.On().Return() that are only to be called and given back once. For example in a paginated API, the arguments you call in each round of the loop are nearly the same, so you can add Once() on to your mock call to say "give us back each page of results only once".
AssertNumberOfCalls; if the mocked-out code you're testing is called in a loop, this allows us to check that the loop ran the expected number of times.

To try these out, let's give some test coverage where sloths' favorite snack is on page 2.

func reqSlothFacts(q AnimalFactsQuery) bool {
    switch strings.ToLower(q.AnimalName) {
    case "sloth",
        "lemur: caffeine-free edition",
        "descendants of the giants who made today's avocados happen":

        return true
    default:
        return false
    }
}

var page1 = AnimalFactsResponse{
    Facts: []string{
        "Sloths' slowness is actually used as a form of camouflage",
        "Baby sloths make the cutest li'l squeak 🥰",
        "Sloths need their beauty sleep, plz send noise-cancelling headphones",
    },
    AreThereMore: true,
    NextPageToken: "some more facts on the next page",
}

var page2 = AnimalFactsResponse{
    Facts: []string{"Sloths' favorite snack is hibiscus flowers"},
}

func TestGetSlothsFavoriteSnackOnPage2(t *testing.T) {
    c := newMockClient()
    c.On("ListAnimalFacts", mock.MatchedBy(reqSlothFacts)).
        Return(&page1, nil).
        Once()
    c.On("ListAnimalFacts", mock.MatchedBy(reqSlothFacts)).
        Return(&page2, nil).
        Once()

    favSnack, err := getSlothsFavoriteSnack(c)
    if err != nil {
        t.Fatalf("got error getting sloths' favorite snack: %v", err)
    }

    if favSnack != "hibiscus flowers" {
        t.Errorf(
            "expected favorite snack to be hibiscus flowers, got %s",
            favSnack,
        )
    }

    c.AssertNumberOfCalls(t, "ListAnimalFacts", 2)
}

Here's what happens in our test:

Outside of our test, we define the function reqSlothFacts we'll pass into MatchedBy. We match the word "sloth", as well as a couple other ways to describe a sloth just for fun: "lemur: caffeine-free edition" and "descendants of the giants who made today's avocados happen" (giant ground sloths 10,000 years ago ate what avocados looked like 10,000 years ago, so next time you have some guac, thank a sloth). We also define two pages of search results.
Inside our TestGetSlothsFavoriteSnackOnPage2, we define two calls to ListAnimalFacts similar to before, but this time instead of passing in an exact query, we pass in mock.MatchedBy(reqSlothFacts) to say "a call to ListAnimalFacts matches this call to Mock.On() if its argument, passed into reqSlothFacts, returns true".
On each call to Mock.On(), we add on a call to .Once(). When we call getSlothsFavoriteSnack, the first time we get back the results we defined with page1 and the second time we get back the results we defined with page2.
Then we run the test the same way as before, but at the tail end, we run AssertNumberOfCalls to show that we really did call ListAnimalFacts twice, rather than getting the results on the first page.

By the way, in addition to MatchedBy, if you don't need such fine-grained control over the arguments you pass in, there also is mock.Anything that you can pass into Mock.On to match any argument at all.

For example, a lot of API clients these days take in a context.Context, which is often in charge of things like traces with observability tools or giving a request a deadline to finish with context.WithDeadline. But often that does not affect the headers and body of our request or response. So if the context isn't a focal point of what you're testing, you can mock it out with Anything. For example if our client's interface looked like this:

type ZooClient interface {
    ListAnimalFacts(
        ctx context.Context,
        q AnimalFactsQuery,
    ) (*AnimalFactsResponse, error)
}

You would do your Mock.On call like this:

  c := newMockClient()
  c.On(
      "ListAnimalFacts",
+     mock.Anything,
      firstPageAPIReq,
  ).Return(&AnimalFactsResponse{

🧰 Testify Mock is just one tool for testing your web apps

As you can see, Testify Mock is a convenient tool for testing code that needs to make an API call, particularly when it comes to HTTP responses like 500 errors that you're not suppposed to consistently be getting.

But there's another way we could have tested all these API interactions without having to rely on the real API:

Make an httptest.Server that has HTTP handlers to mimic the API endpoints we're testing, deserializing requests and serializing responses
Point the HTTP implementation of the API client at that server and run tests using that implementation
Do assertions on the mock responses you get back

Both the httptest approach and Testify Mock approach are valid ways to test the code. Furthermore, while both kinds of mocks are useful themselves, that's not to say testing against the real API isn't itself beneficial. By testing against the very same servers your code talks to in production, you can sniff out server behaviors your dev team wasn't aware of from just reading the server's API documentation. While this isn't a hard and fast rule, my personal style for when to use which kind of Go test is:

When I'm the one implementing the client to the API I'm working with, I would go with the httptest Server approach for its unit tests. Part of the client's job is to serialize and deserialize Go types, so the thing being tested, so the focal point of the test is that the headers, request, and response bodies are correct.
For code that's built on top of a client, such as getSlothsFavoriteSnack, my go-to is Testify Mock. Either I wrote and unit-tested the HTTP implementation of the client or the open source developer I'm go getting the client from did. So now the focal point of a test is the question "given that the API client works as expected, does my code do the right thing with the response it gets back"?
- However if you got the client as a dependency but it doesn't have test coverage for the API calls you're making, it might still be a good idea to test using the httptest.Server approach if its implementation allows you to point it at a custom httptest Server URL.
Finally, for testing that your app works end-to-end, having your continuous integration platform run a nightly build that runs tests against the real API will help you detect bugs and unexpected behaviors your mocks might not have taken into account.
- ⚠️WARNING⚠️: Chances are, the server you're talking to in this nightly build will need to obtain and use some kind of API key or authentication token. BE CAREFUL configuring your continuous integration so that that authentication doesn't get stolen. In particular, do not put the key/token directly in your code.