Go Package for Equality: github.com/google/go-cmp

Software testing boils down, literally, to comparing two values: actual output and expected output. To generate that expected output the steps are usually pretty straightforward:

1. There's an API that receives some sort of input (or inputs), for example a function or a method,
2. That API is invoked with some well-known inputs, and
3. The actual output of the call is compared to what the expected output (or outputs) should be.

Those steps may need to be adjusted depending on the actual API being tested, but that's the common pattern.

Writing tests in Go is not a difficult task because the standard library already includes a package that provides support for automated testing, perhaps the hard part is what method to use when comparing actual and expected results.

To be fair, because most of the times we are comparing basic types using the equal/not equal operators our final goal is achieved easily, let's discuss what's hard about that in other cases.

Comparing results

Below there are some code snippets, please refer to the final repository for actually running the complete code examples.

When using basic types comparing results is simple, for example testing the function Sum below is easy to test:

// operation.go
func Sum(a, b int) int { /* ... */ }

// operation_test.go -> TestSumSimple
if actual := Sum(2, 1); actual != 3 {
    t.Fatalf("expected 3, actual %d", actual)
}

And those operators also work when using non basic types, like a struct type:

// operation.go
type Dollar struct {
    Superunit int
    Subunit   int
}

func SumDollars(a, b Dollar) Dollar { /* ... */ }

// operation_test.go -> TestSumDollars_PrimitiveTypes
expected := Dollar{4, 10}
if actual := SumDollars(Dollar{1, 20}, Dollar{2, 90}); actual != expected {
    t.Fatalf("expected %#v, actual %#v", expected, actual)
}

However it starts to get complicated when non-basic types are involved, like slices or maps. So, what are our options?

Option 1: Use equal and not equal operators

Let's take slices for example, when comparing results doing something like the following works:

// operation.go
func ConcatenateSlice(a, b []int) []int { /* ... */ }

// operation_test.go -> TestConcatenateSlice_Equally
actual := ConcatenateSlice(test.input.a, test.input.b)

if len(actual) != len(test.expected) {
    t.Fatalf("result lengths are different: expected %d, actual %d", len(test.expected), len(actual))
}

for i, v := range actual {
    if v != test.expected[i] {
        t.Fatalf("values at %d index are different: expected %d, actual %d", i, test.expected[i], v)
    }
}

One thing to keep in mind about the code above is that for making sure the slices are the same we are literally comparing them one by one using their index value, and probably that is what we are trying to test, but if instead we were trying to confirm both slices contain the same values no matter their order then our code would change to something like:

// operation_test.go -> TestConcatenateSlice_Semantically
actual := ConcatenateSlice(test.input.a, test.input.b)

if len(actual) != len(test.expected) {
        t.Fatalf("result lengths are different: expected %d, actual %d", len(test.expected), len(actual))
}

found := make(map[int]struct{})

for _, v := range actual {
        found[v] = struct{}{}
}

for _, v := range test.expected {
        if _, ok := found[v]; !ok {
                t.Fatalf("value %d is missing in the result", v)
        }

        delete(found, v)
}

if len(found) != 0 {
        t.Fatal("result does not contain expected values")
}

But that code above does not cover the case when inputs contain duplicated values, we will need to update it for supporting that.

Option 2: Use reflect.DeepEqual

Another way to compare results is using reflect.DeepEqual. This function compares values to be literally equal, so similar to option 1, semantic equality is not supported out of the box:

// operation.go
func ConcatenateSlice(a, b []int) []int { /* ... */ }

// operation_test.go -> TestConcatenateSlice_Reflect
actual := ConcatenateSlice(test.input.a, test.input.b)

if !reflect.DeepEqual(test.expected, actual) {
    t.Fatalf("values are not the same: expected %#v, actual %#v", test.expected, actual)
}

One thing to keep in mind about reflect.DeepEqual is because of the way it is implemented you could get positive results when the values are not actually the same, see this comment for reference.

Option 3: Use github.com/google/go-cmp

Another way to compare results is using the github.com/google/go-cmp package, this alternative compared to the previous two has some very important differences and features.

Default comparison behavior can be overridden if needed

For example to allow results to be the same using approximate values, in this case math.Pi and close-enough values with at least 4 decimals are equal:

// operation_test.go -> TestCmp_ApproximatePi
if !cmp.Equal(math.Pi, test.input, cmpopts.EquateApprox(0, 0.0001)) {
    t.Fatalf("value diverge too far: %f", test.input)
}

or when testing values where we are care about their contents and not their order, like []int slices:

// operation_test.go -> TestCmp_SlicesSematicallySorting
opt := cmpopts.SortSlices(func(a, b int) bool {
    return a < b
})

if !cmp.Equal(test.input, test.output, opt) {
    t.Fatalf("values are not the same %s", cmp.Diff(test.input, test.output, opt))
}

and a similar situation to when comparing maps, like map[int]string:

// operation_test.go -> TestCmp_MapsSematicallySorting
opt := cmpopts.SortMaps(func(a, b int) bool {
    return a < b
})

if !cmp.Equal(test.input, test.output, opt) {
    t.Fatalf("values are not the same %s", cmp.Diff(test.input, test.output, opt))
}

Types can define an Equal method to be used for equality

For example in cases where we need to explicitly allow a different way to indicate equality:

// operation.go
type Message string

func (m Message) Equal(b Message) bool {
    return strings.ToLower(string(m)) == strings.ToLower(string(b))
}

// operation_test.go -> TestMessage_Equal
if !cmp.Equal(test.input, test.output) {
    t.Fatalf("values are not the same %s", cmp.Diff(test.input, test.output))
}

Unexported fields are not compared by default

This is the opposite of what reflect.DeepEqual does. When using this package we have to explicitly indicate what our plans are when comparing unexported fields. To ignore them we can use something like:

// operation.go
type Alert struct {
    Message Message
    code    int
}

// operation_test.go -> TestAlert
if !cmp.Equal(test.input, test.output, cmpopts.IgnoreUnexported(operation.Alert{})) {
    t.Fatalf("values are not the same %s", cmp.Diff(test.input, test.output, cmpopts.IgnoreUnexported(operation.Alert{})))
}

If comparing unexported fields is needed, then using Exporter is the recommended way.

There is a way to get a diff of compared values

I really like this feature, for example:

type Person struct {
    Name string
    Age  int
}

fmt.Println(cmp.Diff(Person{"Name", 99}, Person{"Name", 100}))

Prints out:

  main.Person{
    Name: "Name",
-   Age:  99,
+   Age:  100,
  }

That output is useful to debug test failures when those don't pass.

Final thoughts

Although there are options already available in the standard library, using github.com/google/go-cmp for testing results gives us new flexibility and extra features to indicate how to compare values and determine what exactly is different, not only that but there are another packages like protobuf/testing/protocmp and gotest.tools/v3/assert that act as complementary to this one and could be useful as well depending on the project you're working on and your needs at the moment.

In the end github.com/google/go-cmp is a nice external package that makes testing in Go more pleasant.