DEV Community

Cover image for Test in Go: trick of Cleanup
Xuanyu Wang
Xuanyu Wang

Posted on • Originally published at xuanyu.ca

Test in Go: trick of Cleanup

My team is using testify as the testing framework for Go. We noticed that many methods provided by testify/suite can be easily achieved by the built-in testing package. For example, using Cleanup can do the same of AfterTest provided by testify/suite. It makes me wondering if we can just use Cleanup to replace as much methods from third-party packages as possible.

TL;DR

Cleanup can replace methods that are not relevant to subtests, which are AfterTest, TearDownAllSuite and TearDownTestSuite.

Cleanup can’t replace TearDownSubTest because there is a risk that a subtest pollute its parent test.

The experiment

I use a global variable externalResource as the external resource. It’s value is 1.

When preparing a subtest at SetupSubTest, externalResource is changed to 2. At the same time, a cleanup function is registered to restore externalResource to 1.

I expect that externalResource is set to 2 before the subtest and restored to 1 once the subtest is finished. So, from the point of parent test, externalResource is always 1. From the point of subtest, externalResource is always 2.

The code is

var externalResource = 1

type TestUnexpectedCleanupOrder struct {
    suite.Suite
}

func (s *TestUnexpectedCleanupOrder) TearDownTest() {
    fmt.Println("TearDownTest")
}

func (s *TestUnexpectedCleanupOrder) SetupSubTest() {
    externalResource = 2
    fmt.Printf("SetupSubTest. The externalResoure is: %d\n", externalResource)
    s.T().Cleanup(func() {
        externalResource = 1
        fmt.Printf("SetupSubTest. The externalResoure is: %d\n", externalResource)
    })
}

// Test case
func (s *TestUnexpectedCleanupOrder) TestCaseA() {
    fmt.Printf("Start TestCase A. The externalResoure is: %d\n", externalResource)

    s.Run("Sub-TestCase A", func() {
        fmt.Printf("Finish Sub-TestCase A. The externalResoure is: %d\n", externalResource)
    })
    fmt.Printf("Finish TestCase A. The externalResource is: %d\n", externalResource)
}
Enter fullscreen mode Exit fullscreen mode

The result

The output of the above code is (with line number)

1 Start TestCase A. The externalResoure is: 1
2   SetupSubTest. The externalResoure is: 2
3   Finish Sub-TestCase A. The externalResoure is: 2
4 Back to TestCase A. The externalResource is: 2
5 TearDownTest
6   Cleanup subTest. The externalResoure is: 1
Enter fullscreen mode Exit fullscreen mode

Notice line 4 and line 6. The externalResource is still 2 when the subtest is finished and the parent test is resuming! externalResource is not cleaned until the parent test is finished.

According to the documentation of Cleanup:

Cleanup registers a function to be called when the test (or subtest) and all its subtests complete. Cleanup functions will be called in last added, first called order.

It only guarantee that a registered function is called after something.

It does NOT guarantee when a registered function is called before something.

The solution

If we add a TearDownSubTest to reset externalResource, then its value is cleaned as we need

func (s *TestUnexpectedCleanupOrder) TearDownSubTest() {
    externalResource = 1
    fmt.Printf("\tTeardown subTest. The externalResoure is: %d\n", externalResource)
}
Enter fullscreen mode Exit fullscreen mode
Start TestCase A. The externalResoure is: 1
    SetupSubTest. The externalResoure is: 2
    Finish Sub-TestCase A. The externalResoure is: 2
    Teardown subTest. The externalResoure is: 1
Back to TestCase A. The externalResource is: 1
TearDownTest
    Cleanup subTest. The externalResoure is: 1
Enter fullscreen mode Exit fullscreen mode

Summary

We can use Cleanup to clean up resources for normal tests. However, we need to use TearDownSubTest to clean the resource held by subtests.

The source code

https://github.com/xuanyuwang/testify-examples/tree/main/unexpected-cleanup-order

Top comments (0)