TIL: A (maybe) unexpected usage of the -count flag in Go tests

Normally when you add the -count $N flag to the go test command the result is that any tests and/or benchmarks that would be run are run $N times as we can see in the following run.

go test -v -run 1 -bench 1 -count 3
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
goos: linux
goarch: amd64
Benchmark1Sort
Benchmark1Sort-12          10370            113657 ns/op
Benchmark1Sort-12          10488            114596 ns/op
Benchmark1Sort-12           8942            115987 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        5.422s

Just a side note, the 1 put as argument of -run and -bench are regular expressions to identify respectively which tests and which benchmarks to run; for more informations on benchmarks, you can read this article.

Getting back to the situation described in the title, let's look at this situation where we have a test run that is cached after the first time it is run.

$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.017s
$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        (cached)
$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        (cached)

Tests results are cached when they are run in package list mode as long as there are no differences in the package code. This is done in order to avoid repeating tests that are already passing and for which no change has been done.

However, what if we didn't want to use the cache but repeat the tests anyway?

Here is where the flag -count comes to our aid.

$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.019s
$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.015s
$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.014s

As indicated in the documentation that can be found by running go help testflag:

To disable test caching, use any test flag or argument other than the cacheable flags. The idiomatic way to disable test caching explicitly is to use -count=1.

And so this was the unexpected usage of the -count flag in go test, a hidden gem in the go documentation. To read more about this and testing in go in general both go help test and go help testflag are great resources to look at.

Hope this was helpful, thanks a lot for your time reading it!