Code coverage is typically used as a code quality metric.
One can run
go test -coverpkg ./... -coverprofile test.cov ./...
and receive a
test.cov file containing information on how often particular parts of code were triggered during execution.
This is a great tool to keep your tests relevant.
However, code coverage can also help you to debug large codebases!
Imagine a situation when a seemingly innocent change in code leads to broken tests. Getting to the root cause might be non-trivial if there is a lot of code involved in execution.
In smaller cases, you could walk through all the statements with a debugger and compare state and conditions, but when you have thousands of statements, this approach is not really feasible.
Code coverage can help you to narrow down the scope of debugging.
Here is the recipe. The prerequisite is that you have a test that passes in original code, and fails in new.
Collect coverages of both failure and pass.
It is important to collect coverage across all packages to have a wholistic picture. You can use
./... or something like
Apply the change and run the test.
go test -cover -coverpkg ./... -coverprofile new.cover -run ^Test_Foo$ ./path/to/tested/package
Then revert the change and collect coverage again.
go test -cover -coverpkg ./... -coverprofile orig.cover -run ^Test_Foo$ ./path/to/tested/package
After that, you will end up having two files:
They might be big, but hopefully not very different.
Coverage file may look like this.
mode: set github.com/swaggest/jsonschema-go/camelcase.go:14.31,21.22 6 1 github.com/swaggest/jsonschema-go/camelcase.go:21.22,22.27 1 1 github.com/swaggest/jsonschema-go/camelcase.go:22.27,24.4 1 1 github.com/swaggest/jsonschema-go/camelcase.go:26.3,26.27 1 1 github.com/swaggest/jsonschema-go/camelcase.go:26.27,28.4 1 1 github.com/swaggest/jsonschema-go/camelcase.go:30.3,30.27 1 1 .....
The first line says which mode of coverage collection was used.
All the other lines describe a span of code (
file:start_line.start_col,end_line.end_col) with a total number of statements in that span and a number of statements executed.
Coverage files are ordered alphabetically, so they are friendly to
diff orig.cover new.cover > diff.cover
The resulting file may look somewhat similar to this.
305c305 < github.com/swaggest/jsonschema-go/helper.go:75.46,78.16 2 1 --- > github.com/swaggest/jsonschema-go/helper.go:75.46,78.16 2 0 307c307 < github.com/swaggest/jsonschema-go/helper.go:82.2,84.46 2 1 --- > github.com/swaggest/jsonschema-go/helper.go:82.2,84.46 2 0 309c309 < github.com/swaggest/jsonschema-go/helper.go:88.2,88.15 1 1
You can already spot that the lines here are mostly different in the number of executed statements. This is the clue for us that those places in code are the best candidates to have a deeper look with a debugger.
Let's make this diff more convenient to follow and check.
Lines starting with
< represent the original code, lines starting with
> are about the new code.
We can build filtered coverage reports from the diff.
For that, we need to restore the
mode: set and remove
"< " or
"> " from the lines.
echo "mode: set" > orig_flt.cover && grep "< " diff.cover | sed 's/< //' >> orig_flt.cover echo "mode: set" > new_flt.cover && grep "> " diff.cover | sed 's/> //' >> new_flt.cover
Now, there are two coverage files that only contain differences between two runs.
We can conveniently inspect them with standard
go tool cover.
go tool cover -html=orig_flt.cover -o orig_flt.html go tool cover -html=new_flt.cover -o new_flt.html
You can see which parts of code were executed differently, this can lead to an idea of what could be the root cause or at least can give a hint where debugger break point should be set.
I hope you've enjoyed reading this as much as I enjoyed digging into a huuuuge code base with this approach. :)