DEV Community

Discussion on: How do you measure your test coverage?

Collapse
 
ben profile image
Ben Halpern

However it seems odd to me that you'd have a test for an immutable model that tests all fields.

We ignore files that don't necessarily need to be tested at the moment, even if they don't fit the above criteria. Code coverage is inherently pretty arbitrary so chasing a number without purpose is probably not ideal.

If 100% indicates success, set yourself up for 100% by only measuring the important parts. Once you get near 100% you could expand for other nice to haves.

I'll add by saying that code coverage is a pretty "bad" metric, but no metrics are perfect, and I do think it can serve a purpose when used correctly.

Collapse
 
cubiclebuddha profile image
Cubicle Buddha

Yea I’m with Ben on the imperfection of code coverage as a metric. I set up Istanbul/NYC to ignore files that are just wrappers on API calls, files with constants, and any file where the type system is doing all of he heavy lifting. But honestly, I don’t use code coverage as anything more than a fun thing to watch on the build. Instead, I use the number of bugs that pop up in a given sprint as my measure of quality. And if a bug occurs then I know that my code coverage was low and/or that our tests were flaky. So I immediately go beef up the tests. :)

Collapse
 
johnson_cor profile image
Corey Johnson

I agree, I think having some idea of coverage is better than none. I tend to use acceptance criteria to write tests and then break out unit tests where it makes sense. This gives me a pretty good spread of tests as I work.

Collapse
 
mykezero profile image
Mykezero

If 100% indicates success, set yourself up for 100% by only measuring the important parts. Once you get near 100% you could expand for other nice to haves.

Yes! Aim for 100% on the important things, but just as important, do not sacrifice simplicity for the 100%. If you have a pretty high coverage already, and the core rules are covered, 80~90+ should be fine.

The two rules I try to follow when programming are:

  • Don't make me think.
  • Don't make me work.

If I have to do any of those two while maintaining the code, then something might be off.

If I have to combine 10 components to complete a behavior it's too much work (it might cover the 100% though). If It takes me more than 2 minutes to understand the design, I'm being set up for failure (I'll probably introduce a bug).

The domain and the amount of time you have to manually test things will determine what number your percentage may be (I work on an open source project and since I didn't design it with tests, and I don't have the time any more to test it manually (testing on games take a lot of time), I release almost zero features per year...)

As Woody Zuill would say "Maximize the good". Figure out what works and then maximize it for the best result. No hard or fast rules here: as long as people care for the code base is what matters most.