in If you don’t know what code coverage is, lucky you! I think it can stay that way, but in short: code coverage gives you a percentage of what portion of the codebase is covered with tests. In case you are not also aware of what a test is, an entirely new world is ahead of you!
Seriously. Tests are vital for sure. I’m not sure that TDD (test-driven development) is that big deal, to be honest. I use it in some cases, but not in all. It’s one of the tools we devs can use but is not the appropriate solution for every case. And I argue that code coverage is also a friendly tool, which can be used, but not everywhere automatically. For sure, code coverage should not be tracked by management.
But let’s go slowly. Why do I think that? Where are the weaknesses?
I trust that code coverage can be well measured for unit tests, but it’s a different story when integration tests come on stage. Of course, it will depend on your system and the tool’s abilities. Sometimes it can track integration tests as well, but often it will not work that nicely. Mostly when integration tests are not even part of the repository. End-to-end testing, like using Webdriver for websites, is even more problematic. I haven’t seen any tool to cope with that.
We could argue that more essential tests are unit ones anyway, so it’s okay if we focus on them. I believe if there is a way to write a unit test instead of another type of test, it’s better to use it. But in some cases, the end-to-end test is just more suitable. Even though such tests are usually less stable, they might provide better results with less complexity.
To be clear, I don’t want to suggest that not writing tests is better. Quite the contrary, I find tests helpful. I’m trying to say that tests should work for us, not against us. There is no point in having 100% coverage in unit tests when we could cover some parts with simpler end-to-end tests. The time needed to maintain tests should also be counted in deciding what type of coverage is necessary.
Probably the most problematic issue is that no tool can tell me what makes sense to cover. The 80-20 rule can be applied to software: 80% of the work is done by 20% of the code. From my experience, this rule is not far from the truth. That means having 50% coverage is a pretty astonishing number already! The question is: is it the proper 50%?
This question is hard to answer; tools might help but usually don’t. And I have never seen anyone (who cared about percentage) honestly care what percentage is covered. Therefore I think it’s better not to track it anyhow and absolutely not force developers always to increase it. Because writing tests for not crucial parts is easier in most cases, you will end up with tons of tests that are challenging to maintain and not needed at all or could be replaced with one integration test.
Here is what I find helpful about code coverage tools: it gives you not only what percentage is covered but also what files or functions are not. If you are unsure what is covered and what is not in your module and don’t want to miss anything, this is a fantastic tool!
Also, it might be helpful if you have a huge app and look into what is covered. But be careful. Always consult your gut about the most critical part and focus on those uncovered parts. There is not much point in spending valuable time on something minor.
To summarize, running code coverage to track the percentage in your pipelines after every commit wastes resources. But keep the tool around for ad-hoc cases so developers can manually run it to have data for a potentially better decision on where to spend more time in a beautiful world called testing, which makes our lives less stressful.
Writing tests should relax your stress, not the other way around.
feed