Unit testing is fine by Mark Seemann
Unit testing considered harmful? I think not.
Once in a while, some article, video, or podcast makes the rounds on social media, arguing that unit testing is bad, overrated, harmful, or the like. I'm not going to link to any specific resources, because this post isn't an attack on any particular piece of work. Many of these are sophisticated, thoughtful, and make good points, but still arrive at the wrong conclusion.
The line of reasoning tends to be to show examples of bad unit tests and conclude that, based on the examples, unit tests are bad.
The power of examples is great, but clearly, this is a logical fallacy.
Symbolisation #
In case it isn't clear that the argument is invalid, I'll demonstrate it using the techniques I've learned from Howard Pospesel's introduction to predicate logic.
We can begin by symbolising the natural-language arguments into well-formed formulas. I'll keep this as simple as possible:
∃xBx ⊢ ∀xBx (Domain: unit tests; Bx = x is bad)
Basically, Bx states that x is bad, where x is a unit test. ∃xBx is a statement that there exists a unit test x for which x is bad; i.e. bad unit tests exist. The statement ∀xBx claims that for all unit tests x, x is bad; i.e. all unit tests are bad. The turnstile symbol ⊢ in the middle indicates that the antecedent on the left proves the consequent to the right.
Translated back to natural language, the claim is this: Because bad unit tests exist, all unit tests are bad.
You can trivially prove this sequent invalid.
Logical fallacy #
One way to prove the sequent invalid is to use a truth tree:
Briefly, the way this works is that the statements on the left-hand side represent truth, while the ones to the right are false. By placing the antecedent on the left, but the consequent on the right, you're basically assuming the sequent to be wrong. This is is also the way you prove correct sequents true; if the conclusion is assumed false, a logical truth should lead to a contradiction. If it doesn't, the sequent is invalid. That's what happens here.
The tree remains open, which means that the original sequent is invalid. It's a logical fallacy.
Counter examples #
You probably already knew that. All it takes to counter a universal assertion such as all unit tests are bad is to produce a counter example. One is sufficient, because if just a single good unit test exists, it can't be true that all are bad.
Most of the think pieces that argue that unit testing is bad do so by showing examples of bad unit tests. These tests typically involve lots of mocks and stubs; they tend to test the interaction between internal components instead of the components themselves, or the system as a whole. I agree that this often leads to fragile tests.
While I still spend my testing energy according to the Test Pyramid, I don't write unit tests like that. I rarely use dynamic mock libraries. Instead, I push impure actions to the boundary of the system and write most of the application code as pure functions, which are intrinsically testable. No test-induced damage there.
I follow up with boundary tests that demonstrate that the functions are integrated into a working system. That's just another layer in the Test Pyramid, but smaller. You don't need that many integration tests when you have a foundation of good unit tests.
While I'm currently working on a larger body of work that showcases this approach, this blog already has examples of this.
Conclusion #
You often hear or see the claim that unit tests are bad. The supporting argument is that a particular (popular, I admit) style of unit testing is bad.
If the person making this claim only knows of that single style of unit testing, it's natural to jump to the conclusion that all unit testing must be bad.
That's not the case. I write most of my unit tests in a style dissimilar from the interaction-heavy, mocks-and-stubs-based style that most people use. These test have a low maintenance burden and don't cause test-induced damage.
Unit testing is fine.
