Confidence from Facade Tests by Mark Seemann
Recycling an old neologism of mine, I try to illustrate a point about the epistemology of testing function composition.
This article continues the introduction of a series on the epistemology of interaction testing. In the first article, I attempted to explain how to test the composition of functions. Despite my best efforts, I felt that that article somehow fell short of its potential. Particularly, I felt that I ought to have been able to provide some illustrations.
After publishing the first article, I finally found a way to illustrate what I'd been trying to communicate. That's this article. Better late than never.
Previously, on epistemology of interaction testing #
A brief summary of the previous article may be in order. The question this article series tries to address is how to unit test composition of functions - particularly pure functions.
Consider the illustration from the previous article, repeated here for your convenience:
When the leaves are pure functions they are intrinsically testable. That's not the hard part, but how do we test the internal nodes or the root?
What are the alternatives?
An alternative I find useful is to test groups of functions composed together. Particularly when they are pure functions, you have no problem with non-deterministic behaviour. On the other hand, this approach seems to run afoul of the problem with combinatorial explosion of integration testing so eloquently explained by J.B. Rainsberger.
What I suggest, however, isn't quite integration testing.
If it isn't integration testing, then what is it? What do we call it?
I'm going to resurrect and recycle an old term of mine: Facade Tests. Ten years ago I had a more narrow view of a term like 'unit test' than I do today, but the overall idea seems apt in this new context. A Facade Test is a test that exercises a Facade.
These days, I don't find it productive to distinguish narrowly between different kinds of tests. At least not to the the degree that I wish to fight over terminology. On the other hand, occasionally it's useful to have a name for a thing, in order to be able to differentiate it from some other thing.
The term Facade Tests is my attempt at a neologism. I hope it helps.
Code coverage as a proxy for confidence #
The question I'm trying to address is how to test functions that compose other functions - the internal nodes or the root in the above graph. As I tried to explain in the previous article, you need to build confidence that various parts of the composition work. How do you gain confidence in the leaves?
One way is to test each leaf individually.
The first test or two may exercise a tiny slice of the System Under Test (SUT):
The next few tests may exercise another part of the SUT:
Keep adding more tests:
Stop when you have good confidence that the SUT works as intended:
If you're now thinking of code coverage, I can't blame you. To be clear, I haven't changed my position about code coverage. Code coverage is a useless target measure. On the other hand, there's no harm in having a high degree of code coverage. It still might give you confidence that the SUT works as intended.
You may think of the amount of green in the above diagrams as a proxy for confidence. The more green, the more confident you are in the SUT.
None of the arguments here hinge on code coverage per se. What matters is confidence.
Facade testing confidence #
With all the leaves covered, you can move on to the internal nodes. This is the actual problem that I'm trying to address. We would like to test an internal node, but it has dependencies. Fortunately, the context of this article is that the dependencies are pure functions, so we don't have a problem with non-deterministic behaviour. No need for Test Doubles.
It's really simple, then. Just test the internal node until you're confident that it works:
The goal is to build confidence in the internal node, the new SUT. While it has dependencies, covering those with tests is no longer the goal. This is the key difference between Facade Testing and Integration Testing. You're not trying to cover all combinations of code paths in the integrated set of components. You're still just trying to test the new SUT.
Whether or not these tests exercise the leaves is irrelevant. The leaves are already covered by other tests. What 'coverage' you get of the leaves is incidental.
Once you've built confidence in internal nodes, you can repeat the process with the root node:
The test covers enough of the root node to give you confidence in it. Some of the dependencies are also partially exercised by the tests, but this is still secondary. The way I've drawn the diagram, the left internal node is exercised in such a way that its dependencies (the leaves) are partially exercised. The test apparently also exercises the right internal node, but none of that activity makes it interact with the leaves.
These aren't integration tests, so they avoid the problem of combinatorial explosion.
This article was an attempt to illustrate the prose in the previous article. You can unit test functions that compose other functions by first unit testing the leaf functions and then the compositions. While these tests exercise an 'integration' of components, the purpose is not to test the integration. Thus, they aren't integration tests. They're facade tests.