How do you comprehensibly unit test two methods that are supposed to have the same behaviour?

Occasionally, you may find yourself in situations where you need to have two (or more) public methods with the same behaviour. How do you properly cover them with unit tests?

The obvious answer is to copy and paste all the tests, but that leads to test duplication. People with a superficial knowledge of DAMP (Descriptive And Meaningful Phrases) would argue that test duplication isn't a problem, but it is, simply because it's more code that you need to maintain.

Another approach may be to simply ignore one of those methods, effectively not testing it, but if you're writing Wildlife Software, you have to have mechanisms in place that can protect you from introducing breaking changes.

This article outlines one approach you can use if your unit testing framework supports Parameterized Tests.

Example problem #

Recently, I was working on a class with a complex AppendAsync method:

public Task AppendAsync(T @event)
{
    // Lots of stuff going on here...
}

To incrementally define the behaviour of that AppendAsync method, I had used Test-Driven Development to implement it, ending with 13 test methods. None of those test methods were simple one-liners; they ranged from 3 to 18 statements, and averaged 8 statements per test.

After having fully implemented the AppendAsync method, I wanted to make the containing class implement IObserver<T>, with the OnNext method implemented like this:

public void OnNext(T value)
{
    this.AppendAsync(value).Wait();
}

That was my plan, but how could I provide appropriate coverage with unit tests of that OnNext implementation, given that I didn't want to copy and paste those 13 test methods?

Parameterized Tests with varied behaviour #

In this code base, I was already using xUnit.net with its nice support for Parameterized Tests; in fact, I was using AutoFixture.Xunit with attribute-based test input, so my tests looked like this:

[TheoryAutoAtomData]
public void AppendAsyncFirstEventWritesPageBeforeIndex(
    [Frozen(As = typeof(ITypeResolver))]TestEventTypeResolver dummyResolver,
    [Frozen(As = typeof(IContentSerializer))]XmlContentSerializer dummySerializer,
    [Frozen(As = typeof(IAtomEventStorage))]SpyAtomEventStore spyStore,
    AtomEventObserver<XmlAttributedTestEventX> sut,
    XmlAttributedTestEventX @event)
{
    sut.AppendAsync(@event).Wait();
 
    var feed = Assert.IsAssignableFrom<AtomFeed>(
        spyStore.ObservedArguments.Last());
    Assert.Equal(sut.Id, feed.Id);
}

This is the sort of test I needed to duplicate, but instead of calling sut.AppendAsync(@event).Wait(); I needed to call sut.OnNext(@event);. The only variation in the test should be in the exercise SUT phase of the test. How can you do that, while maintaining the terseness of using attributes for defining test cases? The problem with using attributes is that you can only supply primitive values as input.

First, I briefly experimented with applying the Template Method pattern, and while I got it working, I didn't like having to rely on inheritance. Instead, I devised this little, test-specific type hierarchy:

public interface IAtomEventWriter<T>
{
    void WriteTo(AtomEventObserver<T> observer, T @event);
}
 
public enum AtomEventWriteUsage
{
    AppendAsync = 0,
    OnNext
}
 
public class AppendAsyncAtomEventWriter<T> : IAtomEventWriter<T>
{
    public void WriteTo(AtomEventObserver<T> observer, T @event)
    {
        observer.AppendAsync(@event).Wait();
    }
}
 
public class OnNextAtomEventWriter<T> : IAtomEventWriter<T>
{
    public void WriteTo(AtomEventObserver<T> observer, T @event)
    {
        observer.OnNext(@event);
    }
}
 
public class AtomEventWriterFactory<T>
{
    public IAtomEventWriter<T> Create(AtomEventWriteUsage use)
    {
        switch (use)
        {
            case AtomEventWriteUsage.AppendAsync:
                return new AppendAsyncAtomEventWriter<T>();
            case AtomEventWriteUsage.OnNext:
                return new OnNextAtomEventWriter<T>();
            default:
                throw new ArgumentOutOfRangeException("Unexpected value.");
        }
    }
}

This is test-specific code that I added to my unit tests, so there's no test-induced damage here. These types are defined in the unit test library, and are not available in the SUT library at all.

Notice that I defined an interface to abstract how to use the SUT for this particular test purpose. There are two small classes implementing that interface: one using AppendAsync, and one using OnNext. However, this small polymorphic type hierarchy can't be supplied from attributes, so I also added an enum, and a concrete factory to translate from the enum to the writer interface.

This test-specific type system enabled me to refactor my unit tests to something like this:

[Theory]
[InlineAutoAtomData(AtomEventWriteUsage.AppendAsync)]
[InlineAutoAtomData(AtomEventWriteUsage.OnNext)]
public void WriteFirstEventWritesPageBeforeIndex(
    AtomEventWriteUsage usage,
    [Frozen(As = typeof(ITypeResolver))]TestEventTypeResolver dummyResolver,
    [Frozen(As = typeof(IContentSerializer))]XmlContentSerializer dummySerializer,
    [Frozen(As = typeof(IAtomEventStorage))]SpyAtomEventStore spyStore,
    AtomEventWriterFactory<XmlAttributedTestEventX> writerFactory,
    AtomEventObserver<XmlAttributedTestEventX> sut,
    XmlAttributedTestEventX @event)
{
    writerFactory.Create(usage).WriteTo(sut, @event);
 
    var feed = Assert.IsAssignableFrom<AtomFeed>(
        spyStore.ObservedArguments.Last());
    Assert.Equal(sut.Id, feed.Id);
}

Notice that this is the same test case as before, but because of the two occurrences of the [InlineAutoAtomData] attribute, each test method is being executed twice: once with AppendAsync, and once with OnNext.

This gave me coverage, and protection against breaking changes of both methods, without making any test-induced damage.

The entire source code in this article is available here. The commit just before the refactoring is this one, and the commit with the completed refactoring is this one.


Comments

I'm not familiar with xUnit, but does it now allow to use something along the lines of nUnit's TestCaseSource to provide the different implementations? From an OOD your implementation is flawless, but I just believe that for tests, simplicity is key so that readability is not affected and intent is instantly captured.
2014-10-29 14:34 UTC

Ricardo, thank you for writing. xUnit.net's equivalent to [TestCaseSource] is to combine the [Theory] attribute with one of the built-in [PropertyData], [ClassData], etc. attributes. However, in this particular case, there's a couple of reasons that I didn't find those attractive:

  • When supplying test data in this way, you can write any code you'd like, but you have to supply values for all the parameters for all the test cases. In the above example, I wanted to vary the System Under Test (SUT), while I didn't care about varying any of the other parameters. The above approach enabled my to do so in a fairly DRY manner.
  • In general, I don't like using [PropertyData], [ClassData] (or NUnit's [TestCaseSource]) unless I truly have a set of reusable test cases. Having a member or type encapsulating a set of test cases implies that these test cases are reusable - but they rarely are.
Perhaps a better approach would be to use something like Exude, but while I wish such semantics were built into xUnit.net or NUnit directly, I didn't want to take a dependency on Exude only for a couple of tests.

2014-10-30 12:26 UTC

That's an interesting take on test reuse. Exude also reminds me of F#'s Expecto. I'd be curious what you think of a few other reuse approaches.

First, I've found expecto's composable tests lists make reuse quite intuitive.

Second, some dev friends of mine have been contemplating test reuse.

The root idea sourced from a friend thinking about the fragile test problem. He concluded that tests should be decoupled from the system. Effectively, the test creates it's own anti-corruption layer which allows tests to stay clean even if the system is messy. This pushes tests to reflect requirements over implementations, which are much more stable. Initially skeptical, I've found significant value in the approach. Systems with role-based interfaces (e.g. ports and adapters) can write one core test suite verifying behaviors for all implementations of that interface.

I've been using this paradigm to map test reuse back to C#. The test list is a class with facts/theories and a constructor that takes the test list's anti-corruption abstraction as a constructor argument. Then I can make implementation-specific tests by inheriting the list and passing an implementation. The same could be done against a role interface instead of a test-specific abstraction if desired.

Taking it even further, I've been wondering if such tests could practically act as acceptance tests between teams as a lower cost alternative to gherkin.

That ended up long. Any depth of feedback would be appreciated.

2022-02-22 02:26 UTC

Spencer, thank you for writing. Tests as first-class values are certainly much to be preferred. That's how I write parametrised tests in Haskell, because HUnit models tests as values. I'm aware that Expecto does the same.

I also agree with you that decoupling tests from implementation details is beneficial. People tend to think of this as only possible with Acceptance Tests or BDD-style tests, but I find that it's also possible in the small. In general, I've found property-based testing to be useful as a technique to add some distance between the test and the system under test.

The idea about using Acceptance Tests between teams sounds like a technique that Ian Robinson calls Consumer-Driven Contracts.

2022-02-25 14:45 UTC

Thank you for your feedback. I'm glad you've also found value in tests as values and decoupling tests from implementations.

Consumer-Driven Contracts does seem to share a lot in common with my idea for using implementation-decoupled tests as acceptance tests. Ian's idea starts from thinking about the provider and how it can manage message compatibility with consumers. Mine started from the consumer and testing multiple implementations of a port abstraction. Then, realizing those tests could also be reused by external consumers (extension contributors, upstream teams).

I think the test-focused approach is able to partially mitigate a few of his concerns. First, the overall added work may be less because the tests already need to be defined for the consumer whether or not there is a complex externally-owned provider. Second, package systems have reached general adoption since his post and greatly improve coordination of shared code. Unit tests [can already be shared with a package manager](https://spencerfarley.com/2021/11/05/acceptance-test-logisitcs/), though there is definitely room for improvement.

I'm a smidge sad because I thought I'd had an original idea, but I'm very glad to discover another author to follow. Thanks for connecting me!

2022-02-28 17:51 UTC


Wish to comment?

You can add a comment to this post by sending me a pull request. Alternatively, you can discuss this post on Twitter or somewhere else with a permalink. Ping me with the link, and I may respond.

Published

Thursday, 09 October 2014 18:45:00 UTC

Tags



"Our team wholeheartedly endorses Mark. His expert service provides tremendous value."
Hire me!
Published: Thursday, 09 October 2014 18:45:00 UTC