An example of interaction-based testing in C# by Mark Seemann

An example of using Mocks and Stubs for unit testing in C#.

This article is an instalment in an article series about how to move from interaction-based testing to state-based testing. In this series, you'll be presented with some alternatives to interaction-based testing with Mocks and Stubs. Before we reach the alternatives, however, we need to establish an example of interaction-based testing, so that you have something against which you can compare those alternatives. In this article, I'll present a simple example, in the form of C# code.

The code shown in this article is available on GitHub.

Connect two users #

For the example, I'll use a simplified version of the example that runs through my two Clean Coders videos, Church Visitor and Preserved in translation.

The desired functionality is simple: implement a REST API that enables one user to connect to another user. You could imagine some sort of social media platform, or essentially any sort of online service where users might be interested in connecting with, or following, other users.

In essence, you could imagine that a user interface makes an HTTP POST request against our REST API:

POST /connections/42 HTTP/1.1
Content-Type: application/json

{
    "otherUserId": 1337
}

Let's further imagine that we implement the desired functionality with a C# method with this signature:

public IHttpActionResult Post(string userId, string otherUserId)

We'll return to the implementation later, but I want to point out a few things.

First, notice that both userId and otherUserId are string arguments. While the above example encodes both IDs as numbers, essentially, both URLs and JSON are text-based. Following Postel's law, the method should also accept JSON like { "otherUserId": "1337" }. That's the reason the Post method takes string arguments instead of int arguments.

Second, the return type is IHttpActionResult. Don't worry if you don't know that interface. It's just a way to model HTTP responses, such as 200 OK or 400 Bad Request.

Depending on the input values, and the state of the application, several outcomes are possible:

		Other user
		Found	Not found	Invalid
User	Found	Other user	"Other user not found."	"Invalid ID for other user."
	Not found	"User not found."	"User not found."	"User not found."
	Invalid	"Invalid user ID."	"Invalid user ID."	"Invalid user ID."

You'll notice that although this is a 3x3 matrix, there's only five distinct outcomes. This is just an implementation decision. If the first user ID is invalid (e.g. if it's a string like "foo" that doesn't represent a number), then it doesn't matter if the other user exists. Likewise, even if the first user ID is well-formed, it might still be the case that no user with that ID exists in the database.

The assumption here is that the underlying user database uses integers as row IDs.

When both users are found, the other user should be returned in the HTTP response, like this:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "id": 1337,
    "name": "ploeh",
    "connections": [{
        "id": 42,
        "name": "fnaah"
    }, {
        "id": 2112,
        "name": "ndøh"
    }]
}

The intent is that when the first user (e.g. the one with the 42 ID) successfully connects to user 1337, a user interface can show the full details of the other user, including the other user's connections.

Happy path test case #

Since there's five distinct outcomes, you ought to write at least five test cases. You could start with the happy-path case, where both user IDs are well-formed and the users exist.

All tests in this article use xUnit.net 2.3.1, Moq 4.8.1, and AutoFixture 4.1.0.

[Theory, UserManagementTestConventions]
public void UsersSuccessfullyConnect(
    [Frozen]Mock<IUserReader> readerTD,
    [Frozen]Mock<IUserRepository> repoTD,
    User user,
    User otherUser,
    ConnectionsController sut)
{
    readerTD
        .Setup(r => r.Lookup(user.Id.ToString()))
        .Returns(Result.Success<User, IUserLookupError>(user));
    readerTD
        .Setup(r => r.Lookup(otherUser.Id.ToString()))
        .Returns(Result.Success<User, IUserLookupError>(otherUser));
 
    var actual = sut.Post(user.Id.ToString(), otherUser.Id.ToString());
 
    var ok = Assert.IsAssignableFrom<OkNegotiatedContentResult<User>>(
        actual);
    Assert.Equal(otherUser, ok.Content);
    repoTD.Verify(r => r.Update(user));
    Assert.Contains(otherUser.Id, user.Connections);
}

To be clear, as far as Overspecified Software goes, this isn't a bad test. It only has two Test Doubles, readerTD and repoTD. My current habit is to name any Test Double with the TD suffix (for Test Double), instead of explicitly naming them readerStub and repoMock. The latter would have been more correct, though, since the Mock<IUserReader> object is consistently used as a Stub, whereas the Mock<IUserRepository> object is used only as a Mock. This is as it should be, because it follows the rule that you should use Mocks for Commands, Stubs for Queries.

IUserRepository.Update is, indeed a Command:

public interface IUserRepository
{
    void Update(User user);
}

Since the method returns void, unless it doesn't do anything at all, the only thing it can do is to somehow change the state of the system. The test verifies that IUserRepository.Update was invoked with the appropriate input argument.

This is fine.

I'd like to emphasise that this isn't the biggest problem with this test. A Mock like this verifies that a desired interaction took place. If IUserRepository.Update isn't called in this test case, it would constitute a defect. The software wouldn't have the desired behaviour, so the test ought to fail.

The signature of IUserReader.Lookup, on the other hand, implies that it's a Query:

public interface IUserReader
{
    IResult<User, IUserLookupError> Lookup(string id);
}

In C# and most other languages, you can't be sure that implementations of the Lookup method have no side effects. If, however, we assume that the code base in question obeys the Command Query Separation principle, then, by elimination, this must be a Query (since it's not a Command, because the return type isn't void).

For a detailed walkthrough of the IResult<S, E> interface, see my Preserved in translation video. It's just an Either with different terminology, though. Right is equivalent to SuccessResult, and Left corresponds to ErrorResult.

The test configures the IUserReader Stub twice. It's necessary to give the Stub some behaviour, but unfortunately you can't just use Moq's It.IsAny<string>() for configuration, because in order to model the test case, the reader should return two different objects for two different inputs.

This starts to look like Overspecified Software.

Ideally, a Stub should just be present to 'make happy noises' in case the SUT decides to interact with the dependency, but with these two Setup calls, the interaction is overspecified. The test is tightly coupled to how the SUT is implemented. If you change the interaction implemented in the Post method, you could break the test.

In any case, what the test does specify is that when you query the UserReader, it returns a Success object for both user lookups, a 200 OK result is returned, and the Update method was called with user.

Invalid user ID test case #

If the first user ID is invalid (i.e. not an integer) then the return value should represent 400 Bad Request and the message body should indicate as much. This test verifies that this is the case:

[Theory, UserManagementTestConventions]
public void UsersFailToConnectWhenUserIdIsInvalid(
    [Frozen]Mock<IUserReader> readerTD,
    [Frozen]Mock<IUserRepository> repoTD,
    string userId,
    User otherUser,
    ConnectionsController sut)
{
    Assert.False(int.TryParse(userId, out var _));
    readerTD
        .Setup(r => r.Lookup(userId))
        .Returns(Result.Error<User, IUserLookupError>(
            UserLookupError.InvalidId));
 
    var actual = sut.Post(userId, otherUser.Id.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Invalid user ID.", err.Message);
    repoTD.Verify(r => r.Update(It.IsAny<User>()), Times.Never());
}

This test starts with a Guard Assertion that userId isn't an integer. This is mostly an artefact of using AutoFixture. Had you used specific example values, then this wouldn't have been necessary. On the other hand, had you written the test case as a property-based test, it would have been even more important to explicitly encode such a constraint.

Perhaps a better design would have been to use a domain-specific method to check for the validity of the ID, but there's always room for improvement.

This test is more brittle than it looks. It only defines what should happen when IUserReader.Lookup is called with the invalid userId. What happens if IUserReader.Lookup is called with the Id associated with otherUser?

This currently doesn't matter, so the test passes.

The test relies, however, on an implementation detail. This test implicitly assumes that the implementation short-circuits as soon as it discovers that userId is invalid. What if, however, you'd made some performance measurements, and you'd discovered that in most cases, the software would run faster if you Lookup both users in parallel?

Such an innocuous performance optimisation could break the test, because the behaviour of readerTD is unspecified for all other cases than for userId.

Invalid ID for other user test case #

What happens if the other user ID is invalid? This unit test exercises that test case:

[Theory, UserManagementTestConventions]
public void UsersFailToConnectWhenOtherUserIdIsInvalid(
    [Frozen]Mock<IUserReader> readerTD,
    [Frozen]Mock<IUserRepository> repoTD,
    User user,
    string otherUserId,
    ConnectionsController sut)
{
    Assert.False(int.TryParse(otherUserId, out var _));
    readerTD
        .Setup(r => r.Lookup(user.Id.ToString()))
        .Returns(Result.Success<User, IUserLookupError>(user));
    readerTD
        .Setup(r => r.Lookup(otherUserId))
        .Returns(Result.Error<User, IUserLookupError>(
            UserLookupError.InvalidId));
 
    var actual = sut.Post(user.Id.ToString(), otherUserId);
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Invalid ID for other user.", err.Message);
    repoTD.Verify(r => r.Update(It.IsAny<User>()), Times.Never());
}

Notice how the test configures readerTD twice: once for the Id associated with user, and once for otherUserId. Why does this test look different from the previous test?

Why is the first Setup required? Couldn't the arrange phase of the test just look like the following?

Assert.False(int.TryParse(otherUserId, out var _));
readerTD
    .Setup(r => r.Lookup(otherUserId))
    .Returns(Result.Error<User, IUserLookupError>(
        UserLookupError.InvalidId));

If you wrote the test like that, it would resemble the previous test (UsersFailToConnectWhenUserIdIsInvalid). The problem, though, is that if you remove the Setup for the valid user, the test fails.

This is another example of how the use of interaction-based testing makes the tests brittle. The tests are tightly coupled to the implementation.

Missing users test cases #

I don't want to belabour the point, so here's the two remaining tests:

[Theory, UserManagementTestConventions]
public void UsersDoNotConnectWhenUserDoesNotExist(
    [Frozen]Mock<IUserReader> readerTD,
    [Frozen]Mock<IUserRepository> repoTD,
    string userId,
    User otherUser,
    ConnectionsController sut)
{
    readerTD
        .Setup(r => r.Lookup(userId))
        .Returns(Result.Error<User, IUserLookupError>(
            UserLookupError.NotFound));
 
    var actual = sut.Post(userId, otherUser.Id.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("User not found.", err.Message);
    repoTD.Verify(r => r.Update(It.IsAny<User>()), Times.Never());
}
 
[Theory, UserManagementTestConventions]
public void UsersDoNotConnectWhenOtherUserDoesNotExist(
    [Frozen]Mock<IUserReader> readerTD,
    [Frozen]Mock<IUserRepository> repoTD,
    User user,
    int otherUserId,
    ConnectionsController sut)
{
    readerTD
        .Setup(r => r.Lookup(user.Id.ToString()))
        .Returns(Result.Success<User, IUserLookupError>(user));
    readerTD
        .Setup(r => r.Lookup(otherUserId.ToString()))
        .Returns(Result.Error<User, IUserLookupError>(
            UserLookupError.NotFound));
 
    var actual = sut.Post(user.Id.ToString(), otherUserId.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Other user not found.", err.Message);
    repoTD.Verify(r => r.Update(It.IsAny<User>()), Times.Never());
}

Again, notice the asymmetry of these two tests. The top one passes with only one Setup of readerTD, whereas the bottom test requires two in order to pass.

You can add a second Setup to the top test to make the two tests equivalent, but people often forget to take such precautions. The result is Fragile Tests.

Post implementation #

In the spirit of test-driven development, I've shown you the tests before the implementation.

public class ConnectionsController : ApiController
{
    public ConnectionsController(
        IUserReader userReader,
        IUserRepository userRepository)
    {
        UserReader = userReader;
        UserRepository = userRepository;
    }
 
    public IUserReader UserReader { get; }
    public IUserRepository UserRepository { get; }
 
    public IHttpActionResult Post(string userId, string otherUserId)
    {
        var userRes = UserReader.Lookup(userId).SelectError(
            error => error.Accept(UserLookupError.Switch(
                onInvalidId: "Invalid user ID.",
                onNotFound:  "User not found.")));
        var otherUserRes = UserReader.Lookup(otherUserId).SelectError(
            error => error.Accept(UserLookupError.Switch(
                onInvalidId: "Invalid ID for other user.",
                onNotFound:  "Other user not found.")));
 
        var connect =
            from user in userRes
            from otherUser in otherUserRes
            select Connect(user, otherUser);
 
        return connect.SelectBoth(Ok, BadRequest).Bifold();
    }
 
    private User Connect(User user, User otherUser)
    {
        user.Connect(otherUser);
        UserRepository.Update(user);
 
        return otherUser;
    }
}

This is a simplified version of the code shown towards the end of my Preserved in translation video, so I'll refer you there for a detailed explanation.

Summary #

The premise of Refactoring is that in order to be able to refactor, the "precondition is [...] solid tests". In reality, many development organisations have the opposite experience. When programmers attempt to make changes to how their code is organised, tests break. In xUnit Test Patterns this problem is called Fragile Tests, and the cause is often Overspecified Software. This means that tests are tightly coupled to implementation details of the System Under Test (SUT).

It's easy to inadvertently fall into this trap when you use Mocks and Stubs, even when you follow the rule of using Mocks for Commands and Stubs for Queries. In my experience, it's often the explicit configuration of Stubs that tend to make tests brittle. A Command represents an intentional side effect, and you want to verify that such a side effect takes place. A Query, on the other hand, has no side effect, so a black-box test shouldn't be concerned with any interactions involving Queries.

Yet, using an 'isolation framework' such as Moq, FakeItEasy, NSubstitute, and so on, will pull you towards overspecifying the interactions the SUT has with its Query dependencies.

How can we improve? One strategy is to move towards a more functional design, which is intrinsically testable. In the next article, you'll see how to rewrite both tests and implementation in Haskell.

Next: An example of state-based testing in Haskell.

---

Comments

• ← Previous
• Archive
• Next →