An example of state-based testing in C# by Mark Seemann

An example of avoiding Mocks and Stubs in C# unit testing.

This article is an instalment in an article series about how to move from interaction-based testing to state-based testing. In the previous article, you saw an example of a pragmatic state-based test in F#. You can now take your new-found knowledge and apply it to the original C# example.

In the spirit of xUnit Test Patterns, in this article you'll see how to refactor the tests while keeping the implementation code constant.

The code shown in this article is available on GitHub.

Connect two users #

The previous article provides more details on the System Under Test (SUT), but here it is, repeated, for your convenience:

public class ConnectionsController : ApiController
{
    public ConnectionsController(
        IUserReader userReader,
        IUserRepository userRepository)
    {
        UserReader = userReader;
        UserRepository = userRepository;
    }
 
    public IUserReader UserReader { get; }
    public IUserRepository UserRepository { get; }
 
    public IHttpActionResult Post(string userId, string otherUserId)
    {
        var userRes = UserReader.Lookup(userId).SelectError(
            error => error.Accept(UserLookupError.Switch(
                onInvalidId: "Invalid user ID.",
                onNotFound:  "User not found.")));
        var otherUserRes = UserReader.Lookup(otherUserId).SelectError(
            error => error.Accept(UserLookupError.Switch(
                onInvalidId: "Invalid ID for other user.",
                onNotFound:  "Other user not found.")));
 
        var connect =
            from user in userRes
            from otherUser in otherUserRes
            select Connect(user, otherUser);
 
        return connect.SelectBoth(Ok, BadRequest).Bifold();
    }
 
    private User Connect(User user, User otherUser)
    {
        user.Connect(otherUser);
        UserRepository.Update(user);
 
        return otherUser;
    }
}

This implementation code is a simplification of the code example that serves as an example running through my two Clean Coders videos, Church Visitor and Preserved in translation.

A Fake database #

As in the previous article, you can define a test-specific Fake database:

public class FakeDB : Collection<User>, IUserReader, IUserRepository
{
    public IResult<User, IUserLookupError> Lookup(string id)
    {
        if (!(int.TryParse(id, out int i)))
            return Result.Error<User, IUserLookupError>(UserLookupError.InvalidId);
 
        var user = this.FirstOrDefault(u => u.Id == i);
        if (user == null)
            return Result.Error<User, IUserLookupError>(UserLookupError.NotFound);
 
        return Result.Success<User, IUserLookupError>(user);
    }
 
    public bool IsDirty { get; set; }
 
    public void Update(User user)
    {
        IsDirty = true;
        if (!Contains(user))
            Add(user);
    }
}

This is one of the few cases where I find inheritance more convenient than composition. By deriving from Collection<User>, you don't have to explicitly write code to expose a Retrieval Interface. The entirety of a standard collection API is already available via the base class. Had this class been part of a public API, I'd be concerned that inheritance could introduce future breaking changes, but as part of a suite of unit tests, I hope that I've made the right decision.

Although you can derive a Fake database from a base class, you can still implement required interfaces - in this case IUserReader and IUserRepository. The Update method is the easiest one to implement, since it simply sets the IsDirty flag to true and adds the user if it's not already part of the collection.

The IsDirty flag is the only custom Retrieval Interface added to the FakeDB class. As the previous article explains, this flag provides a convenient was to verify whether or not the database has changed.

The Lookup method is a bit more involved, since it has to support all three outcomes implied by the protocol:

• If the id is invalid, a result to that effect is returned.
• If the user isn't found, a result to that effect is returned.
• If the user with the requested id is found, then that user is returned.

Happy path test case #

This is all you need in terms of Test Doubles. You now have a test-specific IUserReader and IUserRepository implementation that you can pass to the Post method. Notice that a single class implements multiple interfaces. This is often key to be able to implement a Fake object in the first place.

Like in the previous article, you can start by exercising the happy path where a user successfully connects with another user:

[Theory, UserManagementTestConventions]
public void UsersSuccessfullyConnect(
    [Frozen(Matching.ImplementedInterfaces)]FakeDB db,
    User user,
    User otherUser,
    ConnectionsController sut)
{
    db.Add(user);
    db.Add(otherUser);
    db.IsDirty = false;
 
    var actual = sut.Post(user.Id.ToString(), otherUser.Id.ToString());
 
    var ok = Assert.IsAssignableFrom<OkNegotiatedContentResult<User>>(actual);
    Assert.Equal(otherUser, ok.Content);
    Assert.True(db.IsDirty);
    Assert.Contains(otherUser.Id, user.Connections);
}

This, and all other tests in this article use xUnit.net 2.3.1 and AutoFixture 4.1.0.

The test is organised according to my standard heuristic for formatting tests according to the Arrange Act Assert pattern. In the Arrange phase, it adds the two valid User objects to the Fake db and sets the IsDirty flag to false.

Setting the flag is necessary because this is object-oriented code, where objects have mutable state. In the previous articles with examples in F# and Haskell, the User types were immutable. Connecting two users didn't mutate one of the users, but rather returned a new User value, as this F# example demonstrates:

// User -> User -> User
let addConnection user otherUser =
    { user with ConnectedUsers = otherUser :: user.ConnectedUsers }

In the current object-oriented code base, however, connecting one user to another is an instance method on the User class that mutates its state:

public void Connect(User otherUser)
{
    connections.Add(otherUser.Id);
}

As a consequence, the Post method could, if someone made a mistake in its implementation, call user.Connect, but forget to invoke UserRepository.Update. Even if that happened, then all the other assertions would pass. This is the reason that you need the Assert.True(db.IsDirty) assertion in the Assert phase of the test.

While we can apply to object-oriented code what we've learned from functional programming, the latter remains simpler.

Error test cases #

While there's one happy path, there's four distinct error paths that you ought to cover. You can use the Fake database for that as well:

[Theory, UserManagementTestConventions]
public void UsersFailToConnectWhenUserIdIsInvalid(
    [Frozen(Matching.ImplementedInterfaces)]FakeDB db,
    string userId,
    User otherUser,
    ConnectionsController sut)
{
    Assert.False(int.TryParse(userId, out var _));
    db.Add(otherUser);
    db.IsDirty = false;
 
    var actual = sut.Post(userId, otherUser.Id.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Invalid user ID.", err.Message);
    Assert.False(db.IsDirty);
}
 
[Theory, UserManagementTestConventions]
public void UsersFailToConnectWhenOtherUserIdIsInvalid(
    [Frozen(Matching.ImplementedInterfaces)]FakeDB db,
    User user,
    string otherUserId,
    ConnectionsController sut)
{
    Assert.False(int.TryParse(otherUserId, out var _));
    db.Add(user);
    db.IsDirty = false;
 
    var actual = sut.Post(user.Id.ToString(), otherUserId);
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Invalid ID for other user.", err.Message);
    Assert.False(db.IsDirty);
}
 
[Theory, UserManagementTestConventions]
public void UsersDoNotConnectWhenUserDoesNotExist(
    [Frozen(Matching.ImplementedInterfaces)]FakeDB db,
    int userId,
    User otherUser,
    ConnectionsController sut)
{
    db.Add(otherUser);
    db.IsDirty = false;
 
    var actual = sut.Post(userId.ToString(), otherUser.Id.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("User not found.", err.Message);
    Assert.False(db.IsDirty);
}
 
[Theory, UserManagementTestConventions]
public void UsersDoNotConnectWhenOtherUserDoesNotExist(
    [Frozen(Matching.ImplementedInterfaces)]FakeDB db,
    User user,
    int otherUserId,
    ConnectionsController sut)
{
    db.Add(user);
    db.IsDirty = false;
 
    var actual = sut.Post(user.Id.ToString(), otherUserId.ToString());
 
    var err = Assert.IsAssignableFrom<BadRequestErrorMessageResult>(actual);
    Assert.Equal("Other user not found.", err.Message);
    Assert.False(db.IsDirty);
}

There's little to say about these tests that hasn't already been said in at least one of the previous articles. All tests inspect the state of the Fake database after calling the Post method. The exact interactions between Post and db aren't specified. Instead, these tests rely on setting up the initial state, exercising the SUT, and verifying the final state. These are all state-based tests that avoid over-specifying the interactions.

Specifically, none of these tests use Mocks and Stubs. In fact, at this incarnation of the test code, I was able to entirely remove the reference to Moq.

Summary #

The premise of Refactoring is that in order to be able to refactor, the "precondition is [...] solid tests". In reality, many development organisations have the opposite experience. When programmers attempt to make changes to how their code is organised, tests break. In xUnit Test Patterns this problem is called Fragile Tests, and the cause is often Overspecified Software. This means that tests are tightly coupled to implementation details of the SUT.

It's easy to inadvertently fall into this trap when you use Mocks and Stubs, even when you follow the rule of using Mocks for Commands and Stubs for Queries. Refactoring tests towards state-based testing with Fake objects, instead of interaction-based testing, could make test suites more robust to changes.

It's intriguing, though, that state-based testing is simpler in functional programming. In Haskell, you can simply write your tests in the State monad and compare the expected outcome to the actual outcome. Since state in Haskell is immutable, it's trivial to compare the expected with the actual state.

As soon as you introduce mutable state, structural equality is no longer safe, and instead you have to rely on other inspection mechanisms, such as the IsDirty flag seen in this, and the previous, article. This makes the tests slightly more brittle, because it tends to pull towards interaction-based testing.

While you can implement the State monad in both F# and C#, it's probably more pragmatic to express state-based tests using mutable state and the occasional IsDirty flag. As always, there's no panacea.

While this article concludes the series on moving towards state-based testing, I think that an appendix on Test Spies is in order.

Next: A pure Test Spy.

Comments

• ← Previous
• Archive
• Next →