In a previous post I described how AutoFixture's Do method can let you invoke commands on your SUT with Dummy values for the parameters you don't care about.

The Get method is the equivalent method you can use when the member you are invoking returns a value. In other words: if you want to call a method on your SUT to get a value, but you don't want the hassle of coming up with values you don't care about, you can let the Get method supply those values for you.

In today's example I will demonstrate a unit test that verifies the behavior of a custom ASP.NET MVC ModelBinder. If you don't know anything about ASP.NET MVC it doesn't really matter. The point is that a ModelBinder must implement the IModelBinder interface that defines a single method:

In many cases we don't care about one or the other of these parameters, but we still need to supply them when unit testing.

The example is a bit more complex than some of my other sample code, but once in a while I like to provide you with slightly more realistic AutoFixture examples. Still, it's only 10 lines of code, but it looks like a lot more because I have wrapped several of the lines so that the code is still readable on small screens.

The first part simply creates the Fixture object and customizes it to disable AutoProperties for all ControllerContext instances (otherwise we would have to set up a lot of Test Doubles for such properties as HttpContext, RequestContext etc.).

The next part of the test sets up a ModelBindingContext instance that will be used to exercise the SUT. In this test case, the bindingContext parameter of the BindModel method is important, so I explicitly set that up. On the other hand, I don't care about the controllerContext parameter in this test case, so I ask the Get method to take care of that for me:

The Get method creates a Dummy value for the ControllerContext, whereas I can still use the outer variable bindingContext to call the BindModel method. The return value of the BindModel method is returned to the result variable by the Get method.

Like the Do methods, the Get methods are generic methods. The one invoked in this example has this signature:

There are also overloads that works with the versions of the Func delegate that takes two, three and four parameters.

As the Do methods, the Get methods are convenience methods that let you concentrate on writing the test code you care about while it takes care of all the rest. You could have written a slightly more complex version that didn't use Get but instead used the CreateAnonymous method to create an Anonymous Variable for the ControllerContext, but this way is slightly more succinct.

In unit testing an important step is to exercise the SUT. The member you want to invoke is often a method that takes one or more parameters, but in some test cases you don't care about the values of those parameters – you just want to invoke the method.

You can always make up one or more Dummy parameters and pass them to the method in question, but you could also use one of AutoFixture's Do convenience methods. There are several overloads that all take delegates that specify the action in question while providing you with Dummies of any parameters you don't care about.

A good example is WPF's ICommand interface. The most prevalent method is the Execute method that takes a single parameter parameter:

Most of the time we don't really care about the parameter because we only care that the command was invoked. However, we still need to supply a value for the parameter when we unit test our ICommand implementations. Obviously, we could just pass in a new System.Object every time, but why not let AutoFixture take care of that for us?

(You may think that new'ing up a System.Object is something you can easily do yourself, but imagine other APIs that require much more complex input parameters, and you should begin to see the potential.)

Here's a state-based unit test that verifies that the Message property of the MyViewModel class has the correct value after the FooCommand has been invoked:

Notice how the Do method takes an Action<object> to specify the method to invoke. AutoFixture automatically supplies an instance for the parameter parameter using the same engine to create Anonymous variables that it uses for everything else.

The Do method in question is really a generic method with this signature:

There are also overloads that take two, three or four input parameters, corresponding to the available Action types available in the BCL.

These methods are simply convenience methods that allow you to express your test code more succinctly than you would otherwise have been able to do.

A few months ago I described how you can use AutoFixture's With method to assign property values as part of building up an anonymous variable. In that scenario, the With method is used to explicitly assign a particular, pre-defined value to a property.

There's another overload of the With method that doesn't take an explicit value, but rather uses AutoFixture to create an anonymous value for the property. So what's the deal with that if AutoFixture's default behavior is to assign anonymous values to all writable properties?

In short it's an opt-in mechanism that only makes sense if you decide to opt out of the AutoProperties features.

As always, let's look at an example. This time, I've decided to show you a slightly more realistic example so that you can get an idea of how AutoFixture can be used to aid in unit testing. This also means that the example is going to be a little more complex than usual, but it's still simple.

Imagine that we wish to test that an ASP.NET MVC Controller Action returns the correct result. More specifically, we wish to test that the Profile method on MyController returns a ViewResult with the correct Model (the current user's name, to make things interesing).

Here's the entire test. It may look more complicated than it is – it's really only 10 lines of code, but I had to break them up to prevent unpleasant wrapping if your screen is narrow.

[TestMethod]

public void ProfileWillReturnResultWithCorrectUserName()

{

    // Fixture setup

    var fixture = new Fixture();

    fixture.Customize<ControllerContext>(ob => ob

        .OmitAutoProperties()

        .With(cc => cc.HttpContext));

    var expectedUserName = 

        fixture.CreateAnonymous("UserName");

    var httpCtxStub = new Mock<HttpContextBase>();

    httpCtxStub.SetupGet(x => x.User).Returns(

        new GenericPrincipal(

            new GenericIdentity(expectedUserName), null));

    fixture.Register(httpCtxStub.Object);

    var sut = fixture.Build<MyController>()

        .OmitAutoProperties()

        .With(c => c.ControllerContext)

        .CreateAnonymous();

    // Exercise system

    ViewResult result = sut.Profile();

    // Verify outcome

    var actual = result.ViewData.Model;

    Assert.AreEqual(expectedUserName, actual, "User");

    // Teardown

}

Apart from AutoFixture, I'm also making use of Moq to stub out HttpContextBase.

You can see the Anonymous With method in two different places: in the call to Customize and when the SUT is being built. In both cases you can see that the call to With follows a call to OmitAutoProperties. In other words: we are telling AutoFixture that we don't want any of the writable properties to be assigned a value except the one we identify.

Let me highlight some parts of the test.

fixture.Customize<ControllerContext>(ob => ob

    .OmitAutoProperties()

    .With(cc => cc.HttpContext));

This line of code instructs AutoFixture to always create a ControllerContext in a particular way: I don't want to use AutoProperties here, because ControllerContext has a lot of writable properties of abstract types, and that would require me to set up a lot of Test Doubles if I had to assign values to all of those. It's much easier to simply opt out of this mechanism. However, I would like to have the HttpContext property assigned, but I don't care about the value in this statement, so the With method simply states that AutoFixture should assign a value according to whatever rule it has for creating instances of HttpContextBase.

I can now set up a Stub that populates the User property of HttpContextBase:

var httpCtxStub = new Mock<HttpContextBase>();

httpCtxStub.SetupGet(x => x.User).Returns(

    new GenericPrincipal(

        new GenericIdentity(expectedUserName), null));

fixture.Register(httpCtxStub.Object);

This is registered with the fixture instance which closes the loop to the previous customization.

I can now create an instance of my SUT. Once more, I don't want to have to set up a lot of irrelevant properties on MyController, so I opt out of AutoProperties and then explicitly opt in on the ControllerContext. This will cause AutoFixture to automatically populate the ControllerContext with the HttpContext Stub:

var sut = fixture.Build<MyController>()

    .OmitAutoProperties()

    .With(c => c.ControllerContext)

    .CreateAnonymous();

For completeness' sake, here's the MyController class that I am testing:

public class MyController : Controller

{

    public ViewResult Profile()

    {

        object userName = this.User.Identity.Name;

        return this.View(userName);

    }

}

This test may seem complex, but it really accomplishes a lot in only 10 lines of code, considering that ASP.NET MVC Controllers with a working HttpContext are not particularly trivial to set up.

In summary, this With method overload lets you opt in on one or more explicitly identified properties when you have otherwise decided to omit AutoProperties. As all the other Builder methods, this method can also be chained.

If you are doing Rich UI, INotifyPropertyChanged is a pretty important interface. This is as true for WPF as it was for Windows Forms. Consisting solely of an event, it's not any harder to unit test than other events.

You can certainly write each test manually like the following.

[TestMethod]

public void ChangingMyPropertyWillRaiseNotifyEvent_Classic()

{

    // Fixture setup

    bool eventWasRaised = false;

    var sut = new MyClass();

    sut.PropertyChanged += (sender, e) =>

        {

            if (e.PropertyName == "MyProperty")

            {

                eventWasRaised = true;

            }

        };

    // Exercise system

    sut.MyProperty = "Some new value";

    // Verify outcome

    Assert.IsTrue(eventWasRaised, "Event was raised");

    // Teardown

}

Even for a one-off test, this one has a few problems. From an xUnit Test Patterns point of view, there's the issue that the test contains conditional logic, but that aside, the main problem is that if you have a lot of properties, writing all these very similar tests become old hat very soon.

To make testing INotifyPropertyChanged events easier, I created a simple fluent interface that allows me to write the same test like this:

[TestMethod]

public void ChangingMyPropertyWillRaiseNotifyEvent_Fluent()

{

    // Fixture setup

    var sut = new MyClass();

    // Exercise system and verify outcome

    sut.ShouldNotifyOn(s => s.MyProperty)

        .When(s => s.MyProperty = "Some new value");

    // Teardown

}

You simply state for which property you want to verify the event when a certain operation is invoked. This is certainly more concise and intention-revealing than the previous test.

If you have interdependent properties, you can specify than an event was raised when another property was modified.

[TestMethod]

public void ChangingMyPropertyWillRaiseNotifyForDerived()

{

    // Fixture setup

    var sut = new MyClass();

    // Exercise system and verify outcome

    sut.ShouldNotifyOn(s => s.MyDerivedProperty)

        .When(s => s.MyProperty = "Some new value");

    // Teardown

}

The When method takes any Action<T>, so you can also invoke methods, use Closures and what not.

There's also a ShouldNotNotifyOn method to verify that an event was not raised when a particular operation was invoked.

This fluent interface is implemented with an extension method on INotifyPropertyChanged, combined with a custom class that performs the verification. Here are the extension methods:

public static class NotifyPropertyChanged

{

    public static NotifyExpectation<T>

        ShouldNotifyOn<T, TProperty>(this T owner,

        Expression<Func<T, TProperty>> propertyPicker) 

        where T : INotifyPropertyChanged

    {

        return NotifyPropertyChanged.CreateExpectation(owner,

            propertyPicker, true);

    }

    public static NotifyExpectation<T> 

        ShouldNotNotifyOn<T, TProperty>(this T owner,

        Expression<Func<T, TProperty>> propertyPicker)

        where T : INotifyPropertyChanged

    {

        return NotifyPropertyChanged.CreateExpectation(owner,

            propertyPicker, false);

    }

    private static NotifyExpectation<T>

        CreateExpectation<T, TProperty>(T owner,

        Expression<Func<T, TProperty>> pickProperty,

        bool eventExpected) where T : INotifyPropertyChanged

    {

        string propertyName =

            ((MemberExpression)pickProperty.Body).Member.Name;

        return new NotifyExpectation<T>(owner,

            propertyName, eventExpected);

    }

}

And here's the NotifyExpectation class returned by both extension methods:

public class NotifyExpectation<T>

    where T : INotifyPropertyChanged

{

    private readonly T owner;

    private readonly string propertyName;

    private readonly bool eventExpected;

    public NotifyExpectation(T owner,

        string propertyName, bool eventExpected)

    {

        this.owner = owner;

        this.propertyName = propertyName;

        this.eventExpected = eventExpected;

    }

    public void When(Action<T> action)

    {

        bool eventWasRaised = false;

        this.owner.PropertyChanged += (sender, e) =>

        {

            if (e.PropertyName == this.propertyName)

            {

                eventWasRaised = true;

            }

        };

        action(this.owner);

        Assert.AreEqual<bool>(this.eventExpected,

            eventWasRaised,

            "PropertyChanged on {0}", this.propertyName);

    }

}

You can replace the Assertion with one that matches your test framework of choice (this one was written for MSTest).

Since AutoFixture is a Test Data Builder, one of its most important tasks is to build up graphs of fully populated, yet semantically correct, strongly typed objects. As such, its default behavior is to assign a value to every writable property in the object graph.

While this is sometimes the desired behavior, at other times it is not.

This is particularly the case when you want to test that a newly created object has a property of a particular value. When you want to test the default value of a writable property, AutoFixture's AutoProperty feature is very much in the way.

Let's consider as an example a piece of software that deals with vehicle registration. By default, a vehicle should have four wheels, since this is the most common occurrence. Although I always practice TDD, I'll start by showing you the Vehicle class to illustrate what I mean.

public class Vehicle

{

    public Vehicle()

    {

        this.Wheels = 4;

    }

    public int Wheels { get; set; }

}

Here's a test that ensures that the default number of wheels is 4 – or does it?

In fact the assertion fails because the actual value is 1, not 4.

[TestMethod]

public void AnonymousVehicleHasWheelsAssignedByFixture()

{

    // Fixture setup

    var fixture = new Fixture();

    var sut = fixture.CreateAnonymous<Vehicle>();

    // Exercise system

    var result = sut.Wheels;

    // Verify outcome

    Assert.AreEqual<int>(4, result, "Wheels");

    // Teardown

}

Why does the test fail when the value of Wheels is set to 4 in the constructor? It fails because AutoFixture is designed to create populated test data, so it assigns a value to every writable property. Wheels is a writable property, so AutoFixture assigns an integer value to it using its default algorithm for creating anonymous numbers. Since no other numbers are being created during this test, the number assigned to Wheels is 1. This is AutoFixture's AutoProperties feature in effect.

When you want to test constructor logic, or otherwise wish to disable the AutoProperties feature, you can use a customized Builder with the OmitAutoProperties method:

[TestMethod]

public void VehicleWithoutAutoPropertiesWillHaveFourWheels()

{

    // Fixture setup

    var fixture = new Fixture();

    var sut = fixture.Build<Vehicle>()

        .OmitAutoProperties()

        .CreateAnonymous();

    // Exercise system

    var result = sut.Wheels;

    // Verify outcome

    Assert.AreEqual<int>(4, result, "Wheels");

    // Teardown

}

The OmitAutoProperties method instructs AutoFixture to skip assigning automatic Anonymous Values to all writable properties in the object graph. Any properties specifically assigned by the With method will still be assigned.

The test using OmitAutoProperties succeeds.

Some time ago, my good friend Martin Gildenpfennig from Ative made me aware that AutoFixture (among other things) is an generic implementation of the Test Data Builder pattern, and indeed it is.

In the original Gang of Four definition of a Design Pattern, several people must independently have arrived at the same general solution to a given problem before we can call a coding idiom a true Pattern. In this spirit, the Test Data Builder pattern is on the verge of becoming a true Design Pattern, since I came up with AutoFixture without having heard about Test Data Builder :)

The problem is the same: How do we create semantically correct test objects in a manner that is flexible and yet hides away irrelevant complexity?

Like others before me, I tried the Object Mother (anti?)pattern and found it lacking. To me the answer was AutoFixture, a library that heuristically builds object graphs using Reflection and built-in rules for specific types.

Although my original approach was different, you can certainly use AutoFixture as a generic Test Data Builder.

To demonstrate this, let's work with this (oversimplified) Order object model:

Assuming that we have an instance of the Fixture class (called fixture), we can create a new instance of the Order class with a ShippingAddress in Denmark:

var order = fixture.Build<Order>()

    .With(o => o.ShippingAddress, 

        fixture.Build<Address>()

        .With(a => a.Country, "Denmark")

        .CreateAnonymous())

    .CreateAnonymous();

While this works and follows the Test Data Builder pattern, I find this more concise and readable:

var order = fixture.CreateAnonymous<Order>();

order.ShippingAddress.Country = "Denmark";

The result is the same.

We can also add anonymous order lines to the order using this fluent interface:

var order = fixture.Build<Order>()

    .Do(o => fixture.AddManyTo(o.OrderLines))

    .CreateAnonymous();

but again, I find it easier to simply let AutoFixture create a fully anonymous Order instance, and then afterwards modify the relevant parts:

var order = fixture.CreateAnonymous<Order>();

fixture.AddManyTo(order.OrderLines);

Whether you prefer one style over the other, AutoFixture supports them both.

When I talk with people about TDD and unit testing, the discussion often moves into the area of Testability – that is, the software's susceptibility to unit testing. A couple of years back, Roy even discussed the seemingly opposable forces of Object-Oriented Design and Testability.

Lately, it has been occurring to me that there really isn't any conflict. Encapsulation is important because it manifests expert knowledge so that other developers can effectively leverage that knowledge, and it does so in a way that minimizes misuse.

However, too much encapsulation goes against the Open/Closed Principle (that states that objects should be open for extension, but closed for modification). From a Testability perspective, the Open/Closed Principle pulls object-oriented design in the desired direction. Equivalently, done correctly, making your API Testable is simply opening it up for extensibility.

As an example, consider a simple WPF ViewModel class called MainWindowViewModel. This class has an ICommand property that, when invoked, should show a message box. Showing a message box is good example of breaking testability, because if the SUT were to show a message box, it would be very hard to automatically verify and we wouldn't have fully automated tests.

For this reason, we need to introduce an abstraction that basically models an action with a string as input. Although we could define an interface for that, an Action<string> fits the bill perfectly.

To enable that feature, I decide to use Constructor Injection to inject that abstraction into the MainWindowViewModel class:

public MainWindowViewModel(Action<string> notify)

{

    this.ButtonCommand = new RelayCommand(p => 

    { notify("Button was clicked!"); });

}

When I recently did that at a public talk I gave, one member of the audience initially reacted by assuming that I was now introducing test-specific code into my SUT, but that's not the case.

What I'm really doing here is opening the MainWindowViewModel class for extensibility. It can still be used with message boxes:

var vm = new MainWindowViewModel(s => MessageBox.Show(s));

but now we also have the option of notifying by sending off an email; writing to a database; or whatever else we can think of.

It just so happens that one of the things we can do instead of showing a message box, is unit testing by passing in a Test Double.

// Fixture setup

var mockNotify = 

    MockRepository.GenerateMock<Action<string>>();

mockNotify.Expect(a => a("Button was clicked!"));

var sut = new MainWindowViewModel(mockNotify);

// Exercise system

sut.ButtonCommand.Execute(new object());

// Verify outcome

mockNotify.VerifyAllExpectations();

// Teardown

Once again, TDD has lead to better design. In this case it prompted me to open the class for extensibility. There really isn't a need for Testability as a specific concept; the Open/Closed Principle should be enough to drive us in the right direction.

Pragmatically, that's not the case, so we use TDD to drive us towards the Open/Closed Principle, but I think it's important to note that we are not only doing this to enable testing: We are creating a better and more flexible API at the same time.

Dear reader, I hope you are still with me!

After eight posts of AutoFixture feature walkthroughs, I can't blame you for wondering why this tool might even be relevant to you. In this post, we'll finally begin to look at how AutoFixture can help you towards Zero-Friction TDD!

In an earlier post, I described how the Fixture Object pattern can help you greatly reduce the amount of test code that you have to write. Since AutoFixture was designed to act as a general-purpose Fixture Object, it can help you reduce the amount of test code even further, letting you focus on specifying the behavior of your SUT.

In that former post, the original example was this complex test that I will repeat in it's entirety for your benefit (or horror):

[TestMethod]

public void NumberSumIsCorrect_Naïve()

{

    // Fixture setup

    Thing thing1 = new Thing()

    {

        Number = 3,

        Text = "Anonymous text 1"

    };

    Thing thing2 = new Thing()

    {

        Number = 6,

        Text = "Anonymous text 2"

    };

    Thing thing3 = new Thing()

    {

        Number = 1,

        Text = "Anonymous text 3"

    };

    int expectedSum = new[] { thing1, thing2, thing3 }.

        Select(t => t.Number).Sum();

    IMyInterface fake = new FakeMyInterface();

    fake.AddThing(thing1);

    fake.AddThing(thing2);

    fake.AddThing(thing3);

    MyClass sut = new MyClass(fake);

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

This test consists of 18 lines of code.

Using the Fixture Object pattern, I was able to cut that down to 7 lines of code, which is a 61% improvement (however, the downside was an additional 19 lines of (reusable) code for MyClassFixture, so the benefit can only be reaped when you have multiple tests leveraged by the same Fixture Object. This was all covered in the former post, to which I will refer you).

With AutoFixture, we can do much better. Here's a one-off rewrite of the unit test using AutoFixture:

[TestMethod]

public void NumberSumIsCorrect_AutoFixture()

{

    // Fixture setup

    Fixture fixture = new Fixture();

    IMyInterface fake = new FakeMyInterface();

    fixture.Register<IMyInterface>(() => fake);

    var things = fixture.CreateMany<Thing>().ToList();

    things.ForEach(t => fake.AddThing(t));

    int expectedSum = things.Select(t => t.Number).Sum();

    MyClass sut = fixture.CreateAnonymous<MyClass>();

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

In this test, I map the concrete fake instance to the IMyInterface type in the fixture object, and then use its ability to create many anonymous instances with one method call. Before exercising the SUT, I also use the fixture instance as a SUT Factory.

Apart from AutoFixture (and FakeMyInterface, which is invariant for all variations, and thus kept out of the comparison), this test stands alone, but still manages to reduce the number of code lines to 10 lines – a 44% improvement! In my book, that's already a significant gain in productivity and maintainability, but we can do better!

If we need to test MyClass repeatedly in similar ways, we can move the common code to a Fixture Object based on AutoFixture, and the test can be refactored to this:

[TestMethod]

public void NumberSumIsCorrect_DerivedFixture()

{

    // Fixture setup

    MyClassFixture fixture = new MyClassFixture();

    fixture.AddManyTo(fixture.Things);

    int expectedSum = 

        fixture.Things.Select(t => t.Number).Sum();

    MyClass sut = fixture.CreateAnonymous<MyClass>();

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

Now we are back at 7 lines of code, which is on par with the original Fixture Object-based test, but now MyClassFixture is reduced to 8 lines of code:

internal class MyClassFixture : Fixture

{

    internal MyClassFixture()

    {

        this.Things = new List<Thing>();

        this.Register<IMyInterface>(() =>

            {

                var fake = new FakeMyInterface();

                this.Things.ToList().ForEach(t =>

                    fake.AddThing(t));

                return fake;

            });

    }

    internal IList<Thing> Things { get; private set; }

}

Notice how I've moved the IMyInterface-to-FakeMyInterface mapping to MyClassFixture. Whenever it's asked to create a new instance of IMyInterface, MyClassFixture makes sure to add all the Thing instances to the fake before returning it.

Compared to the former Fixture Object of 19 lines, that's another 58% improvement. Considering some of the APIs I encounter in my daily work, the above example is even rather simple. The more complex and demanding your SUT's API is, the greater the gain from using AutoFixture will be, since it's going to figure out much of the routine stuff for you.

With this post, I hope I have given you a taste of the power that AutoFixture provides. It allows you to focus on specifying the behavior of your SUT, while taking care of all the infrastructure tedium that tends to get in the way.

At May 12 2009 I'll be speaking at 7N's IT Konference 2009 (in Danish, so that's no spelling error). You can read the program here.

The topic of my talk will be TDD patterns and terminology, so I'll discuss Fixtures, Stubs, Mocks and the like. As always, xUnit Test Patterns will form the basis of my vocabulary.

Of the other speakers, I'm particularly looking forward to hear my good friend Martin Gildenpfennig from Ative speak!

It gives me great pleasure to announce my latest project: AutoFixture!

What is AutoFixture?

In essence, AutoFixture is a library that creates Anonymous Variables for you when you write unit tests. The intention is that it should enhance your productivity when you do Test-Driven Development – as it has done mine.

Instead of using mental resources on creating Anonymous Variables, AutoFixture can do it for you. By default, it uses Constrained Non-Determinism, but you can configure it to behave differently if you wish.

Here’s a very basic example:

[TestMethod]

public void IntroductoryTest()

{

    // Fixture setup

    Fixture fixture = new Fixture();

    int expectedNumber = fixture.CreateAnonymous<int>();

    MyClass sut = fixture.CreateAnonymous<MyClass>();

    // Exercise system

    int result = sut.Echo(expectedNumber);

    // Verify outcome

    Assert.AreEqual<int>(expectedNumber, result, "Echo");

    // Teardown

}

The Fixture class is your main entry point to AutoFixture. You can use it as is, customize it, or derive from it as you please; it makes a great base class for a Fixture Object.

The expectedNumber variable may be an Explicit Expectation, but its value is Anonymous, so instead of coming up with a number ourselves, we can let the CreateAnonymous<T> method do it for us.

This method can create instances of most CLR types as long as they have a public constructor (it uses Reflection), but for many primitive types (like Int32), it has specific, customizable algorithms for creating values using Constrained Non-Determinism.

When creating the SUT, we can also use Fixture as an excellent SUT Factory. Since it will do whatever it can to create an instance of the type you ask for, it is pretty robust if you decide to refactor the SUT’s constructor.

The above example only hints at what AutoFixture can do. Since the example is very simple, it may be hard to immediately understand its value, so in future posts I will expand on specific AutoFixture features and principles.

Getting started with AutoFixture is as simple as downloading it from CodePlex and referencing the assembly in Visual Studio.

(A Zero-Friction TDD post)

For a simple API, setting up the Fixture may be as simple as creating a new instance of the SUT, and possibly any Expected or Anonymous Variables. On the other hand, for a complex API, setting up the fixture may require quite a bit of (potentially repetitive) code.

Since the DRY principle also applies to test code, it quickly becomes necessary to create test-specific helper methods and other SUT API Encapsulation, and I've found that instead of creating a more or less unplanned set of disconnected helper methods, it's much cleaner (and, not to mention, much more object-oriented) to create a single object that represents the Fixture, and attach the helper methods to this object.

Let's look at an example.

Here's a unit test with a complex Fixture Setup:

[TestMethod]

public void NumberSumIsCorrect_Naïve()

{

    // Fixture setup

    Thing thing1 = new Thing()

    {

        Number = 3,

        Text = "Anonymous text 1"

    };

    Thing thing2 = new Thing()

    {

        Number = 6,

        Text = "Anonymous text 2"

    };

    Thing thing3 = new Thing()

    {

        Number = 1,

        Text = "Anonymous text 3"

    };

    int expectedSum = new[] { thing1, thing2, thing3 }.

        Select(t => t.Number).Sum();

    IMyInterface fake = new FakeMyInterface();

    fake.AddThing(thing1);

    fake.AddThing(thing2);

    fake.AddThing(thing3);

    MyClass sut = new MyClass(fake);

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

If this was a truly one-off test and you know with certainty that there's going to be no other tests just remotely similar to this one, just hard-coding the entire Fixture Setup inline is in order, but as soon as the need for similar tests arises, this approach leads to repetitive code, and hence unmaintainable tests.

The more repetitive code that can be delegated to helper methods the better. A common refactoring of the previous test might then look something like this:

[TestMethod]

public void NumberSumIsCorrect_Helpers()

{

    // Fixture setup

    Thing thing1 = MyClassTest.CreateAnonymousThing();

    Thing thing2 = MyClassTest.CreateAnonymousThing();

    Thing thing3 = MyClassTest.CreateAnonymousThing();

    Thing[] things = new[] { thing1, thing2, thing3 };

    int expectedSum = things.Select(t => t.Number).Sum();

    IMyInterface fake = new FakeMyInterface();

    MyClassTest.AddThingsToMyInterface(fake, things);

    MyClass sut = new MyClass(fake);

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

While this is better, the helper methods are static methods, so it's necessary to pass too much state around. The array of Things and the fake is both needed in the test itself, as well as in the AddThingsToMyInterface helper method.

By moving and refactoring the helper methods to a new class that represents the Fixture, the test code becomes both more reusable and more readable.

[TestMethod]

public void NumberSumIsCorrect_FixtureObject()

{

    // Fixture setup

    MyClassFixture fixture = new MyClassFixture();

    fixture.AddAnonymousThings();

    int expectedSum = 

        fixture.Things.Select(t => t.Number).Sum();

    MyClass sut = fixture.CreateSut();

    // Exercise system

    int result = sut.CalculateSumOfThings();

    // Verify outcome

    Assert.AreEqual<int>(expectedSum, result,

        "Sum of things");

    // Teardown

}

The MyClassFixture instance now holds the state of the Fixture, so there's much less need to pass around as much data as before. The set of Things is now contained within the Fixture object itself, and the fake has totally disappeared from the test; it's still present, but now encapsulated within MyClassFixture.

internal class MyClassFixture

{

    public MyClassFixture()

    {

        this.Fake = new FakeMyInterface();

        this.Things = new List<Thing>();

    }

    internal FakeMyInterface Fake { get; private set; }

    internal IList<Thing> Things { get; private set; }

    internal void AddAnonymousThings()

    {

        int many = 3;

        for (int i = 0; i < many; i++)

        {

            Thing t = this.CreateAnonymousThing();

            this.Things.Add(t);

            this.Fake.AddThing(t);

        }

    }

    internal MyClass CreateSut()

    {

        return new MyClass(this.Fake);

    }

    private Thing CreateAnonymousThing()

    {

        Thing t = new Thing();

        t.Number = this.Things.Count + 1;

        t.Text = Guid.NewGuid().ToString();

        return t;

    }

}

The CreateAnonymousThing method uses Constrained Non-Determinism to create unique Thing instances. The AddAnonymousThings method uses 3 as an equivalence of many, and the CreateSut method acts as a SUT Factory.

This is both more reusable and more expressive than a collection of disjointed static helper methods.

Whenever I begin to feel that setting up a Test Fixture is becoming too cumbersome, Fixture Object is the first pattern I consider.

In the last couple of years, there’s been a lot of debate in the community on the philosophy behind TDD and where to put the emphasis – even to the point of debating whether the acronym stands for Test-Driven Development or Test-Driven Design.

Other people don’t like the emphasis on tests, since that makes TDD sound like a Testing discipline, and not a Development discipline. Instead, they prefer terms like Example-Driven Design/Development (EDD) or even Design by Example (DbE).

This view seems to me to be particularly prevalent in Microsoft, where there’s a rather sharp distinction between developers and testers (job titles to the contrary) – I guess that’s one of the reasons why xUnit.net (a project initiated by Microsoft employees) uses the attribute Fact instead of Test or TestMethod.

For people used to SCRUM or other agile methodologies, this distinction is more blurred, and they also seem to accept the T in TDD more willingly.

However, the adherents of EDD claim that the mere presence of the word test make some developers block any further input and stop listening. They may be right in that.

They also claim that the tests in TDD/EDD are nothing more than accidental artifacts of the development process, and hence argue that we shouldn’t call them tests at all. However, if that’s true, this little story related by Ayende must be an example of EDD in its purest form :)

To me, the tests are also important. Since 2003 I’ve been practicing TDD, and while I love how it helps me arrive at better design, I also savor the safety net that my suite of tests gives me. The tests that I write during TDD define the behavior of the software. In many cases, I’d even claim that such a regression test suite is more valuable than a Quality Assurance (QA) regression test suite – after all, a QA suite may catch some edge cases, but they don’t focus on the intended behavior of the system, but often more on how to break it - but I digress…

My recent posts on Executable Specification and Constrained Non-Determinism help explain my current stance in this debate: In my opinion, EDD fails to establish a relationship by not providing Derived Values. After all, what does a test like the following specify?

[TestMethod]

public void InvertWillReverseText_Naïve()

{

    // Fixture setup

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert("ploeh");

    // Verify outcome

    Assert.AreEqual<string>("heolp", result, "Invert");

    // Teardown

}

How would you implement the Invert method? Here’s one possible implementation:

return "heolp";

Obviously, you could now write a new test that gives a second example of input and outcome and force me to write a more sophisticated algorithm. However, with only two examples, I might still be tempted to write a switch statement with some hard-coded return values until you’ve written so many ‘examples’ that you’ve coerced me into writing the more general (and correct) algorithm.

Such an approach I find inefficient.

Instead, by using Constrained Non-Determinism to force myself to define Derived Values, each test fully specifies the desired behavior. It doesn’t provide examples. It provides the specification, and instead of having to write several similar examples to coerce a general algorithm to emerge, I can usually nail it in a single test.

This approach could be styled Specification-Driven Development, and that’s how I’ve been writing code for the last year or so.

This may turn out to be the most controversial of my Zero-Friction TDD posts so far, as it supposedly goes against conventional wisdom. However, I have found this approach to be really powerful since I began using it about a year ago.

In my previous post, I explained how Derived Values help ensure that tests act as Executable Specification. In short, a test should clearly specify the relationship between input and outcome, as this test does:

[TestMethod]

public void InvertWillReverseText()

{

    // Fixture setup

    string anonymousText = "ploeh";

    string expectedResult =

        new string(anonymousText.Reverse().ToArray());

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert(anonymousText);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result,

        "DoWork");

    // Teardown

}

However, it is very tempting to just hardcode the expected value. Consistently using Derived Values to establish the relationship between input and outcome requires discipline.

To help myself enforce this discipline, I use well-defined, but essentially random, input, because when the input is random, I don't know the value at design time, and hence, it is impossible for me to accidentally hard-code any assertions.

[TestMethod]

public void InvertWillReverseText_Cnd()

{

    // Fixture setup

    string anonymousText = Guid.NewGuid().ToString();

    string expectedResult =

        new string(anonymousText.Reverse().ToArray());

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert(anonymousText);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result,

        "DoWork");

    // Teardown

}

For strings, I prefer Guids, as the above example demonstrates. For numbers, I often just use the sequence of natural numbers (i.e. 1, 2, 3, 4, 5...). For booleans, I often use an alternating sequence (i.e. true, false, true, false...).

While this technique causes input to become non-deterministic, I always pick the non-deterministic value-generating algorithm in such a way that it creates 'nice' values; I call this principle Constrained Non-Determinism. Values are carefully generated to stay far away from any boundary conditions that may cause the SUT to behave differently in each test run.

Conventional unit testing wisdom dictates that unit tests should be deterministic, so how can I possibly endorse this technique?

To understand this, it's important to know why the rule about deterministic unit tests exist. It exists because we want to be certain that each time we execute a test suite, we verify the exact same behavior as we did the last time (given that no tests changed). Since we also use test suites as regression tests, it's important that we can be confident that each and every test run verifies the exact same specification.

Constrained Non-Determinism doesn't invalidate that goal, because the algorithm that generates the values must be carefully picked to always create values that stay within the input's Equivalence Class.

In a surprisingly large set of APIs, strings, for example, are treated as opaque values that don't influence behavior in themselves. Many enterprise applications mostly store and read data from persistent data stores, and the value of a string in itself is often inconsequential from the point of view of the code's execution path. Data stores may have constraints on the length of strings, so Constrained Non-Determinism dictates that you should pick the generating algorithm so that the string length always stays within (or exceeds, if that's what you want to test) the constraint. Guid.ToString always returns a string with the length of 36, which is fine for a large number of scenarios.

Note that Constrained Non-Determinism is only relevant for Anonymous Variables. For input where the value holds a particular meaning in the context of the SUT, you will still need to hand-pick values as always. E.g. if the input is expected to be an XML string conforming to a particular schema, a Guid string makes no sense.

A secondary benefit of Constrained Non-Determinism is that you don't have to pause to come up with values for Anonymous Variables when you are writing the test.

While this advice may be controversial, I can only recommend it - I've been using this technique for about a year now, and have only become more fond of it as I have gained more experience with it.

In this Zero-Friction TDD post, I’d like to take a detour around the concept of tests as Executable Specification.

An important aspect of test maintainability is readability. Tests should act both as Executable Specification as well as documentation, which puts a lot of responsibility on the test.

One facet of test readability is to make the relationship between the Fixture, the SUT and the verification as easy to understand as possible. In other words, it should be clear to the Test Reader what is being asserted, and why.

Consider a test like this one:

[TestMethod]

public void InvertWillReverseText_Naïve()

{

    // Fixture setup

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert("ploeh");

    // Verify outcome

    Assert.AreEqual<string>("heolp", result, "DoWork");

    // Teardown

}

Since this test is so simple, I expect that you can easily figure out that it implies that the Invert method should simply reverse its input argument, but one of the reasons this seems to be evident is because of the proximity of the two strings, as well as the test’s name.

In a test of a more complex API, this may not be quite as evident.

[TestMethod]

public void DoItWillReturnCorrectResult_Naïve()

{

    // Fixture setup

    MyClass sut = new MyClass();

    // Exercise system

    int result = sut.DoIt("ploeh");

    // Verify outcome

    Assert.AreEqual<int>(42, result, "DoIt");

    // Teardown

}

In this test, there's no apparent relationship between the input (ploeh) and the output (42). Whatever the algorithm is behind the DoIt method, it's completely opaque to the Test Reader, and the test fails in its role as specification and documentation.

Returning to the first example, it would be better if the relationship between input and output was explicitly described:

[TestMethod]

public void InvertWillReverseText()

{

    // Fixture setup

    string anonymousText = "ploeh";

    string expectedResult =

        new string(anonymousText.Reverse().ToArray());

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert(anonymousText);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result,

        "DoWork");

    // Teardown

}

In this case, the input and expected outcome are clearly related, and we call the expectedResult variable a Derived Value, since we explicitly derive the expected result from the input.

Note that I’m not asking you to re-implement the whole algorithm in the test, but only to establish a relationship. One of the main rules of thumb of unit testing is that a test should never contain conditional branches, so there must be at least one test case per path though the SUT.

In the example, the Invert method actually looks like this:

public string Invert(string message)

{

    double d;

    if (double.TryParse(message, out d))

    {

        return (1d / d).ToString();

    }

    return new string(message.Reverse().ToArray());

}

Note that the above test only reproduces that part of the algorithm that corresponds to the Equivalence Class defined by the input, whereas the branch that is triggered by a number string can be tested by another test case that doesn’t specify string reversion.

[TestMethod]

public void InvertWillInvertNumber()

{

    // Fixture setup

    double anonymousNumber = 10;

    string numberText = anonymousNumber.ToString();

    string expectedResult = 

        (1d / anonymousNumber).ToString();

    MyClass sut = new MyClass();

    // Exercise system

    string result = sut.Invert(numberText);

    // Verify outcome

    Assert.AreEqual<string>(expectedResult, result,

        "DoWork");

    // Teardown

}

In this way, we can break down the test cases to individual Executable Specifications that define the expected behavior for each Equivalence Class.

While such tests more clearly provide both specification and documentation, it requires discipline to write tests in this way. Particularly when the algorithm is so simple as is the case here, it's very tempting to just hard-code the values directly into the assertion.

In a future post, I’ll explain how we can force ourselves to do the right thing per default.

In my Zero-Friction TDD series, I focus on establishing a set of good habits that can potentially make you more productive while writing tests TDD style. While being able to quickly write good tests is important, this is not the only quality on which you should focus.

Maintainability, not only of your production code, but also of your test code, is important, and the DRY principle is just as applicable here.

Consider a test like this:

[TestMethod]

public void SomeTestUsingConstructorToCreateSut()

{

    // Fixture setup

    MyClass sut = new MyClass();

    // Exercise system

    // ...

    // Verify outcome

    // ...

    // Teardown

}

Such a test represents an anti-pattern you can easily fall victim to. The main item of interest here is that I create the SUT using its constructor. You could say that I have hard-coded this particular constructor usage into my test.

This is not a problem if there's only one test of MyClass, but once you have many, this starts to become a drag on your ability to refactor your code.

Imagine that you want to change the constructor of MyClass from the default constructor to one that takes a dependency, like this:

public MyClass(IMyInterface dependency)

If you have many (in this case, not three, but dozens) tests using the default constructor, this simple change will force you to visit all these tests and modify them to be able to compile again.

If, instead, we use a factory to create the SUT in each test, there's a single place where we can go and update the creation logic.

[TestMethod]

public void SomeTestUsingFactoryToCreateSut()

{

    // Fixture setup

    MyClass sut = MyClassFactory.Create();

    // Exercise system

    // ...

    // Verify outcome

    // ...

    // Teardown

}

The MyClassFactory class is a test-specific helper class (more formally, it's part of our SUT API Encapsulation) that is part of the unit test project. Using this factory, we only need to modify the Create method to implement the constructor change.

internal static MyClass Create()

{

    IMyInterface fake = new FakeMyInterface();

    return new MyClass(fake);

}

Instead of having to modify many individual tests to support the signature change of the constructor, there's now one central place where we can go and do that. This pattern supports refactoring much better, so consider making this a habit of yours.

One exception to this rule concerns tests that explicitly deal with the constructor, such as this one:

[ExpectedException(typeof(ArgumentNullException))]

[TestMethod]

public void CreateWithNullMyInterfaceWillThrow()

{

    // Fixture setup

    IMyInterface nullMyInterface = null;

    // Exercise system

    new MyClass(nullMyInterface);

    // Verify outcome (expected exception)

    // Teardown

}

In a case like this, where you explicitly want to deal with the constructor in an anomalous way, I consider it reasonable to deviate from the rule of using a factory to create the SUT. Although this may result in a need to fix the SUT creation logic in more than one place, instead of only in the factory itself, it's likely to be constrained to a few places instead of dozens or more, since normally, you will only have a handful of these explicit constructor tests.

Compared to my Zero-Friction TDD tips and tricks, this particular advice has the potential to marginally slow you down. However, this investments pays off when you want to refactor your SUT's constructor, and remember that you can always just write the call to the factory and move on without implementing it right away.

In my original post on Zero-Friction TDD, I continually updated the list of posts in the series, so that there would always be a central 'table of contents' for this topic.

Since I'll lose the ability to keep editing any of my previous postings on the old ploeh blog, the present post now contains the most updated list of Zero-Friction TDD articles:

• Naming SUT Test Variables
• Naming Direct Output Variables
• Anonymous Variables
• Ignore Irrelevant Return Values
• testmethod Code Snippet
• 3 Is Many
• Why Use AreEqual<T>?
• Assert Messages Are Not Optional
• Use The Generate Method Stub Smart Tag To Stay In The Zone
• Test-Driven Properties
• SUT Factory
• Derived Values Ensure Executable Specification
• Constrained Non-Determinism
• Explicit Expectations
• Fixture Object
• AutoFixture As Fixture Object