# Sunday, April 19, 2009

(Just back after 14 days in Morocco, I'll pick up where I left…)

The last group of built-in special creation algorithms for AutoFixture, besides strings and numbers, concerns Booleans.

The default algorithm is to alternate between true and false, starting with true; i.e. the first time you invoke

fixture.CreateAnonymous<bool>();

it will return true, the next time false, then true again, etc.

The reason I chose this algorithm was because I wanted to ensure that the first time AutoFixture creates an anonymous Boolean, it will return true, which is different than the default (false, in case you were in doubt). This gives us better assurance that a given constructor argument or property is being assigned a real value instead of the default.

Like with numbers, using the overload that takes a seed has no effect, and the seed is simply ignored.

fixture.CreateAnonymous(true);

In other words, the above method is still going to return false every second time, so it doesn’t really make sense to use this overload at all for Booleans (or numbers).

posted on Sunday, April 19, 2009 7:40:36 PM (Romance Daylight Time, UTC+02:00)  #    Comments [0] Trackback
# Friday, April 03, 2009

Previously, we saw how AutoFixture creates strings. In this post, I’ll explain how it creates numbers. Once again, the algorithm that I’ll explain here is the default algorithm, but if you don’t like it, you can replace it with something else.

It’s very simple: Numbers are returned in the ordered sequence of natural numbers (1, 2, 3, 4, 5, …). The first time you call

fixture.CreateAnonymous<int>();

the returned number will be 1, the second time 2, etc.

The reason I chose that particular algorithm is because it creates numbers that we, as humans, find… well… natural!

A lot of the domains we model work with natural numbers, and even if you write an API where negative numbers are allowed, it’s fairly unlikely that positive numbers will not be allowed. Thus, in most cases, small positive integers tend to be ‘nice’ values in most APIs – and recall that when we do TDD, we focus on the Happy Path, so it’s important to pick values that take us down that path.

Using the overload that takes a seed, like this:

fixture.CreateAnonymous(42);

has no effect – the seed (in this case 42) is simply ignored, so if you call this after first calling the parameterless overload twice, the return number is going to be 3.

Each number type, however, has its own sequence, so even if you’ve been creating a few Int32 instances like above,

fixture.CreateAnonymous<decimal>();

will return 1.

The following number types all work that way:

  • Byte
  • Decimal
  • Double
  • Int16
  • Int32
  • Int64
  • SByte
  • Single
  • UInt16
  • UInt32
  • UInt64
posted on Friday, April 03, 2009 11:07:13 PM (Romance Daylight Time, UTC+02:00)  #    Comments [4] Trackback
# Thursday, April 02, 2009

As previously hinted, AutoFixture creates primitive types like strings, numbers, etc. using special algorithms.

In this post, I’ll describe the default algorithm for strings. If you don’t like this particular algorithm, you can replace it with your own.

If you don’t care about the created string at all, you can just create it like this:

string anonymousText = 
    fixture.CreateAnonymous<string>();

The algorithm is simply to create a new Guid and convert it to a string, so that anonymousText will have a value like “f5cdf6b1-a473-410f-95f3-f427f7abb0c7”. Obviously, you don’t know exactly which value will be returned, but that’s the whole point of Constrained Non-Determinism.

When I create string values as Explicit Expectations, I prefer that the Assert failure message contains some sort of hint for me, so I can instead provide a hint to the CreateAnonymous method:

string anonymousName = fixture.CreateAnonymous("Name");

This overload is still generic, but since I provide a string as input, type inferencing takes care of the rest.

This is still going to create a Guid, but will now prepend the hint, giving a string like “Name30a35da1-d681-441b-9db3-77ff51728b58”.

Now, when my test fails, I’ll get an error message equivalent to

“Assert.AreEqual failed. Expected:<Namef2b1f55b-e9dc-4aac-a1ab-128dc80d3b71>. Actual:<ploeh>. Boo hiss”

which I find marginally more informative than if the hint hadn’t been there.

In a future post, I’ll explain how you can replace this algorithm with something else.

posted on Thursday, April 02, 2009 7:29:35 AM (Romance Daylight Time, UTC+02:00)  #    Comments [0] Trackback
# Tuesday, March 24, 2009

AutoFixture creates Anonymous Variables, but you’d probably like to know how it does it. This post explains how.

As we previously saw, the CreateAnonymous method can create a new instance of a type known to it only from its type parameter:

MyClass sut = fixture.CreateAnonymous<MyClass>();

AutoFixture was never compiled with any knowledge of the MyClass type, so it obviously uses Reflection to create the instance. That’s hardly surprising in itself.

In the case of MyClass, it has a default constructor, so creating an instance is as simple as it can be, but what happens if we instead ask for a more complex instance?

As an example, the ComplexParent type has this constructor:

public ComplexParent(ComplexChild child)

ComplexChild, however, has two constructors:

public ComplexChild(string name)

and

public ComplexChild(string name, int number)

So what happens when we ask AutoFixture to create an instance of ComplexParent?

ComplexParent only has a single public constructor, so AutoFixture doesn’t have any other choice than picking that. This means that it must now create an anonymous instance of ComplexChild.

Fortunately, AutoFixture’s raison d’être is creating objects, so creating an instance of ComplexChild isn’t a big deal; the only thing it needs to figure out is which constructor to pick. When multiple public constructors are available, it always picks the one with the fewest number of arguments – in this case ComplexChild(string).

Obviously, it then needs to create an anonymous string value. For primitive types like strings, numbers and booleans, AutoFixture has custom algorithms for value creation. Since I’ll cover those mechanisms later, suffice it to say that Constrained Non-Determinism is used to create an anonymous string.

At this point, AutoFixture has all the information it needs, and it can now return a properly initialized instance of ComplexParent.

This ability to create instances of almost arbitrarily complex types is a real time-saver: That, more than the ability to create single strings or numbers, was the reason I originally created AutoFixture, since I got tired of initializing complex object graphs just to satisfy some API that the Test Fixture requires.

It also has the additional advantage that it hides all the irrelevant object creation code that the Test Fixture needs, but which isn’t relevant for the test at hand.

posted on Tuesday, March 24, 2009 9:22:49 PM (Romance Standard Time, UTC+01:00)  #    Comments [4] Trackback
# Monday, March 23, 2009

Today I spent my first working day at Safewhere, where I’ll be working as a Senior Software Engineer. In the last couple of years, Safewhere and I have crossed paths a couple of times, and each time they always left me with the impression of a very professional and congenial ISV, so I’m super-excited to be joining!

Expect the occasional Geneva or Federation-related blog post to intermingle with the usual TDD stuff in the future :)

posted on Monday, March 23, 2009 8:03:38 PM (Romance Standard Time, UTC+01:00)  #    Comments [3] Trackback
# Sunday, March 22, 2009

It gives me great pleasure to announce my latest project: AutoFixture!

What is AutoFixture?

In essence, AutoFixture is a library that creates Anonymous Variables for you when you write unit tests. The intention is that it should enhance your productivity when you do Test-Driven Development – as it has done mine.

Instead of using mental resources on creating Anonymous Variables, AutoFixture can do it for you. By default, it uses Constrained Non-Determinism, but you can configure it to behave differently if you wish.

Here’s a very basic example:

[TestMethod]
public void IntroductoryTest()
{
    // Fixture setup
    Fixture fixture = new Fixture();
 
    int expectedNumber = fixture.CreateAnonymous<int>();
    MyClass sut = fixture.CreateAnonymous<MyClass>();
    // Exercise system
    int result = sut.Echo(expectedNumber);
    // Verify outcome
    Assert.AreEqual<int>(expectedNumber, result, "Echo");
    // Teardown
}

The Fixture class is your main entry point to AutoFixture. You can use it as is, customize it, or derive from it as you please; it makes a great base class for a Fixture Object.

The expectedNumber variable may be an Explicit Expectation, but its value is Anonymous, so instead of coming up with a number ourselves, we can let the CreateAnonymous<T> method do it for us.

This method can create instances of most CLR types as long as they have a public constructor (it uses Reflection), but for many primitive types (like Int32), it has specific, customizable algorithms for creating values using Constrained Non-Determinism.

When creating the SUT, we can also use Fixture as an excellent SUT Factory. Since it will do whatever it can to create an instance of the type you ask for, it is pretty robust if you decide to refactor the SUT’s constructor.

The above example only hints at what AutoFixture can do. Since the example is very simple, it may be hard to immediately understand its value, so in future posts I will expand on specific AutoFixture features and principles.

Getting started with AutoFixture is as simple as downloading it from CodePlex and referencing the assembly in Visual Studio.

posted on Sunday, March 22, 2009 7:50:54 AM (Romance Standard Time, UTC+01:00)  #    Comments [2] Trackback
# Monday, March 16, 2009

(A Zero-Friction TDD post)

For a simple API, setting up the Fixture may be as simple as creating a new instance of the SUT, and possibly any Expected or Anonymous Variables. On the other hand, for a complex API, setting up the fixture may require quite a bit of (potentially repetitive) code.

Since the DRY principle also applies to test code, it quickly becomes necessary to create test-specific helper methods and other SUT API Encapsulation, and I've found that instead of creating a more or less unplanned set of disconnected helper methods, it's much cleaner (and, not to mention, much more object-oriented) to create a single object that represents the Fixture, and attach the helper methods to this object.

Let's look at an example.

Here's a unit test with a complex Fixture Setup:

[TestMethod]
public void NumberSumIsCorrect_Naïve()
{
    // Fixture setup
    Thing thing1 = new Thing()
    {
        Number = 3,
        Text = "Anonymous text 1"
    };
    Thing thing2 = new Thing()
    {
        Number = 6,
        Text = "Anonymous text 2"
    };
    Thing thing3 = new Thing()
    {
        Number = 1,
        Text = "Anonymous text 3"
    };
 
    int expectedSum = new[] { thing1, thing2, thing3 }.
        Select(t => t.Number).Sum();
 
    IMyInterface fake = new FakeMyInterface();
    fake.AddThing(thing1);
    fake.AddThing(thing2);
    fake.AddThing(thing3);
 
    MyClass sut = new MyClass(fake);
    // Exercise system
    int result = sut.CalculateSumOfThings();
    // Verify outcome
    Assert.AreEqual<int>(expectedSum, result,
        "Sum of things");
    // Teardown
}

If this was a truly one-off test and you know with certainty that there's going to be no other tests just remotely similar to this one, just hard-coding the entire Fixture Setup inline is in order, but as soon as the need for similar tests arises, this approach leads to repetitive code, and hence unmaintainable tests.

The more repetitive code that can be delegated to helper methods the better. A common refactoring of the previous test might then look something like this:

[TestMethod]
public void NumberSumIsCorrect_Helpers()
{
    // Fixture setup
    Thing thing1 = MyClassTest.CreateAnonymousThing();
    Thing thing2 = MyClassTest.CreateAnonymousThing();
    Thing thing3 = MyClassTest.CreateAnonymousThing();
    Thing[] things = new[] { thing1, thing2, thing3 };
 
    int expectedSum = things.Select(t => t.Number).Sum();
 
    IMyInterface fake = new FakeMyInterface();
    MyClassTest.AddThingsToMyInterface(fake, things);
 
    MyClass sut = new MyClass(fake);
    // Exercise system
    int result = sut.CalculateSumOfThings();
    // Verify outcome
    Assert.AreEqual<int>(expectedSum, result,
        "Sum of things");
    // Teardown
}

While this is better, the helper methods are static methods, so it's necessary to pass too much state around. The array of Things and the fake is both needed in the test itself, as well as in the AddThingsToMyInterface helper method.

By moving and refactoring the helper methods to a new class that represents the Fixture, the test code becomes both more reusable and more readable.

[TestMethod]
public void NumberSumIsCorrect_FixtureObject()
{
    // Fixture setup
    MyClassFixture fixture = new MyClassFixture();
    fixture.AddAnonymousThings();
 
    int expectedSum = 
        fixture.Things.Select(t => t.Number).Sum();
    MyClass sut = fixture.CreateSut();
    // Exercise system
    int result = sut.CalculateSumOfThings();
    // Verify outcome
    Assert.AreEqual<int>(expectedSum, result,
        "Sum of things");
    // Teardown
}

The MyClassFixture instance now holds the state of the Fixture, so there's much less need to pass around as much data as before. The set of Things is now contained within the Fixture object itself, and the fake has totally disappeared from the test; it's still present, but now encapsulated within MyClassFixture.

internal class MyClassFixture
{
    public MyClassFixture()
    {
        this.Fake = new FakeMyInterface();
        this.Things = new List<Thing>();
    }
 
    internal FakeMyInterface Fake { get; private set; }
 
    internal IList<Thing> Things { get; private set; }
 
    internal void AddAnonymousThings()
    {
        int many = 3;
        for (int i = 0; i < many; i++)
        {
            Thing t = this.CreateAnonymousThing();
            this.Things.Add(t);
            this.Fake.AddThing(t);
        }
    }
 
    internal MyClass CreateSut()
    {
        return new MyClass(this.Fake);
    }
 
    private Thing CreateAnonymousThing()
    {
        Thing t = new Thing();
        t.Number = this.Things.Count + 1;
        t.Text = Guid.NewGuid().ToString();
        return t;
    }
}

The CreateAnonymousThing method uses Constrained Non-Determinism to create unique Thing instances. The AddAnonymousThings method uses 3 as an equivalence of many, and the CreateSut method acts as a SUT Factory.

This is both more reusable and more expressive than a collection of disjointed static helper methods.

Whenever I begin to feel that setting up a Test Fixture is becoming too cumbersome, Fixture Object is the first pattern I consider.

posted on Monday, March 16, 2009 9:13:29 PM (Romance Standard Time, UTC+01:00)  #    Comments [0] Trackback
# Wednesday, March 11, 2009

In previous Zero-Friction TDD posts, I've discussed naming SUT and Direct Output variables, as well as the importance of explicitly describing the relationship between input and expected results.

Everything you can do to help the Test Reader understand what's going on increases the quality of the test. Having a naming convention for expectations help in that regard, and it also helps coming up with variable names, thus saving yourself a bit of mental context switching.

My naming convention is to always prefix my expectation variables with the term expected.

[TestMethod]
public void DoStuffWillReturnMessage()
{
    // Fixture setup
    string expectedResult = "ploeh";
    MyClass sut = new MyClass();
    // Exercise system
    string result = sut.DoStuff(expectedResult);
    // Verify outcome
    Assert.AreEqual<string>(expectedResult, result,
        "DoStuff");
    // Teardown
}

In a test like this, the relationship between the input and output is straightforward, but in other cases it may be more complicated. Since tests should explicitly state the relationship between input and output, it may sometimes be necessary to reproduce parts of the SUT's behavior in the test to specify this association.

Do I really recommend duplicating the SUT's code in the test? Isn't this a violation of the DRY principle? And do I really think that embedding complex code in a test is a good idea?

No, no, and no.

What I really mean is best illustrated with an example. Imagine that you want to write an extension method that converts a string to PascalCase. There are several different rules that must be applied to such an algorithm, such as

  • Convert the first letter in a word to upper case
  • Convert the remaining letters in the word to lower case
  • Remove white space

The real algorithm would need to split the string into words along white space boundaries, then loop through this list and perform the conversion for each word, and finally concatenate all the words. However, I don't think you should reproduce this algorithm in any single test.

What you can do instead is to split this behavior into several tests that each test a small part of this specification, carefully avoiding any control flow language features (such as if, switch, for, etc.).

One such test might look like this:

[TestMethod]
public void ToPascalCaseWillConvertFirstLetterToUpper()
{
    // Fixture setup
    string anonymousText = "pLOeh";
    string expectedLetter =
        anonymousText.First().ToString().ToUpper();
    // Exercise system
    string result =
        anonymousText.ToPascalCase();
    // Verify outcome
    Assert.AreEqual<string>(expectedLetter,
        result.First().ToString(), "ToPascalCase");
    // Teardown
}

While the complete implementation of ToPascalCase is more complex, I've extracted a tiny bit of the specification and simulated just that for the special case where there's only one word. Granted, there's a lot of method calls, but I expect that these have already been thoroughly tested, so I use them with confidence. The cyclomatic complexity of the test is minimal.

As an aside, note that I'm using LINQ queries to get the first letter of the string, instead of Substring(0, 1), since I find that the LINQ methods much better communicate intent.

I use a similar naming convention for unexpected values, imaginatively prefixing my variables with the term unexpected.

[TestMethod]
public void CreateThingWillCreateThingWithCorrectGuid()
{
    // Fixture setup
    Guid unexpectedId = Guid.Empty;
    MyClass sut = new MyClass();
    // Exercise system
    Thing result = sut.CreateThing();
    // Verify outcome
    Assert.AreNotEqual<Guid>(unexpectedId, result.Id,
        "CreateThing");
    // Teardown
}

Having a naming convention for expected values not only increases your productivity when writing tests, but also increases test maintainability.

posted on Wednesday, March 11, 2009 8:54:38 PM (Romance Standard Time, UTC+01:00)  #    Comments [0] Trackback
# Tuesday, March 10, 2009

In the last couple of years, there’s been a lot of debate in the community on the philosophy behind TDD and where to put the emphasis – even to the point of debating whether the acronym stands for Test-Driven Development or Test-Driven Design.

Other people don’t like the emphasis on tests, since that makes TDD sound like a Testing discipline, and not a Development discipline. Instead, they prefer terms like Example-Driven Design/Development (EDD) or even Design by Example (DbE).

This view seems to me to be particularly prevalent in Microsoft, where there’s a rather sharp distinction between developers and testers (job titles to the contrary) – I guess that’s one of the reasons why xUnit.net (a project initiated by Microsoft employees) uses the attribute Fact instead of Test or TestMethod.

For people used to SCRUM or other agile methodologies, this distinction is more blurred, and they also seem to accept the T in TDD more willingly.

However, the adherents of EDD claim that the mere presence of the word test make some developers block any further input and stop listening. They may be right in that.

They also claim that the tests in TDD/EDD are nothing more than accidental artifacts of the development process, and hence argue that we shouldn’t call them tests at all. However, if that’s true, this little story related by Ayende must be an example of EDD in its purest form :)

To me, the tests are also important. Since 2003 I’ve been practicing TDD, and while I love how it helps me arrive at better design, I also savor the safety net that my suite of tests gives me. The tests that I write during TDD define the behavior of the software. In many cases, I’d even claim that such a regression test suite is more valuable than a Quality Assurance (QA) regression test suite – after all, a QA suite may catch some edge cases, but they don’t focus on the intended behavior of the system, but often more on how to break it - but I digress…

My recent posts on Executable Specification and Constrained Non-Determinism help explain my current stance in this debate: In my opinion, EDD fails to establish a relationship by not providing Derived Values. After all, what does a test like the following specify?

[TestMethod]
public void InvertWillReverseText_Naïve()
{
    // Fixture setup
    MyClass sut = new MyClass();
    // Exercise system
    string result = sut.Invert("ploeh");
    // Verify outcome
    Assert.AreEqual<string>("heolp", result, "Invert");
    // Teardown
}

How would you implement the Invert method? Here’s one possible implementation:

return "heolp";

Obviously, you could now write a new test that gives a second example of input and outcome and force me to write a more sophisticated algorithm. However, with only two examples, I might still be tempted to write a switch statement with some hard-coded return values until you’ve written so many ‘examples’ that you’ve coerced me into writing the more general (and correct) algorithm.

Such an approach I find inefficient.

Instead, by using Constrained Non-Determinism to force myself to define Derived Values, each test fully specifies the desired behavior. It doesn’t provide examples. It provides the specification, and instead of having to write several similar examples to coerce a general algorithm to emerge, I can usually nail it in a single test.

This approach could be styled Specification-Driven Development, and that’s how I’ve been writing code for the last year or so.

posted on Tuesday, March 10, 2009 10:04:38 PM (Romance Standard Time, UTC+01:00)  #    Comments [4] Trackback
# Thursday, March 05, 2009

This may turn out to be the most controversial of my Zero-Friction TDD posts so far, as it supposedly goes against conventional wisdom. However, I have found this approach to be really powerful since I began using it about a year ago.

In my previous post, I explained how Derived Values help ensure that tests act as Executable Specification. In short, a test should clearly specify the relationship between input and outcome, as this test does:

[TestMethod]
public void InvertWillReverseText()
{
    // Fixture setup
    string anonymousText = "ploeh";
    string expectedResult =
        new string(anonymousText.Reverse().ToArray());
    MyClass sut = new MyClass();
    // Exercise system
    string result = sut.Invert(anonymousText);
    // Verify outcome
    Assert.AreEqual<string>(expectedResult, result,
        "DoWork");
    // Teardown
}

However, it is very tempting to just hardcode the expected value. Consistently using Derived Values to establish the relationship between input and outcome requires discipline.

To help myself enforce this discipline, I use well-defined, but essentially random, input, because when the input is random, I don't know the value at design time, and hence, it is impossible for me to accidentally hard-code any assertions.

[TestMethod]
public void InvertWillReverseText_Cnd()
{
    // Fixture setup
    string anonymousText = Guid.NewGuid().ToString();
    string expectedResult =
        new string(anonymousText.Reverse().ToArray());
    MyClass sut = new MyClass();
    // Exercise system
    string result = sut.Invert(anonymousText);
    // Verify outcome
    Assert.AreEqual<string>(expectedResult, result,
        "DoWork");
    // Teardown
}

For strings, I prefer Guids, as the above example demonstrates. For numbers, I often just use the sequence of natural numbers (i.e. 1, 2, 3, 4, 5...). For booleans, I often use an alternating sequence (i.e. true, false, true, false...).

While this technique causes input to become non-deterministic, I always pick the non-deterministic value-generating algorithm in such a way that it creates 'nice' values; I call this principle Constrained Non-Determinism. Values are carefully generated to stay far away from any boundary conditions that may cause the SUT to behave differently in each test run.

Conventional unit testing wisdom dictates that unit tests should be deterministic, so how can I possibly endorse this technique?

To understand this, it's important to know why the rule about deterministic unit tests exist. It exists because we want to be certain that each time we execute a test suite, we verify the exact same behavior as we did the last time (given that no tests changed). Since we also use test suites as regression tests, it's important that we can be confident that each and every test run verifies the exact same specification.

Constrained Non-Determinism doesn't invalidate that goal, because the algorithm that generates the values must be carefully picked to always create values that stay within the input's Equivalence Class.

In a surprisingly large set of APIs, strings, for example, are treated as opaque values that don't influence behavior in themselves. Many enterprise applications mostly store and read data from persistent data stores, and the value of a string in itself is often inconsequential from the point of view of the code's execution path. Data stores may have constraints on the length of strings, so Constrained Non-Determinism dictates that you should pick the generating algorithm so that the string length always stays within (or exceeds, if that's what you want to test) the constraint. Guid.ToString always returns a string with the length of 36, which is fine for a large number of scenarios.

Note that Constrained Non-Determinism is only relevant for Anonymous Variables. For input where the value holds a particular meaning in the context of the SUT, you will still need to hand-pick values as always. E.g. if the input is expected to be an XML string conforming to a particular schema, a Guid string makes no sense.

A secondary benefit of Constrained Non-Determinism is that you don't have to pause to come up with values for Anonymous Variables when you are writing the test.

While this advice may be controversial, I can only recommend it - I've been using this technique for about a year now, and have only become more fond of it as I have gained more experience with it.

posted on Thursday, March 05, 2009 9:23:05 PM (Romance Standard Time, UTC+01:00)  #    Comments [2] Trackback