Dealing With Constrained Input

Friday, 01 May 2009 03:56:00 UTC

As a response to my description of how AutoFixture creates objects, Klaus asked:

“[What] if the constructor of ComplexChild imposes some kind of restriction on its parameter? If, for example, instead of the "name" parameter, it would take a "phoneNumber" parameter (as a string), and do some format checking?”

Now that we have covered some of the basic features of AutoFixture, it's time to properly answer this excellent question.

For simplicity's sake, let's assume that the phone number in question is a Danish phone number: This is pretty good for example code, since a Danish phone number is essentially just an 8-digit number. It can have white space and an optional country code (+45), but strip that away, and it's just an 8-digit number. However, there are exceptions, since the emergency number is 112 (equivalent to the American 911), and other 3-digit special numbers exist as well.

With that in mind, let's look at a simple Contact class that contains a contact's name and Danish phone number. The constructor might look like this:

public Contact(string name, string phoneNumber)
{
    this.Name = name;
    this.PhoneNumber = 
        Contact.ParsePhoneNumber(phoneNumber);
}

The static ParsePhoneNumber method strips away white space and optional country code and parses the normalized string to a number. This fits the scenario laid out in Klaus' question.

So what happens when we ask AutoFixture to create an instance of Contact? It will Reflect over Contact's constructor and create two new anonymous string instances - one for name, and one for phoneNumber. As previously described, each string will be created as a Guid prepended with a named hint - in this case the argument name. Thus, the phoneNumber argument will get a value like "phoneNumberfa432351-1563-4769-842c-7588af32a056", which will cause the ParsePhoneNumber method to throw an exception.

How do we deal with that?

The most obvious fix is to modify AutoFixture's algorithm for generating strings. Here an initial attempt:

fixture.Register<string>(() => "112");

This will simply cause all generated strings to be "112", including the Contact instance's Name property. In unit testing, this may not be a problem in itself, since, from an API perspective, the name could in principle be any string.

However, if the Contact class also had an Email property that was parsed and verified from a string argument, we'd be in trouble, since "112" is not a valid email address.

We can't easily modify the string generation algorithm to fit the requirements for both a Danish telephone number and an email address.

Should we then conclude that AutoFixture isn't really useful after all?

On the contrary, this is a hint to us that the Contact class' API could be better. If an automated tool can't figure out how to generate correct input, how can we expect other developers to do it?

Although humans can make leaps of intuition, an API should still go to great lengths to protect its users from making mistakes. Asking for an unbounded string and then expecting it to be in a particular format may not always be the best option available.

In our particular case, the Value Object pattern offers a better alternative. Our first version of the DanishPhoneNumber class simply takes an integer as a constructor argument:

public DanishPhoneNumber(int number)
{
    this.number = number;
}

If we still need to parse strings (e.g. from user input), we could add a static Parse, or even a TryParse, method and test that method in isolation without involving the Contact class.

This neatly solves our original issue with AutoFixture, since it will now create a new instance of DanishPhoneNumber as part of the creation process when we ask for an anonymous Contact instance.

The only remaining issue is that by default, the number fed into the DanishPhoneNumber instance is likely to be considerably less than 112 - actually, if no other Int32 instances are created, it will be 1.

This will be a problem if we modify the DanishPhoneNumber constructor to look like this:

public DanishPhoneNumber(int number)
{
    if ((number < 112) ||
        (number > 99999999))
    {
        throw new ArgumentOutOfRangeException("number");
    }
    this.number = number;
}

Unless a unit test has already caused AutFixture to previously create 111 other integers (highly unlikely), CreateAnonymous<Contact> is going to throw an exception.

This is easy to fix. Once again, the most obvious fix is to modify the creation algorithm for integers.

fixture.Register<int>(() => 12345678);

However, this will cause that particular instance of Fixture to return 12345678 every time you ask it to create an anonymous integer. Depending on the scenario, this may or may not be a problem.

A more targeted solution is to specifically address the algorithm for generating DanishPhoneNumber instances:

fixture.Register<int, DanishPhoneNumber>(i => 
    new DanishPhoneNumber(i + 112));

Here, I've even used the Register overload that automatically provides an anonymous integer to feed into the DanishPhoneNumber constructor, so all I have to do is ensure that the number falls into the proper range. Adding 112 (the minimum) neatly does the trick.

If you don't like the hard-coded value of 112 in the test, you can use that to further drive the design. In this case, we can add a MinValue to DanishPhoneNumber:

fixture.Register<int, DanishPhoneNumber>(i =>
    new DanishPhoneNumber(i + 
        DanishPhoneNumber.MinValue));

Obvously, MinValue will also be used in DanishPhoneNumber's constructor to define the lower limit of the Guard Clause.

In my opinion, a good API should guide the user and make it difficult to make mistakes. In many ways, you can view AutoFixture as an exceptionally dim user of your API. This is the reason I really enjoyed receiving Klaus' original question: Like other TDD practices, AutoFixture drives better design.


Speaking at 7N IT Conference 2009

Tuesday, 28 April 2009 21:02:32 UTC

At May 12 2009 I'll be speaking at 7N's IT Konference 2009 (in Danish, so that's no spelling error). You can read the program here.

The topic of my talk will be TDD patterns and terminology, so I'll discuss Fixtures, Stubs, Mocks and the like. As always, xUnit Test Patterns will form the basis of my vocabulary.

Of the other speakers, I'm particularly looking forward to hear my good friend Martin Gildenpfennig from Ative speak!


Replacing AutoFixture's Default Algorithms

Monday, 27 April 2009 17:42:07 UTC

Several times in my previous AutoFixture posts, I've insinuated that you can change the algorithms used for creating strings, numbers and so on, if you don't like the defaults.

One way you can do this is by simply using the Register method that I introduced in my previous post. Let's say that you want to replace the string algorithm to simply return a specific string:

fixture.Register<string>(() => "ploeh");

No matter how many times you'll call CreateAnonymous<string> on that particular fixture object, it will always return ploeh.

The Register method is really only a type-safe convenience method that wraps access to the TypeMappings property. TypeMappings is just a Dictionary of types mapped to functions. By default, the Fixture class has a set of pre-defined TypeMappings for primitive types such as strings, numbers and booleans, so you could access the function used to generate strings by indexing into this Dictionary with the System.String type.

Equivalent to the above example, you could alternatively replace the string algorithm like this:

fixture.TypeMappings[typeof(string)] = s => "fnaah";

Instead of using the Register method, I here assign a lambda expression directly to the key identified by the System.String type. This is what the Register method does, so the result is exactly the same.

However, you may have noticed that by accessing TypeMappings directly, the signature of the function is different. The Register method takes a Func<T>, whereas the TypeMappings Dictionary expects a Func<object, object>. As you can see, the Register method is more type-safe, but the TypeMappings Dictionary gives you a chance to utilize the optional seed that one of the CreateAnonymous overloads takes.

You could, for example, do this:

fixture.TypeMappings[typeof(string)] = s =>
    string.Format((string)s, new Random().Next(100));

Although this particular algorithm has a built-in weakness (can you spot it?), we can now use the seed to provide a format string, like this:

string result = fixture.CreateAnonymous("Risk: {0}%");

which will yield a result like Risk: 32%.

When I designed the extensibility mechanism for AutoFixture, I seriously considered defining an interface that all TypeMappings had to implement, but I ended up preferring a Func<object, object> instead, since this allows you to redefine a particular algorithm inline in a test by using an anonymous delegate or lambda expression, and you can also reuse an existing algorithm, as long as it fits the signature.


Comments

Paul #
Hello, I am looking at Autofixture now, but cannot find TypeMappings. They were removed from the library?
2012-07-25 07:05 UTC
Yes, TypeMappings were removed in AutoFixture 2.0, which has a much more flexible extensibility model. Ultimately, a custom ISpecimenBuilder can do everything you may want to do.
2012-07-25 08:37 UTC

Dealing With Types Without Public Constructors

Thursday, 23 April 2009 19:51:42 UTC

Now that I've described how AutoFixture creates objects, it's time to look at a bit more advanced scenario. As you may recall, AutoFixture creates an object graph based on the public constructors of the types contained in that object graph.

That's all well and good when all involved types have public constructors, but what happens when this is not the case?

Imagine that the MyClass constructor has this signature:

public MyClass(IMyInterface mi)

Since IMyInterface is an interface it has no public constructors, so this will not work:

Fixture fixture = new Fixture();
MyClass sut = fixture.CreateAnonymous<MyClass>();

The second line of code will throw an ArgumentException ObjectCreationException* stating that “AutoFixture was unable to create an instance of type Ploeh.QualityTools.AutoFixtureDocumentationTest.Intermediate.IMyInterface, since it has no public constructor.” Not terribly surprising, actually.

To resolve this issue, the Register method allows you to specify a custom function that creates an object of the requested type. In the case of IMyInterface, that would be a Func<IMyInterface>:

Fixture fixture = new Fixture();
fixture.Register<IMyInterface>(() => 
    new FakeMyInterface());
MyClass sut = fixture.CreateAnonymous<MyClass>();

Here, I use a lambda expression to register the FakeMyInterface type, so that every time that particular Fixture instance is asked to create an instance of IMyInterface, it will invoke the lambda expression, and thus return an instance of FakeMyInterface (which obviously implements IMyInterface).

The Register method simply lets you map types without public constructors to concrete types created by a function you specify.

A more advanced scenario arises if you wish to use a specific text with this FakeMyInterface constructor overload:

public FakeMyInterface(int number, string text)

Obviously, you can do it manually like this:

Fixture fixture = new Fixture();
int anonymousNumber = fixture.CreateAnonymous<int>();
string knownText = "This text is not anonymous";
fixture.Register<IMyInterface>(() => 
    new FakeMyInterface(anonymousNumber, knownText));

Here, I simply use the fixture object to create an anonymous number, while the knownText variable is explicitly assigned a value, and then both are used as outer variables in the Register function.

This is, however, a common scenario, so the Register method has some convenience overloads that will supply anonymous input parameters for you to use or throw away as you like. This means that I can rewrite the above example to this:

Fixture fixture = new Fixture();
string knownText = "This text is not anonymous";
fixture.Register<int, string, IMyInterface>((i, s) => 
    new FakeMyInterface(i, knownText));

Compared to the previous example, I save a line of code. The drawback is that I have to explicitly specify the type parameters to the Register method. In my book, that's a pretty good tradeoff, as it removes a line of irrelevant code, and allows the test to focus on the relevant parts.

Notice that this Register overload creates both an anonymous integer and an anonymous string, but since I don't want to use the anonymous string (s), I just ignore it and use the knownText variable instead.

*Edit (2009-09-02): Changed the name of the Exception type to correctly reflect a breaking change introduced in AutoFixture .8.5.


Creating Booleans With AutoFixture

Sunday, 19 April 2009 17:40:36 UTC

(Just back after 14 days in Morocco, I'll pick up where I left…)

The last group of built-in special creation algorithms for AutoFixture, besides strings and numbers, concerns Booleans.

The default algorithm is to alternate between true and false, starting with true; i.e. the first time you invoke

fixture.CreateAnonymous<bool>();

it will return true, the next time false, then true again, etc.

The reason I chose this algorithm was because I wanted to ensure that the first time AutoFixture creates an anonymous Boolean, it will return true, which is different than the default (false, in case you were in doubt). This gives us better assurance that a given constructor argument or property is being assigned a real value instead of the default.

Like with numbers, using the overload that takes a seed has no effect, and the seed is simply ignored.

fixture.CreateAnonymous(true);

In other words, the above method is still going to return false every second time, so it doesn't really make sense to use this overload at all for Booleans (or numbers).


Creating Numbers With AutoFixture

Friday, 03 April 2009 21:07:13 UTC

Previously, we saw how AutoFixture creates strings. In this post, I'll explain how it creates numbers. Once again, the algorithm that I'll explain here is the default algorithm, but if you don't like it, you can replace it with something else.

It's very simple: Numbers are returned in the ordered sequence of natural numbers (1, 2, 3, 4, 5, …). The first time you call

fixture.CreateAnonymous<int>();

the returned number will be 1, the second time 2, etc.

The reason I chose that particular algorithm is because it creates numbers that we, as humans, find… well… natural!

A lot of the domains we model work with natural numbers, and even if you write an API where negative numbers are allowed, it's fairly unlikely that positive numbers will not be allowed. Thus, in most cases, small positive integers tend to be ‘nice' values in most APIs - and recall that when we do TDD, we focus on the Happy Path, so it's important to pick values that take us down that path.

Using the overload that takes a seed, like this:

fixture.CreateAnonymous(42);

has no effect - the seed (in this case 42) is simply ignored, so if you call this after first calling the parameterless overload twice, the return number is going to be 3.

Each number type, however, has its own sequence, so even if you've been creating a few Int32 instances like above,

fixture.CreateAnonymous<decimal>();

will return 1.

The following number types all work that way:

  • Byte
  • Decimal
  • Double
  • Int16
  • Int32
  • Int64
  • SByte
  • Single
  • UInt16
  • UInt32
  • UInt64

Comments

florin #
Having them sequence ordered sounds like there will not be so "non-deterministic". Wouldn't a random number be more appropriate as it would allow testing of different cases each time. Sure it would not be reproducible but at least you will get to know that there might be something wrong with the code under test.
2010-05-21 10:29 UTC
The idea about constrained non-determinism is that although test values are essentially unknown, two consecutive test runs must traverse the same branch of source code each time. In some SUTs different integer values may cause different branches to be exectured for each test run, so I found it safer to use a deterministic sequence.

For complex objects (say: a class with two writable properties of the same number type) the ordering of assignment is undefined even though the sequence of numbers is deterministic. In other words, if you have the properties Number1 and Number2, you know one gets the value 1 and the other gets the value 2, but you don't know which one gets what (it's probably deterministic anyway because Reflection is likely to always return properties in the same order, but AutoFixture does nothing to explicitly order properties and fields).

Another similar strategy could be to use random numbers from a constant seed. This would give you another stable sequence of numbers.

If none of the above is a concern (numbers do not influence the execution path) you can also choose to use pure random numbers.

In any case you can use the Register method to override the default behavior for a given type, so it's entirely possible to set it up with random numbers instead of a rising sequence.
2010-05-21 12:33 UTC
If you offer an overload for the constructor that takes a seed, when/how is it used? As you say, it does not work in this case.
2011-03-05 23:16 UTC
Let's say that you want to change the integer algorithm to return the seed. This can be done like this:

fixture.Customize<int>(c => c.FromSeed(i => i));
2011-03-06 04:24 UTC

Creating Strings With AutoFixture

Thursday, 02 April 2009 05:29:35 UTC

As previously hinted, AutoFixture creates primitive types like strings, numbers, etc. using special algorithms.

In this post, I'll describe the default algorithm for strings. If you don't like this particular algorithm, you can replace it with your own.

If you don't care about the created string at all, you can just create it like this:

string anonymousText = 
    fixture.CreateAnonymous<string>();

The algorithm is simply to create a new Guid and convert it to a string, so that anonymousText will have a value like “f5cdf6b1-a473-410f-95f3-f427f7abb0c7”. Obviously, you don't know exactly which value will be returned, but that's the whole point of Constrained Non-Determinism.

When I create string values as Explicit Expectations, I prefer that the Assert failure message contains some sort of hint for me, so I can instead provide a hint to the CreateAnonymous method:

string anonymousName = fixture.CreateAnonymous("Name");

This overload is still generic, but since I provide a string as input, type inferencing takes care of the rest.

This is still going to create a Guid, but will now prepend the hint, giving a string like “Name30a35da1-d681-441b-9db3-77ff51728b58”.

Now, when my test fails, I'll get an error message equivalent to

"Assert.AreEqual failed. Expected:<Namef2b1f55b-e9dc-4aac-a1ab-128dc80d3b71>. Actual:<ploeh>. Boo hiss"

which I find marginally more informative than if the hint hadn't been there.

In a future post, I'll explain how you can replace this algorithm with something else.


How AutoFixture Creates Objects

Tuesday, 24 March 2009 20:22:49 UTC

AutoFixture creates Anonymous Variables, but you'd probably like to know how it does it. This post explains how.

As we previously saw, the CreateAnonymous method can create a new instance of a type known to it only from its type parameter:

MyClass sut = fixture.CreateAnonymous<MyClass>();

AutoFixture was never compiled with any knowledge of the MyClass type, so it obviously uses Reflection to create the instance. That's hardly surprising in itself.

In the case of MyClass, it has a default constructor, so creating an instance is as simple as it can be, but what happens if we instead ask for a more complex instance?

As an example, the ComplexParent type has this constructor:

public ComplexParent(ComplexChild child)

ComplexChild, however, has two constructors:

public ComplexChild(string name)

and

public ComplexChild(string name, int number)

So what happens when we ask AutoFixture to create an instance of ComplexParent?

ComplexParent only has a single public constructor, so AutoFixture doesn't have any other choice than picking that. This means that it must now create an anonymous instance of ComplexChild.

Fortunately, AutoFixture's raison d'être is creating objects, so creating an instance of ComplexChild isn't a big deal; the only thing it needs to figure out is which constructor to pick. When multiple public constructors are available, it always picks the one with the fewest number of arguments - in this case ComplexChild(string).

Obviously, it then needs to create an anonymous string value. For primitive types like strings, numbers and booleans, AutoFixture has custom algorithms for value creation. Since I'll cover those mechanisms later, suffice it to say that Constrained Non-Determinism is used to create an anonymous string.

At this point, AutoFixture has all the information it needs, and it can now return a properly initialized instance of ComplexParent.

This ability to create instances of almost arbitrarily complex types is a real time-saver: That, more than the ability to create single strings or numbers, was the reason I originally created AutoFixture, since I got tired of initializing complex object graphs just to satisfy some API that the Test Fixture requires.

It also has the additional advantage that it hides all the irrelevant object creation code that the Test Fixture needs, but which isn't relevant for the test at hand.


Comments

That sounds pretty neat. But what if the constructor of ComplexChild imposes some kind of restriction on its parameter? If, for example, instead of the "name" parameter, it would take a "phoneNumber" parameter (as a string), and do some format checking?
2009-03-25 09:17 UTC
Hi Klaus

Thank you for your question! It is an excellent question, particularly because it gives me the opportunity to touch on both some of the more advanced features of AutoFixture, as well as to demonstrate how TDD drives good design.

However, because of that, I'd also like to cover a bit more ground on some of AutoFixture's features before I dive into the details of my answer. In other words, I'd like to post a few more entries that will act as prerequisites before I post my answer to your particular question, so stay tuned, and I'll get to that in due time :)
2009-03-25 13:07 UTC
florin #
What if I have two constructors with the same number of parameters, how will it decide on the right one?
2010-05-21 10:51 UTC
It doesn't, so this is a situation best avoided. Essentially AutoFixture orders the constructors according to length, but in case of equal lengths it just picks the first one. Which constructor is the first is up to Reflection to decide, so this is not specified.

This sounds like a serious limitation on AutoFixture's part, but in practice I've found that it rarely matters. In the few cases where a SUT has overloaded constructors, it is essential to the design that none of them puts the SUT into an invalid state, so when you need an anonymous instance it shouldn't matter which constructor is used. This is really what AutoFixture is all about.

That said, in AutoFixture 2.0 it will be possible to register custom types that can use other heuristics to pick constructors.
2010-05-21 12:41 UTC

Joining Safewhere

Monday, 23 March 2009 19:03:38 UTC

Today I spent my first working day at Safewhere, where I'll be working as a Senior Software Engineer. In the last couple of years, Safewhere and I have crossed paths a couple of times, and each time they always left me with the impression of a very professional and congenial ISV, so I'm super-excited to be joining!

Expect the occasional Geneva or Federation-related blog post to intermingle with the usual TDD stuff in the future :)


Comments

Congratulations on the new job :-)
2009-03-24 15:14 UTC
Thanks :)
2009-03-25 07:32 UTC
Mark Rønn #
Hi Mark. Great news - congratulations and all the best on your new job :-)
2009-04-15 08:31 UTC

Announcing: AutoFixture

Sunday, 22 March 2009 06:50:54 UTC

It gives me great pleasure to announce my latest project: AutoFixture!

What is AutoFixture?

In essence, AutoFixture is a library that creates Anonymous Variables for you when you write unit tests. The intention is that it should enhance your productivity when you do Test-Driven Development - as it has done mine.

Instead of using mental resources on creating Anonymous Variables, AutoFixture can do it for you. By default, it uses Constrained Non-Determinism, but you can configure it to behave differently if you wish.

Here's a very basic example:

[TestMethod]
public void IntroductoryTest()
{
    // Fixture setup
    Fixture fixture = new Fixture();
 
    int expectedNumber = fixture.CreateAnonymous<int>();
    MyClass sut = fixture.CreateAnonymous<MyClass>();
    // Exercise system
    int result = sut.Echo(expectedNumber);
    // Verify outcome
    Assert.AreEqual<int>(expectedNumber, result, "Echo");
    // Teardown
}

The Fixture class is your main entry point to AutoFixture. You can use it as is, customize it, or derive from it as you please; it makes a great base class for a Fixture Object.

The expectedNumber variable may be an Explicit Expectation, but its value is Anonymous, so instead of coming up with a number ourselves, we can let the CreateAnonymous<T> method do it for us.

This method can create instances of most CLR types as long as they have a public constructor (it uses Reflection), but for many primitive types (like Int32), it has specific, customizable algorithms for creating values using Constrained Non-Determinism.

When creating the SUT, we can also use Fixture as an excellent SUT Factory. Since it will do whatever it can to create an instance of the type you ask for, it is pretty robust if you decide to refactor the SUT's constructor.

The above example only hints at what AutoFixture can do. Since the example is very simple, it may be hard to immediately understand its value, so in future posts I will expand on specific AutoFixture features and principles.

Getting started with AutoFixture is as simple as downloading it from CodePlex and referencing the assembly in Visual Studio.


Comments

Hi Mark,

You could also hook up Pex to AutoFixture (http://research.microsoft.com/pex) and let Pex generate inputs for your tests. Pex provides an API to generate values, similar to your CreateAnonymous method so this should be a piece of cake ;)

Cheers,
Peli
2009-05-01 15:53 UTC
Hi Peli

Thank you for your comment.

Pex is a very interesting piece of technology that I'm very excited about. As I see it, Pex and AutoFixture are complementary technologies. While I've dealt briefly with this topic in the AutoFixture FAQ, it bears repeating here: AutoFixture addresses a different scenario than Pex.

Roughly speaking, Pex is a Quality Assurance (QA) tool - it's main purpose is to break the application, in the time-honored tradition of QA testing. AutoFixture, on the other hand, is a TDD utility library. It's main purpose is to drive development of features along the happy path.

In a full development cycle, I'd use AutoFixture as part of my TDD effort to develop the desired feature. Subsequently, when my API is fairly stable, I'd let Pex loose on it to test its robustness before I ship.

Some day, before version 1.0, I should really let Pex loose on AutoFixture :)
2009-05-01 21:05 UTC

Page 74 of 75

"Our team wholeheartedly endorses Mark. His expert service provides tremendous value."
Hire me!