ploeh blog danish software design

AutoFixture is designed around the 80-20 principle. If your code is well-designed I'd expect a default instance of Fixture to be able to create specimens of your classes without too much trouble. There are cases where AutoFixture needs extra help:

• If a class consumes interfaces a default Fixture will not be able to create it. However, this is easily fixed through one of the AutoMocking Customizations.
• If an API has circular references, Fixture might enter an infinite recursion, but you can easily customize it to cut off one the references.
• Some constructors may only accept arguments that don't fit with the default specimens created by Fixture. There are ways to deal with that as well.

I could keep on listing examples, but let's keep it at that. The key assumption underlying AutoFixture is that these special cases are a relatively small part of the overall API you want to test - preferably much less than 20 percent.

Still, to address those special cases you'll need to customize a Fixture instance in some way, but you also want to keep your test code DRY.

As the popularity of AutoFixture grows, I'm getting more and more glimpses of how people address this challenge: Some derive from Fixture, others create extension methods or other static methods, while others again wrap creation of Fixture instances in Factories or Builders.

There really is no need to do that. AutoFixture has an idiomatic way to encapsulate Customizations. It's called… well… a Customization, which is just another way to say that there's an ICustomization interface that you can implement. This concept corresponds closely to the modularization APIs for several well-known DI Containers. Castle Windsor has Installers, StructureMap has Registries and Autofac has Modules.

The ICustomization interface is simple and has a very gentle learning curve:

public interface ICustomization
{
    void Customize(IFixture fixture);
}

Anything you can do with a Fixture instance you can also do with the IFixture instance passed to the Customize method, so this is the perfect place to encapsulate common Customizations to AutoFixture. Note that the AutoMocking extensions as well as several other optional behaviors for AutoFixture are already defined as Customizations.

Using a Customization is also easy:

var fixture = new Fixture()
    .Customize(new DomainCustomization());

You just need to invoke the Customize method on the Fixture. That's no more difficult than calling a custom Factory or extension method - particularly if you also use a Visual Studio Code Snippet.

When I start a new unit testing project, one of the first things I always do is to create a new ‘default' customization for that project. It usually doesn't take long before I need to tweak it a bit - if nothing else, then for adding the AutoMoqCustomization. To apply separation of concerns I encapsulate each Customization in its own class and compose them with a CompositeCustomization:

public class DomainCustomization : CompositeCustomization
{
    public DomainCustomization()
        : base(
            new AutoMoqCustomization(),
            new FuncCustomization())
    {
    }
}

Whenever I need to make sweeping changes to my Fixtures I can simply modify DomainCustomization or one of the Customizations it composes.

In fact, these days I rarely explicitly create new Fixture instances, but rather encapsulate them in a custom AutoDataAttribute like this:

public class AutoDomainDataAttribute : AutoDataAttribute
{
    public AutoDomainDataAttribute()
        : base(new Fixture()
            .Customize(new DomainCustomization()))
    {
    }
}

This means that I can reuse the DomainCustomization across normal, imperative unit tests as well as the declarative, xUnit.net-powered data theories I normally prefer:

[Theory, AutoDomainData]
public void CanReserveReturnsTrueWhenQuantityIsEqualToRemaining(
    Capacity sut, Guid id)
{
    var result = sut.CanReserve(sut.Remaining, id);
    Assert.True(result);
}

Using Customizations to encapsulate your own specializations makes it easy to compose and manage them in an object-oriented fashion.

A while back I posed the challenge of resolving closed types with MEF. I received some responses, but I also wanted to provide an alternative outline for a solution. In case you don't remember the problem statement, it revolved around using the Managed Extensibility Framework (MEF) to compose classes in those cases where it's impossible to annotate those classes with the MEF attributes. In the given example I want to compose the Mayonnaise class from EggYolk and OliveOil, but all three classes are sealed and cannot be recompiled.

As I describe in my book, a general solution to this type of problem is to create a sort of adapter the exports the closed type via a read-only attribute, like these EggYolkAdapter and MayonnaiseAdapter classes (the OliveOilAdapter looks just like the EggYolkAdapter):

public class EggYolkAdapter
{
    private readonly EggYolk eggYolk;
 
    public EggYolkAdapter()
    {
        this.eggYolk = new EggYolk();
    }
 
    [Export]
    public virtual EggYolk EggYolk
    {
        get { return this.eggYolk; }
    }
}
 
public class MayonnaiseAdapter
{
    private readonly Mayonnaise mayo;
 
    [ImportingConstructor]
    public MayonnaiseAdapter(
        EggYolk yolk, OliveOil oil)
    {
        if (yolk == null)
        {
            throw new ArgumentNullException("yolk");
        }
        if (oil == null)
        {
            throw new ArgumentNullException("oil");
        }
 
        this.mayo = new Mayonnaise(yolk, oil);
    }
 
    [Export]
    public virtual Mayonnaise Mayonnaise
    {
        get { return this.mayo; }
    }
}

Doing it like this is always possible, but if you have a lot of types that you need to compose, it becomes tedious having to define a lot of similar adapters. Fortunately, we can take it a step further and generalize the idea of a MEF adapter to a small set of generic classes.

The EggYolkAdapter can be generalized as follows:

public class MefAdapter<T> where T : new()
{
    private readonly T export;
 
    public MefAdapter()
    {
        this.export = new T();
    }
 
    [Export]
    public virtual T Export
    {
        get { return this.export; }
    }
}

Notice that I've more or less just replaced the EggYolk class with a type argument (T). However, I also had to add the generic new() constraint, which is often quite restrictive. However, to support a type like Mayonnaise, I can create another, similar generic MEF adapter like this:

public class MefAdapter<T1, T2, TResult>
{
    private readonly static Func<T1, T2, TResult> createExport =
        FuncFactory.Create<T1, T2, TResult>();
    private readonly TResult export;
 
    [ImportingConstructor]
    public MefAdapter(T1 arg1, T2 arg2)
    {
        this.export = createExport(arg1, arg2);
    }
 
    [Export]
    public virtual TResult Export
    {
        get { return this.export; }
    }
}

The major difference from the simple MefAdapter<T> is that we need slightly more complicated code to invoke the constructor of TResult, which is expected to take two constructor arguments of types T1 and T2. This work is delegated to a FuncFactory that builds and compiles the appropriate delegate using an expression tree:

internal static Func<T1, T2, TResult>
    Create<T1, T2, TResult>()
{
    var arg1Exp =
        Expression.Parameter(typeof(T1), "arg1");
    var arg2Exp = 
        Expression.Parameter(typeof(T2), "arg2");
 
    var ctorInfo = 
        typeof(TResult).GetConstructor(new[]
        {
            typeof(T1),
            typeof(T2)
        });
    var ctorExp =
        Expression.New(ctorInfo, arg1Exp, arg2Exp);
 
    return Expression.Lambda<Func<T1, T2, TResult>>(
        ctorExp, arg1Exp, arg2Exp).Compile();
}

With a couple of MEF adapters, I can now compose a MEF Catalog almost like I can with a real DI Container, and resolve the Mayonnaise class:

var catalog = new TypeCatalog(
    typeof(MefAdapter<OliveOil>),
    typeof(MefAdapter<EggYolk>),
    typeof(MefAdapter<EggYolk, OliveOil, Mayonnaise>)
    );
var container = new CompositionContainer(catalog);
 
var mayo = container.GetExportedValue<Mayonnaise>();

If you want to change the Creation Policy to NonShared, you can derive from the MefAdapter classes and annotate them with [PartCreationPolicy] attributes.

I'd never voluntarily choose to use MEF like this, but if I was stuck with MEF in a project and had to use it like a DI Container, I'd do something like this.

Comments

The main principle behind the Register Resolve Release pattern is that loosely coupled object graphs should be composed as a single action in the entry point of the application (the Composition Root). For request-based applications (web sites and services), we use a variation where we compose once per request.

It seems to me that a lot of people are apprehensive when they first hear about this concept. It may sound reasonable from an architectural point of view, but isn't it horribly inefficient? A well-known example of such a concern is Jeffrey Palermo's blog post Constructor over-injection anti-pattern. Is it really a good idea to compose a complete object graph in one go? What if we don't need part of the graph, or only need it later? Doesn't it adversely affect response times?

Normally it doesn't, and if it does, there are elegant ways to address the issue.

In the rest of this blog post I will expand on this topic. To keep the discussion as simple as possible, I'll restrict my analysis to object trees instead of full graphs. This is quite a reasonable simplification as we should strive to avoid circular dependencies, but even in the case of full graphs the arguments and techniques put forward below hold.

Consider a simple tree composed of classes from three different assemblies:

All the A classes (blue) are defined in the A assembly, B classes (green) in the B assembly, and the C1 class (red) in the C assembly. In code we create the tree with Constructor Injection like this:

var t =
    new A1(
        new A2(
            new B1(
                new B2()),
            new A3()),
        new C1(
            new B3()));

Given the tree above, we can now address the most common concerns about composing object trees in one go.

Will it be slow? #

Most likely not. Keep in mind that Injection Constructors should be very simple, so not a lot of work is going on during composition. Obviously just creating new object instances takes a bit of time in itself, but we create objects instances all the time in .NET code, and it's often a very fast operation.

Even when using DI Containers, which perform a lot of (necessary) extra work when creating objects, we can create tens of thousand trees per second. Creation of objects simply isn't that big a deal.

But still: what about assembly loading? #

I glossed over an important point in the above argument. While object creation is fast, it sometimes takes a bit of time to load an assembly. The tree above uses classes from three different assemblies, so to create the tree all three assemblies must be loaded.

In many cases that's a performance hit you'll have to take because you need those classes anyway, but sometimes you might be concerned with taking this performance hit too early. However, I make the claim that in the vast majority of cases, this concern is irrelevant.

In this particular context there are two different types of applications: Request-based applications (web) and all the rest (desktop apps, daemons, batch-jobs, etc.).

Request-based applications #

For request-based applications such as web sites and REST services, an object tree must be composed for each request. However, all requests are served by the same AppDomain, so once an assembly is loaded, it sticks around to be available for all subsequent requests. Thus, the first few requests will suffer a performance penalty from having to load all assemblies, but after that there will be no performance impact.

In short, in request-based applications, you can compose object trees with confidence. In only extremely rare cases should you have performance issues from composing the entire tree in one go.

Long-running applications #

For long-running applications the entire object tree must be composed at start-up. For background services such as daemons and batch processes the start-up time probably doesn't matter much, but for desktop applications it can be of great importance.

In some cases the application requires the entire tree to be immediately available, in which case there's not a lot you can do. Still, once all assemblies have been loaded, actually creating the tree will be very fast.

In other cases an entire branch of the tree may not be immediately required. As an example, if the C1 node in the above graph isn't needed right away, we could improve start-up time if we could somehow defer creating that branch, because this would also defer loading of the entire C assembly.

Deferred branches #

Since object creation is fast, the only case where it makes sense to defer loading of a branch is when creation of that branch causes an assembly to be loaded. If we can defer creation of such a branch, we can also defer loading of the assembly, thus improving the time it takes to compose the initial tree.

Imagine that we wish to defer creation of the C1 branch of the above tree. It will prevent the C assembly from being loaded because that assembly is not used in any other place in the tree. However, it will not prevent the B assembly from being loaded, since that assembly is also being used by the A2 node.

Still, in those rare situations where it makes sense to defer creation of a branch, we can make that cut into a part of the infrastructure of the tree. I originally described this technique as a reaction to the above mentioned post by Jeffrey Palermo, but here's a restatement in the current context.

We can defer creating the C1 node by wrapping it in a lazy implementation of the same interface. The C1 node implements an interface called ISolo<IMarker>, so we can wrap it in a Virtual Proxy that defers creation of C1 until it's needed:

public class LazySoloMarker : ISolo<IMarker>
{
    private readonly Lazy<ISolo<IMarker>> lazy;
 
    public LazySoloMarker(Lazy<ISolo<IMarker>> lazy)
    {
        if (lazy == null)
        {
            throw new ArgumentNullException("lazy");
        }
 
        this.lazy = lazy;
    }
 
    #region ISolo<IMarker> Members
 
    public IMarker Item
    {
        get { return this.lazy.Value.Item; }
    }
 
    #endregion
}

This Virtual Proxy takes a Lazy<ISolo<IMarker>> as input and defers to it to implement the members of the interface. This only causes the Value property to be created when it's first accessed - which may be long after the LazySoloMarker instance was created.

The tree can now be composed like this:

var t =
    new A1(
        new A2(
            new B1(
                new B2()),
            new A3()),
        new LazySoloMarker(
            new Lazy<ISolo<IMarker>>(() => new C1(
                new B3()))));

This retains all the original behavior of the original tree, but defers creation of the C1 node until it's needed for the first time.

The bottom line is this: you can compose the entire object graph with confidence. It's not going to be a performance bottleneck.

Update (2013-08-19 08:09 UTC): For a more detailed treatment of this topic, watch my NDC 2013 talk Big Object Graphs Up Front.

Comments

The Constructor Injection design pattern is a extremely useful way to implement loose coupling. It's easy to understand and implement, but sometime perhaps a bit misunderstood.

The pattern itself is easily described through an example:

private readonly ISpecimenBuilder builder;
 
public SpecimenContext(ISpecimenBuilder builder)
{
    if (builder == null)
    {
        throw new ArgumentNullException("builder");
    }
 
    this.builder = builder;
}

The SpecimenContext constructor statically declares that it requires an ISpecimenBuilder instance as an argument. To guarantee that the builder field is an invariant of the class, the constructor contains a Guard Clause before it assigns the builder parameter to the builder field. This pattern can be repeated for each constructor argument.

It's important to understand that when using Constructor Injection the constructor should contain no additional logic.

An Injection Constructor should do no more than receiving the dependencies.

This is simply a rephrasing of Nikola Malovic's 4th law of IoC. There are several reasons for this rule of thumb:

• When we compose applications with Constructor Injection we often create substantial object graphs, and we want to be able to create these graphs as efficiently as possible. This is Nikola's original argument.
• In the odd (and not recommended) cases where you have circular dependencies, the injected dependencies may not yet be fully initialized, so an attempt to invoke their members at that time may result in an exception. This issue is similar to the issue of invoking virtual members from the constructor. Conceptually, an injected dependency is equivalent to a virtual member.
• With Constructor Injection, the constructor's responsibility is to demand and receive the dependencies. Thus, according to the Single Responsibility Principle (SRP), it should not try to do something else as well. Some readers might argue that I'm misusing the SRP here, but I think I'm simply applying the underlying principle in a more granular context.

There's no reason to feel constrained by this rule, as in any case the constructor is an implementation detail. In loosely coupled code, the constructor is not part of the overall application API. When we consider the API at that level, we are still free to design the API as we'd like.

Please notice that this rule is contextual: it applies to Services that use Constructor Injection. Entities and Value Objects tend not to use DI, so their constructors are covered by other rules.

Comments

.NET developers should be familiar with the standard access modifiers (public, protected, internal, private). However, in loosely coupled code we can regard interface implementations as a fifth access modifier. This concept was originally introduced to me by Udi Dahan the only time I've had the pleasure of meeting him. That was many years ago and while I didn't grok it back then, I've subsequently come to appreciate it quite a lot.

Although I can't take credit for the idea, I've never seen it described, and it really deserves to be.

The basic idea is simple:

If a consumer respects the Liskov Substitution Principle (LSP), the only visible members are those belonging to the interface. Thus, the interface represents a dimension of visibility.

As an example, consider this simple interface from AutoFixture:

public interface ISpecimenContext
{
    object Resolve(object request);
}

A well-behaved consumer can only invoke the Resolve method even though an implementation may have additional public members:

public class SpecimenContext : ISpecimenContext
{
    private readonly ISpecimenBuilder builder;
 
    public SpecimenContext(ISpecimenBuilder builder)
    {
        if (builder == null)
        {
            throw new ArgumentNullException("builder");
        }
 
        this.builder = builder;
    }
 
    public ISpecimenBuilder Builder
    {
        get { return this.builder; }
    }
 
    #region ISpecimenContext Members
 
    public object Resolve(object request)
    {
        return this.Builder.Create(request, this);
    }
 
    #endregion
}

Even though the SpecimenContext class defines the Builder property, as well as a public constructor, any consumer respecting the LSP will only see the Resolve method.

In fact, the Builder property on the SpecimenContext class mostly exists to support unit testing because I sometimes need to assert that a given instance of SpecimenContext contains the expected ISpecimenBuilder. This doesn't break encapsulation since the Builder is exposed as a read-only property, and it more importantly doesn't pollute the API.

To support unit testing (and whichever other clients might be interested in the encapsulated ISpecimenBuilder) we have a public property that follows all framework design guidelines. However, it's essentially an implementation detail, so it's not visible via the ISpecimenContext interface.

When writing loosely coupled code, I've increasingly begun to see the interfaces as the real API. Most other (even public) members are pure implementation details. If the members are public, I still demand that they follow the framework design guidelines, but I don't consider them parts of the API. It's a very important distinction.

The interfaces define the bulk of an application's API. Most other types and members are implementation details.

An important corollary is that constructors are implementation details too, since they can never by part of any interfaces.

In that sense we can regard interfaces as a fifth access modifier - perhaps even the most important one.

Comments

In my previous post I described how to customize a Fixture instance to populate lists with items instead of returning empty lists. While it's pretty easy to do so, the drawback is that you have to do it explicitly for every type you want to influence. In this post I will follow up by describing how to enable some general conventions that simply populates all collections that the Fixture resolves.

This post describes a feature that will be available in AutoFixture 2.1. It's not available in AutoFixture 2.0, but is already available in the code repository. Thus, if you can't wait for AutoFixture 2.1 you can download the source and built it.

Instead of having to create multiple customizations for IEnumerable<int>, IList<int>, List<int>, IEnumerable<string>, IList<string>, etc. you can simply enable these general conventions as easy as this:

var fixture = new Fixture()
    .Customize(new MultipleCustomization());

Notice that enabling conventions for populating sequences and lists with ‘many' items is an optional customization that you must explicitly add.

This feature must be explicitly enabled. There are several reasons for that:

• It would be a breaking change if AutoFixture suddenly started to behave like this by default.
• The MultipleCustomization targets not only concrete types such as List<T> and Collection<T>, but also interfaces such as IEnumerable<T>, IList<T> etc. Thus, if you also use AutoFixture as an Auto-Mocking container, I wanted to provide the ability to define which customization takes precedence.

With that simple customization enabled, all requested IEnumerable<T> are now populated. The following will give us a finite, but populated list of integers:

var integers = 
    fixture.CreateAnonymous<IEnumerable<int>>();

This will give us a populated List<int>:

var list = fixture.CreateAnonymous<List<int>>();

This will give us a populated Collection<int>:

var collection = 
    fixture.CreateAnonymous<Collection<int>>();

As implied above, it also handles common list interfaces, so this gives us a populated IList<T>:

var list = fixture.CreateAnonymous<IList<int>>();

The exact number of ‘many' is as always determined by the Fixture's RepeatCount.

As this code is still (at the time of publishing) in preview, I would love to get feedback on this feature.

How do you get AutoFixture to create populated lists or sequences of items? Recently I seem to have been getting this question a lot, and luckily it's quite easy to answer.

Let's first look at the standard AutoFixture behavior and API.

You can ask AutoFixture to create an anonymous List like this:

var list = fixture.CreateAnonymous<List<int>>();

Seen from AutoFixture's point of view, List<int> is just a class like any other. It has a default constructor, so AutoFixture just uses that and returns an instance. You get back an instance, no exceptions are thrown, but the list is empty. What if you'd rather want a populated list?

There are many ways to go about this. A simple, low-level solution is to populate the list after creation:

fixture.AddManyTo(list);

However, you may instead prefer getting a populated list right away. This is also possible, but before we look at how to get there, I'd like to point out a feature that surprisingly few users notice. You can create many anonymous specimens at once:

var integers = fixture.CreateMany<int>();

Armed with this knowledge, as well as the knowledge of how to map types, we can now create this customization to map IEnumerable<int> to CreateMany<int>:

fixture.Register(() => fixture.CreateMany<int>());

The Register method is really a generic method, but since we have type inference, we don't have to write it out. However, since CreateMany<int>() returns IEnumerable<int>, this is the type we register. Thus, every time we subsequently resolve IEnumerable<int>, we will get back a populated sequence.

Getting back to the original List<int> example, we can now customize it to a populated list like this:

fixture.Register(() =>
    fixture.CreateMany<int>().ToList());

Because the ToList() extension method returns List<T>, this call registers List<int> so that we will get back a populated list of integers every time the fixture resolves List<int>.

What about other collection types that don't have a nice LINQ extension method? Personally, I never use Collection<T>, but if you wanted, you could customize it like this:

fixture.Register(() =>
    new Collection<int>(
        fixture.CreateMany<int>().ToList()));

Since Collection<T> has a constructor overload that take IList<T> we can customize the type to use this specific overload and populate it with ‘many' items.

Finally, we can combine all this to map from collection interfaces to populated lists. As an example, we can map from IList<int> to a populated List<int> like this:

fixture.Register<IList<int>>(() => 
    fixture.CreateMany<int>().ToList());

When we use the Register method to map types we can no longer rely on type inference. Instead, we must explicitly register IList<int> against a delegate that creates a populated List<int>. Because List<int> implements IList<int> this compiles. Whenever this fixture instance resolves IList<int> it will create a populated List<int>.

All of this describes what you can do with the strongly typed API available in AutoFixture 2.0. It's easy and very flexible, but the only important drawback is that it's not general. All of the customizations in this post specifically address lists and sequences of integers, but not lists of any other type. What if you would like to expand this sort of behavior to any List<T>, IEnumerable<T> etc?

Stay tuned, because in the next post I will describe how to do that.

During the last couple of weeks I've been very interested in using a Maybe monad with AutoFixture's Kernel code, but although many examples can be found on the internet, they remain samples. Rinat Abdullin and Zack Owens both posted samples, but I particularly like Mike Hadlow's series about Monads in C# because he also explains how to use LINQ with monads such as the Maybe monad.

As I really wanted a Maybe monad for AutoFixture, I first thought about simply implementing it directly in the AutoFixture source. However, I found it too arbitrary to put such a general purpose programming construct into a specific library such as AutoFixture. My next thought was to create a small open source project just for that single purpose, but then I though about the problem a bit more…

The BCL sort of already has a Maybe monad - you just need to recognize it as such.

What is a Maybe monad really? If you really distill it, it's just a type that either contains a value, or doesn't contain a value. In other words, it's a type that represents a particular range: a set with either zero or one items. That's just a special case of a more general range or collection, and we already have LINQ covering those constructs.

Here it is: the Maybe monad from the BLC (encapsulated in a nice extension method):

public static class LightweightMaybe
{
    public static IEnumerable<T> Maybe<T>(this T value)
    {
        return new[] { value };
    }
}

Obviously, this method returns a Maybe with a value, but we can just as easily represent Nothing with an empty array.

With my ‘new' Maybe monad, I can now write code like this (where request is a System.Object instance):

return (from t in request.Maybe().OfType<Type>()
        let typeArguments = t.GetGenericArguments()
        where typeArguments.Length == 1
        && typeof(IList<>)
            == t.GetGenericTypeDefinition()
        select context.Resolve(typeof(List<>)
            .MakeGenericType(typeArguments)))
        .DefaultIfEmpty(new NoSpecimen(request))
        .SingleOrDefault();

You may think that this looks dense, but before that the code looked like this:

var type = request as Type;
if (type == null)
{
    return new NoSpecimen(request);
}
 
var typeArguments = type.GetGenericArguments();
if (typeArguments.Length != 1)
{
    return new NoSpecimen(request);
}
 
if (typeof(IList<>) != 
    type.GetGenericTypeDefinition())
{
    return new NoSpecimen(request);
}
 
return context.Resolve(typeof(List<>)
    .MakeGenericType(typeArguments));

Notice that in this more traditional approach involving Guard Clauses, I have to construct a new NoSpecimen object in three different places, thus violating the DRY principle. I like not having all those if/return blocks in the code.

Comments

Recently I spent a couple of days with Thomas Jespersen who's working towards a launch of spiir.dk - on Windows Azure. The reason I got to talk to him was to see if I could help with some performance issues he had with Azure Table Storage.

The scenario is really simple: the application needs to load all of a user's bank transactions into memory to enable pretty advanced sorting and filtering. That sounds like a lot, but really isn't more than approximately 200 kB of data retrieved through a single query - so: there are no 1+N problems in play here, but even so it originally took more than two seconds. That's a bit long to wait before you can even start rendering a web page.

By tweaking his partitioning strategy and using parallel queries, Thomas managed to bring down the data retrieval time to approximately one second. Although stress testing indicated that this duration was very stable, even under load, it is still too slow. So we met to see what could be done.

Thomas had done a great job tweaking the query, so I couldn't really suggest some sort of secret API that would make it run significantly faster. Basically, we have to deal with Azure storage being based on REST and that there are a lot of things about run-time behavior we cannot control. Apart from designing a proper partitioning strategy, we can't add indexes to Azure Table Storage.

It was time to take a different approach.

As far as I can tell, Windows Azure is designed to be very scalable. However, just because scalability implies that you can handle an insane amount of work within acceptable time frames, it doesn't mean that you can extrapolate it to mean that under a light load, everything will be lightning fast. That's not the case at all.

Scalability means that performance characteristics remain stable from light to heavy load.

Consequently this means that if performance is adequate under heavy load, it will also be adequate under a light load. Azure Storage is first and foremost designed to be scalable, and as a second priority, as fast as possible.

As Thomas discovered, Azure Table Storage isn't particularly fast.

It may be a masochistic side of me that I'm not otherwise aware of, but I actually appreciate that. It makes us reassess our most basic assumptions.

The data that Thomas needs to read isn't particularly dynamic, so what if we take a snapshot of it? In short, we loaded all of a user's data into memory and serialized it to Azure Blob Storage.

Loading the same data from a binary serialized Blob took only 1/6 of the time it did to load it from Table Storage.

As it turns out, Thomas doesn't even need all the columns from the Table to populate the view, so we could even make the serialized Blob smaller yet.

At this point, however, we now have two representations of the same data: The original data in Table Storage, and a persistent cache in Blob Storage. The remaining challenge is to figure out how to keep these in sync.

This may seem like a hack, but is really represents a paradigm shift. Letting go of ACID opens up a lot of new opportunities.

Actually, I spend most of the next day trying to convince Thomas that CQRS would be the best approach, or that we could at least pick up some of the techniques from asynchronous, messaging based architectures, but that's another story.

The morale here is that on Azure, things may be slower than you are used to, but storage is (relatively) cheap, so denormalization can save you a lot of execution time.

Comments

A week ago I concluded Microsoft Denmark's 2010 .NET Community Christmas Calendar with a challenge about resolving closed types with MEF. As 2011 came around, the deadline ended, so it's now time to pick the winner.

I didn't get a lot of entries, which can be interpreted in at least one (or more) of the following ways:

• The challenge was too difficult
• The challenge wasn't interesting
• The prize wasn't attractive
• People had other things to do during the holidays

Whatever the reason, it made my task of picking a winner that much easier. The best Danish entry came from Daniel Volder Guarnieri who cheated a bit by partially hard-coding the composition of Mayonnaise into the ContainerBuilder. As I wrote in the original challenge, there are many ways to tackle the challenge, and one was to take the unit tests very literally :)

However, honorable mention must go to Boyan Mihaylov who participated just for the honor. He took a more general approach similar to Fluent MEF. This involves implementing a completely new ComposablePartCatalog with associated ComposablePartDefinition and ComposablePart implementations - not a trivial undertaking.

Kudos to Boyan and congratulations to Daniel. My thanks for your submissions, and a happy new year to all my readers!

Comments