Loose Coupling and the Big Picture

Thursday, 02 February 2012 20:37:40 UTC

A common criticism of loosely coupled code is that it's harder to understand. How do you see the big picture of an application when loose coupling is everywhere? When the entire code base has been programmed against interfaces instead of concrete classes, how do we understand how the objects are wired and how they interact?

In this post, I'll provide answers on various levels, from high-level architecture over object-oriented principles to more nitty-gritty code. Before I do that, however, I'd like to pose a set of questions you should always be prepared to answer.

Mu #

My first reaction to that sort of question is: you say loosely coupled code is harder to understand. Harder than what?

If we are talking about a non-trivial application, odds are that it's going to take some time to understand the code base - whether or not it's loosely coupled. Agreed: understanding a loosely coupled code base takes some work, but so does understanding a tightly coupled code base. The question is whether it's harder to understand a loosely coupled code base?

Imagine that I'm having a discussion about this subject with Mary Rowan from my book.

Mary: “Loosely coupled code is harder to understand.”

Me: “Why do you think that is?”

Mary: “It's very hard to navigate the code base because I always end up at an interface.”

Me: “Why is that a problem?”

Mary: “Because I don't know what the interface does.”

At this point I'm very tempted to answer Mu. An interfaces doesn't do anything - that's the whole point of it. According to the Liskov Substitution Principle (LSP), a consumer shouldn't have to care about what happens on the other side of the interface.

However, developers used to tightly coupled code aren't used to think about services in this way. They are used to navigate the code base from consumer to service to understand how the two of them interact, and I will gladly admit this: in principle, that's impossible to do in a loosely coupled code base. I'll return to this subject in a little while, but first I want to discuss some strategies for understanding a loosely coupled code base.

Architecture and Documentation #

Yes: documentation. Don't dismiss it. While I agree with Uncle Bob and like-minded spirits that the code is the documentation, a two-page document that outlines the Big Picture might save you from many hours of code archeology.

The typical agile mindset is to minimize documentation because it tends to lose touch with the code base, but even so, it should be possible to maintain a two-page high-level document so that it stays up to date. Consider the alternative: if you have so much architectural churn that even a two-page overview regularly falls behind, then you're probably having a greater problem than understanding your loosely coupled code base.

Maintaining such a document isn't adverse to the agile spirit. You'll find the same advice in Lean Architecture (p. 127). Don't underestimate the value of such a document.

See the Forest Instead of the Trees #

Understanding a loosely coupled code base typically tends to require a shift of mindset.

Recall my discussion with Mary Rowan. The criticism of loose coupling is that it's difficult to understand which collaborators are being invoked. A developer like Mary Rowan is used to learn a code base by understanding all the myriad concrete details of it. In effect, while there may be ‘classes' around, there are no abstractions in place. In order to understand the behavior of a user interface element, it's necessary to also understand what's happening in the database - and everything in between.

A loosely coupled code base shouldn't be like that.

The entire purpose of loose coupling is that we should be able to reason about a part (a ‘unit', if you will) without understanding the whole.

In a tightly coupled code base, it's often impossible to see the forest for the trees. Although we developers are good at relating to details, a tightly coupled code base requires us to be able to contain the entire code base in our heads in order to understand it. As the size of the code base grows, this becomes increasingly difficult.

In a loosely coupled code base, on the other hand, it should be possible to understand smaller parts in isolation. However, I purposely wrote “should be”, because that's not always the case. Often, a so-called “loosely coupled” code base violates basic principles of object-oriented design.


The criticism that it's hard to see “what's on the other side of an interface” is, in my opinion, central. It betrays a mindset which is still tightly coupled.

In many code bases there's often a single implementation of a given interface, so developers can be forgiven if they think about an interface as only a piece of friction that prevents them from reaching the concrete class on the other side. However, if that's the case with most of the interfaces in a code base, it signifies a violation of the Reused Abstractions Principle (RAP) more than it signifies loose coupling.

Jim Cooper, a reader of my book, put it quite eloquently on the book's forum:

“So many people think that using an interface magically decouples their code. It doesn't. It only changes the nature of the coupling. If you don't believe that, try changing a method signature in an interface - none of the code containing method references or any of the implementing classes will compile. I call that coupled”

Refactoring tools aside, I completely agree with this statement. The RAP is a test we can use to verify whether or not an interface is truly reusable - what better test is there than to actually reuse your interfaces?

The corollary of this discussion is that if a code base is massively violating the RAP then it's going to be hard to understand. It has all the disadvantages of loose coupling with few of the benefits. If that's the case, you would gain more benefit from making it either more tightly coupled or truly loosely coupled.

What does “truly loosely coupled” mean?


According to the LSP a consumer must not concern itself with “what's on the other side of the interface”. It should be possible to replace any implementation with any other implementation of the same interface without changing the correctness of the program.

This is why I previously said that in a truly loosely coupled code base, it isn't ‘hard' to understand “what's on the other side of the interface” - it's impossible. At design-time, there's nothing ‘behind' the interface. The interface is what you are programming against. It's all there is.

Mary has been listening to all of this, and now she protests:

Mary: “At run-time, there's going to be a concrete class behind the interface.”

Me (being annoyingly pedantic): “Not quite. There's going to be an instance of a concrete class which the consumer invokes via the interface it implements.”

Mary: “Yes, but I still need to know which concrete class is going to be invoked.”

Me: “Why?”

Mary: “Because otherwise I don't know what's going to happen when I invoke the method.”

This type of answer often betrays a much more fundamental problem in a code base.


Now we are getting into the nitty-gritty details of class design. What would you expect that the following method does?

public List<Order> GetOpenOrders(Customer customer)

The method name indicates that it gets open orders, and the signature seems to back it up. A single database query might be involved, since this looks a lot like a read-operation. A quick glance at the implementation seems to verify that first impression:

public List<Order> GetOpenOrders(Customer customer)
    var orders = GetOrders(customer);
    return (from o in orders
            where o.Status == OrderStatus.Open
            select o).ToList();

Is it safe to assume that this is a side-effect-free method call? As it turns out, this is far from the case in this particular code base:

private List<Order> GetOrders(Customer customer)
    var gw = new CustomerGateway(this.ConnectionString);
    var orders = gw.GetOrders(customer);
    FixCustomer(gw, orders, customer);
    return orders;

The devil is in the details. What does AuditOrders do? And what does FixCustomer do? One method at a time:

private void AuditOrders(List<Order> orders)
    var user = Thread.CurrentPrincipal.Identity.ToString();
    var gw = new OrderGateway(this.ConnectionString);
    foreach (var o in orders)
        var clone = o.Clone();
        var ar = new AuditRecord
            Time = DateTime.Now,
            User = user
        // We don't want the consumer to see the audit trail.

OK, it turns out that this method actually makes a copy of each and every order and updates that copy, writing it back to the database in order to leave behind an audit trail. It also mutates each order before returning to the caller. Not only does this method result in an unexpected N+1 problem, it also mutates its input, and perhaps even more surprising, it's leaving the system in a state where the in-memory object is different from the database. This could lead to some pretty interesting bugs.

Then what happens in the FixCustomer method? This:

// Fixes the customer status field if there were orders
// added directly through the database.
private static void FixCustomer(CustomerGateway gw,
    List<Order> orders, Customer customer)
    var total = orders.Sum(o => o.Total);
    if (customer.Status != CustomerStatus.Preferred
        && total > PreferredThreshold)
        customer.Status = CustomerStatus.Preferred;

Another potential database write operation, as it turns out - complete with an apology. Now that we've learned all about the details of the code, even the GetOpenOrders method is beginning to look suspect. The GetOrders method returns all orders, with the side effect that all orders were audited as having been read by the user, but the GetOpenOrders filters the output. In the end, it turns out that we can't even trust the audit trail.

While I must apologize for this contrived example of a Transaction Script, it's clear that when code looks like that, it's no wonder why developers think that it's necessary to contain the entire code base in their heads. When this is the case, interfaces are only in the way.

However, this is not the fault of loose coupling, but rather a failure to adhere to the very fundamental principle of Command-Query Separation (CQS). You should be able to tell from the method signature alone whether invoking the method will or will not have side-effects. This is one of the key messages from Clean Code: the method name and signature is an abstraction. You should be able to reason about the behavior of the method from its declaration. You shouldn't have to read the code to get an idea about what it does.

Abstractions always hide details. Method declarations do too. The point is that you should be able to read just the method declaration in order to gain a basic understanding of what's going on. You can always return to the method's code later in order to understand detail, but reading the method declaration alone should provide the Big Picture.

Strictly adhering to CQS goes a long way in enabling you to understand a loosely coupled code base. If you can reason about methods at a high level, you don't need to see “the other side of the interface” in order to understand the Big Picture.

Stack Traces #

Still, even in a loosely coupled code base with great test coverage, integration issues arise. While each class works fine in isolation, when you integrate them, sometimes the interactions between them cause errors. This is often because of incorrect assumptions about the collaborators, which often indicates that the LSP was somehow violated.

To understand why such errors occur, we need to understand which concrete classes are interacting. How do we do that in a loosely coupled code base?

That's actually easy: look at the stack trace from your error report. If your error report doesn't include a stack trace, make sure that it's going to do that in the future.

The stack trace is one of the most important troubleshooting tools in a loosely coupled code base, because it's going to tell you exactly which classes were interacting when an exception was thrown.

Furthermore, if the code base also adheres to the Single Responsibility Principle and the ideals from Clean Code, each method should be very small (under 15 lines of code). If that's the case, you can often understand the exact nature of the error from the stack trace and the error message alone. It shouldn't even be necessary to attach a debugger to understand the bug, but in a pinch, you can still do that.

Tooling #

Returning to the original question, I often hear people advocating tools such as IDE add-ins which support navigation across interfaces. Such tools might provide a menu option which enables you to “go to implementation”. At this point it should be clear that such a tool is mainly going to be helpful in code bases that violate the RAP.

(I'm not naming any particular tools here because I'm not interested in turning this post into a discussion about the merits of various specific tools.)

Conclusion #

It's the responsibility of the loosely coupled code base to make sure that it's easy to understand the Big Picture and that it's easy to work with. In the end, that responsibility falls on the developers who write the code - not the developer who's trying to understand it.


Mark, another great blog post. What do you think about inroducing interfaces to enable unit testing. I we take into the consideration only the production code we could have a 1:1 mapping between interfaces and abstractions and thus violating the RAP principle. But from the point of view of unit testing it's very handy. Do you think that unit testing justify the RAP violation ?

2012-02-02 22:56 UTC
We could also ask ourselves "Why do we use abstraction in our code?" Abstraction is a mean to isolate implementations like method bodies, so when we're implementing a MethodA which is dependend on some interface, from the MethodA perspective, we don't care about the implementation of that interface. In practice, this works pretty well and abstractions bring welcomed flexibility to make our code more refactorable, which leads to more maintainable program. The key is that if you need to know what implementation resides behind an abstraction, you are in trouble: not only there might be multiple different implementations but also, some of these implementations might be created or refactored in the future.
2012-02-02 23:08 UTC
If you introduce interfaces to enable testability then you are not breaking the RAP IMHO. You can't really say that you have a 1:1 mapping as both the production code and your unit tests are consumers of your API.

You can argue that the most important consumer is the production code and I definitely agree with you! But let's not underestimate the role and the importance of the automated test as this is the only consumer that should drive the design of your API.
2012-02-03 16:30 UTC
Thomas, Javi

I'm a big proponent of the concept that, with TDD, unit tests are actually the first consumer of your code, and the final application only a close second. As such, it may seem compelling to state that you're never breaking the RAP if you do TDD. However, as a knee-jerk reaction, I just don't buy that argument, which made me think why that is the case...

I haven't thought this completely through yet, but I think many (not all) unit tests pose a special case. A very specialized Test Double isn't really a piece of reuse as much as it's a simulation of the production environment. Add into this mix any dynamic mock library, and you have a tool where you can simulate anything.

However, simulation isn't the same as reuse.

I think a better test would be if you can write a robust and maintainable Fake. If you can do that, the abstraction is most likely reuseable. If not, it probably isn't.
2012-02-03 19:08 UTC
You're very exigent Mark ;) Puting aside TDD and Unit testing. How would you achieve loosely coupling between assemblies without an interface ? For example an MVC application, I prefer to be dependant on ISmthRepository in my controller than to reference the implementation of ISmthRepository which could be SqlSmthRepository. Even if I know that there will be only 1:1 implementation ?


2012-02-05 09:33 UTC
I'm not saying that you can't program to interfaces, but I'm saying that if you can't reuse those interfaces, it's time to take a good hard look at them. So if you know there's only ever going to be one implementation of ISmthRepository, what does that tell you about that interface?

In any case, please refer back to the original definition of the RAP. It doesn't say that you aren't allowed to program against 1:1 interfaces - it just states that as long as it remains a 1:1 interface, you haven't proved that it's a generalization. Until that happens, it should be treated as suspect.

However, the RAP states that we should discover generalizations after the fact, which implies that we'd always have to go through a stage where we have 1:1 interfaces. As part of the Refactoring step of Red/Green/Refactor, it should be our responsibility to merge interfaces, just as it's our responsibility to remove code duplication.
2012-02-05 10:36 UTC
I would like some clarification as some of these principles seem contradictory in nature. Allow me to explain:
1. RAP says that using an interface that has only one implementation is unnecessary.
2. Dependency inversion principle states that both client and service should depend on an abstraction.
3. Tight coupling is discouraged and often defined as depending on a class (i.e. "newing" up a class for use).

So in order to depend on an abstraction (I understand that "abstraction" does not necessarily mean interface all of the time), you need to create an interface and program to it. If you create an interface for this purpose but it only has one implementation than it is suspect under RAP. I understand also that RAP also refers to pointless interfaces such as "IPerson" that has a Person class as an implementation or "IProduct" that has one Product implementation.

But how about a controller that needs a business service or a business service that needs a data service? I find it easier to build a business service in isolation from a data service or controller so I create an interface for the data services I need but don't create implementations. Then I just mock the interfaces to make sure that my business service works through unit testing. Then I move on to the next layer, making sure that the services then implement the interface defined by the business layer. Thoughts? Opinions?
2012-02-05 19:11 UTC

Remember that with TDD we should move through the three phases of Red, Green and Refactor. During the Refactoring phase we typically look towards eliminating duplication. We are (or at least should be) used to do this for code duplication. The only thing the RAP states is that we should extend that effort towards our interface designs.

Please also refer to the other comment on this page.

2012-02-05 19:38 UTC
First of all, great post!

I think one thing to consider in the whole loose coupling debate is granularity.
Not too many people talk about granularity, and without that aspect, I think it is impossible to really have enough context to say whether something is too tightly or loosely coupled. I wrote about the idea here: http://simpleprogrammer.com/2010/11/09/back-to-basics-cohesion-and-coupling-part-2/

Essentially what I am saying is that some things should be coupled. We don't want to create unneeded abstractions, because they introduce complexity. The example I use is Enterprise FizzBuzz. At the same time, we should be striving to build the seams at the points of change which should align in a well designed system with responsibility.

This is definitely a great topic though. Could talk about it all day.
2012-03-02 14:44 UTC

SOLID is Append-only

Tuesday, 03 January 2012 14:43:47 UTC

SOLID is a set of principles that, if applied consistently, has some surprising effect on code. In a previous post I provided a sketch of what it means to meticulously apply the Single Responsibility Principle. In this article I will describe what happens when you follow the Open/Closed Principle (OCP) to its logical conclusion.

In case a refresher is required, the OCP states that a class should be open for extension, but closed for modification. It seems to me that people often forget the second part. What does it mean?

It means that once implemented, you shouldn't touch that piece of code ever again (unless you need to correct a bug).

Then how can new functionality be added to a code base? This is still possible through either inheritance or polymorphic recomposition. Since the L in SOLID signifies the Liskov Substitution Principle, SOLID code tends to be based on loosely coupled code composed into an application through copious use of interfaces - basically, Strategies injected into other Strategies and so on (also due to Dependency Inversion Principle). In order to add functionality, you can create new implementations of these interfaces and redefine the application's Composition Root. Perhaps you'd be wrapping existing functionality in a Decorator or adding it to a Composite.

Once in a while, you'll stop using an old implementation of an interface. Should you then delete this implementation? What would be the point? At a certain point in time, this implementation was valuable. Maybe it will become valuable again. Leaving it as an potential building block seems a better choice.

Thus, if we think about working with code as a CRUD endeavor, SOLID code can be Created and Read, but never Updated or Deleted. In other words, true SOLID code is append-only code.

Example: Changing AutoFixture's Number Generation Algorithm #

In early 2011 an issue was reported for AutoFixture: Anonymous numbers were created in monotonically increasing sequences, but with separate sequences for each number type:

integers: 1, 2, 3, 4, 5, …

decimals: 1.0, 2.0, 3.0, 4.0, 5.0, …

and so on. However, the person reporting the issue thought it made more sense if all numbers shared a single sequence. After thinking about it a little while, I agreed.

Because the AutoFixture code base is fairly SOLID we decided to leave the old implementations in place and implement the new behavior in new classes.

The old behavior was composed from a set of ISpecimenBuilders. As an example, integers were generated by this class:

public class Int32SequenceGenerator : ISpecimenBuilder
    private int i;
    public int CreateAnonymous()
        return Interlocked.Increment(ref this.i);
    public object Create(object request,
        ISpecimenContext context)
        if (request != typeof(int))
            return new NoSpecimen(request);
        return this.CreateAnonymous();

Similar implementations generated decimals, floats, doubles, etc. Instead of modifying any of these classes, we left them in the code base and created a new ISpecimenBuilder that generates all numbers from a single sequence:

public class NumericSequenceGenerator : ISpecimenBuilder
    private int value;
    public object Create(object request,
        ISpecimenContext context)
        var type = request as Type;
        if (type == null)
            return new NoSpecimen(request);
        return this.CreateNumericSpecimen(type);
    private object CreateNumericSpecimen(Type request)
        var typeCode = Type.GetTypeCode(request);
        switch (typeCode)
            case TypeCode.Byte:
                return (byte)this.GetNextNumber();
            case TypeCode.Decimal:
                return (decimal)this.GetNextNumber();
            case TypeCode.Double:
                return (double)this.GetNextNumber();
            case TypeCode.Int16:
                return (short)this.GetNextNumber();
            case TypeCode.Int32:
                return this.GetNextNumber();
            case TypeCode.Int64:
                return (long)this.GetNextNumber();
            case TypeCode.SByte:
                return (sbyte)this.GetNextNumber();
            case TypeCode.Single:
                return (float)this.GetNextNumber();
            case TypeCode.UInt16:
                return (ushort)this.GetNextNumber();
            case TypeCode.UInt32:
                return (uint)this.GetNextNumber();
            case TypeCode.UInt64:
                return (ulong)this.GetNextNumber();
                return new NoSpecimen(request);
    private int GetNextNumber()
        return Interlocked.Increment(ref this.value);

Adding a new class in itself has no effect, so in order to recompose the default behavior of AutoFixture, we changed a class called DefaultPrimitiveBuilders by removing the old ISpecimenBuilders like Int32SequenceGenerator and instead adding NumericSequenceGenerator:

yield return new StringGenerator(() => 
yield return new ConstrainedStringGenerator();
yield return new StringSeedRelay();
yield return new NumericSequenceGenerator();
yield return new CharSequenceGenerator();
yield return new RangedNumberGenerator();
// even more builders...

NumericSequenceGenerator is the fourth class being yielded here. Before we added NumericSequenceGenerator, this class instead yielded Int32SequenceGenerator and similar classes. These were removed.

The DefaultPrimitiveBuilders class is part of AutoFixture's default Facade and is the closest we get to a Composition Root for the library. Recomposing this Facade enabled us to change the behavior of AutoFixture without modifying (other) existing classes.

As Enrico (who implemented this change) points out, the beauty is that the previous behavior is still in the box, and all it takes is a single method call to bring it back:

var fixture = new Fixture().Customize(
    new NumericSequencePerTypeCustomization());

The only class we had to modify was the DefaultPrimitiveBuilders, which is where the object graph is composed. In applications this corresponds to the Composition Root, so even in the face of SOLID code, you still need to modify the Composition Root in order to recompose the application. However, use of a good DI Container and a strong set of conventions can do much to minimize the required editing of such a class.

SOLID versus Refactoring #

SOLID is a goal I strive towards in the way I write code and design APIs, but I don't think I've ever written a significant code base which is perfectly SOLID. While I consider AutoFixture a ‘fairly' SOLID code base, it's not perfect, and I'm currently performing some design work in order to change some abstractions for version 3.0. This will require changing some of the existing types and thereby violating the OCP.

It's worth noting that as long as you can stick with the OCP you can avoid introducing breaking changes. A breaking change is also an OCP violation, so adhering to the OCP is more than just an academic exercise - particularly if you write reusable libraries.

Still, while none of my code is perfect and I occasionally have to refactor, I don't refactor much. By definition, refactoring means violating the OCP, and while I have nothing against refactoring code when it's required, I much prefer putting myself in a situation where it's rarely necessary in the first place.

I've often been derided for my lack of use of Resharper. When replying that I have little use for Resharper because I write SOLID code and thus don't do much refactoring, I've been ridiculed for being totally clueless. People don't realize the intimate relationship between SOLID and refactoring. I hope this post has helped highlight that connection.


Jon Wingfield
I've found that it's very difficult to accomplish OCP in practice, mostly because of evolutionary design. An example is having the foresight to know when the cohesion/coupling barrier has been crossed. Also, when one gains greater insight into a domain, refactoring is necessary (even crucial). but this violates OCP as well. I find that my designs are pretty crappy up front but evolve as patterns emerge. I prefer this to up-front engineering (except in some cases), because it yields a code base that is cohesive when appropriate, but also decoupled when appropriate.
2012-01-03 16:09 UTC
Agreed, OCP is hard.

However, adhering to OCP doesn't indicate that you have to do BDUF. What it means is that once you get a 'rush of insight' (as Domain-Driven Design puts it) you don't modify existing classes. Instead, you introduce new classes to follow the new model.

This may seem wasteful, but due to the very fine-grained nature of SOLID code, it means that those classes that follow the old model (that you've just realized can be improved) are basically 'wrong' because they model the domain in the 'wrong' way. Re-implementing that part of the application's behavior while leaving the old code in place is typically more efficient because it's only going to be a very small part of the code base (again due to the granularity) and because you can do it in micro-iterations since you're not changing anything. Thus, dangerous 'big-bang' refactorings are avoided.

In any case, I never said that SOLID is easy. What I'm saying is that SOLID code has certain characteristics, and if a code base doesn't exhibit those characteristics, it's not (perfectly) SOLID. In reality, I expect that there are very few code bases that can live up to what is essentially an ideal more than a practically attainable goal.
2012-01-03 16:42 UTC
Jon Wingfield
Not trying to spam your blog, but maybe OCP is better viewed as a means for enhancement, whereas refactoring is intended to improve the existing design. Sometimes when adding functionality, we realize that the existing design doesn't accomodate change very well, and thus we refactor (hopefully separately from implementing new functionality). I'm still a little uneasy about applying OCP when fixing bugs and especially design issues (which you already alleviated to).
2012-01-03 17:33 UTC

Irrelevant of my association with ReSharper, I think that refactoring (be it with or without tools) and SOLID design are not mutually exclusive. You are basing your argument mostly on the premise that we get things right the first and that is not always the case. Test Driven Development in itself for instance is about evolving design.

As for ReSharper not being needed (or replace ReSharper with any other enhancing tool), I find it kind of amusing because it seems that there is some imaginary line that developers draw whereby what's included in Visual Studio is sufficient when it comes to refactoring. Everything else is superflous. That is of course until the next version of Visual Studio includes it. And that's if we think about these types of tools as refactoring only, which is so far from the truth.

Btw, switch statement violates OCP and yes it doesn't change until it does change. I'd add that normally when I violate OCP I try and make sure the tests are in pace to let me know if something breaks.
2012-01-03 17:55 UTC
Jon, that's well put.

When it comes to fixing bugs, the OCP specifically states that it's OK to modify existing code, so you shouldn't be uneasy about that.
2012-01-03 19:32 UTC
Hadi, I'd never claim that it's possible to get things right on the first attempt. Looking at AutoFixture (again), I had to do a lot of refactoring and redesign to arrive at version 2.0, which is 'fairly' SOLID. Still, I have more changes in store for version 3.0, which indicates to me that it's still not SOLID - although it's vastly better than before.

Still, instead of refactoring, sometimes it makes more sense to leave the old stuff to atrophy and build the new, better API up around it. That's basically the Strangler pattern applied at the code level.

That said, there are some of the refactorings that ReSharper has that I'd love to have in my IDE. It's just that I think that already VS is too slow and heavy on my machine - even without ReSharper...
2012-01-03 19:42 UTC
You could build a whole set of tools around append-only programming; version control in particular would be a piece of cake. An editor that let you use a class or function as a starting point and then save an edited version as a new class or function (without affecting the existing version) would be quite helpful.

I tried an extremely SOLID design for a prototype recently, with very few stable dependencies and leaning on the container for almost everything. I didn't quite adhere to OCP as you've described it here, but in retrospect it would have almost eliminated regressions. There was a lot of pushback to the design as soon as we got another developer on (just before we threw away the prototype), though.

The usual complaints about being unable to see the big picture (due to container registrations being made mostly linearly) came through, and my choice to compose functions rather than objects certainly didn't help as it resulted in some quite foreign-looking code. I think tooling could have helped, but we've decided to stick to KISS and stabilise more dependencies for the upcoming releases.

On the subject of tooling, I think something that's missing from DI tooling is a graphical designer containing a tree view of what your container will resolve at runtime, with markers for missing dependencies and such. A "DIML" file, perhaps, that generates a .diml.cs or .diml.vb when saved. Then you could have a find-and-replace-style feature for replacing dependencies, respecting OCP.
2012-01-03 21:53 UTC
Andreas Triesch
This might seem a bit like a newbie question (well, with regards to SOLID I am one) - does that mean one should mostly use 'protected virtual' methods as opposed to 'private', in case I need something just a little different? Or does the notion I might need to do so in a particular case rather imply my class might be too large (violating SRP) and I should try to achieve flexibility by breaking it up and composing my logic via DI instead of using inheritance?
2012-01-03 22:11 UTC
Let me support you, Mark, in not having much use for ReSharper.

Event though I´m using it, I´m not using it much for refactoring. Out of all the refactorings I use maybe just 2-3 (rename, extract method, move class to separate file).

My main use for ReSharper is as a test runner.

So I agree: ReSharper is a tool for developers wrestling with tons of legacy code they need to refactor. But if your code base is clean... then the need for larger rearrangements is rare. "Refactoring to deeper insight" sometimes requires such rearrangements. But this too need not be that hard, if the functional units are fine grained.
2012-01-04 10:15 UTC
Jeff, I think that the use of a DI Container and application of the OCP are perpendicular concerns.

Andreas, the 'old' definition of OCP (Meyer's) is to use inheritance to extend an existing class. However, when we favor composition over inheritance, the default way to extend a class is to apply a Decorator to it.

Ralf, I think you've nailed it. The reason why I've never felt much need for ReSharper is because I avoid legacy code like the plague. In fact, I've turned down job offers (that payed better than anything I've ever received) because it involved dealing with too much legacy code. If I were ever to deal substantially with legacy code, I might very well install ReSharper.
2012-01-04 18:30 UTC
Mark, you're right. I went off on a bit of a tangent!

I guess the only relation that I was riffing off is that a great tool for writing and composing SOLID code would help with both.
2012-01-04 19:32 UTC
You said that you don't refactor often. Does that mean that you don't practice TDD? As I understand it, refactoring is an essential step in each TDD cycle.
Refactoring aside, it seems to me that TDD practice makes you violate OCP since you start with the simplest implementation and keep improving it (hence changing existing code) to make new tests pass.
2012-03-17 08:51 UTC
Payman, I do practice TDD, and I do refactor regularly as part of the Red/Green/Refactor cycle.

What I meant (but perhaps did not explicitly state) was that once a piece of code is released to production, it changes status. That kind of code I don't often refactor.
2012-03-18 14:02 UTC

Testing Container Configurations

Wednesday, 21 December 2011 13:25:32 UTC

Here's a question I often get:

“Should I test my DI Container configuration?”

The motivation for asking mostly seems to be that people want to know whether or not their applications are correctly wired. That makes sense.

A related question I also often get is whether or not a particular container has a self-test feature? In this post I'll attempt to answer both questions.

Container Self-testing #

Some DI Containers have a method you can invoke to make it perform a consistency check on itself. As an example, StructureMap has the AssertConfigurationIsValid method that, according to documentation does “a full environment test of the configuration of [the] container.” It will “try to create every configured instance [...]”

Calling the method is really easy:


Such a self-test can often be an expensive operation (not only for StructureMap, but in general) because it's basically attempting to create an instance of each and every Service registered in the container. If the configuration is large, it's going to take some time, but it's still going to be faster than performing a manual test of the application.

Two questions remain: Is it worth invoking this method? Why don't all containers have such a method?

The quick answer is that such a method is close to worthless, which also explains why many containers don't include one.

To understand the answer, consider the set of all components contained in the container in this figure:

The container contains the set of components IFoo, IBar, IBaz, Foo, Bar, Baz, and Qux so a self-test will try to create a single instance of each of these seven types. If all seven instances can be created, the test succeeds.

All this accomplishes is to verify that the configuration is internally consistent. Even so, an application could require instances of the ICorge, Corge or Grault types which are completely external to the configuration, in which case resolution would fail.

Even more subtly, resolution would also fail whenever the container is queried for an instance of IQux, since this interface isn't part of the configuration, even though it's related to the concrete Qux type which is registered in the container. A self-test only verifies that the concrete Qux class can be resolved, but it never attempts to create an instance of the IQux interface.

In short, the fact that a container's configuration is internally consistent doesn't guarantee that all services required by an application can be served.

Still, you may think that at least a self-test can constitute an early warning system: if the self-test fails, surely it must mean that the configuration is invalid? Unfortunately, that's not true either.

If a container is being configured using Auto-registration/Convention over Configuration to scan one or more assemblies and register certain types contained within, chances are that ‘too many' types will end up being registered - particularly if one or more of these assemblies are reusable libraries (as opposed to application-specific assemblies). Often, the number of redundant types added is negligible, but they may make the configuration internally inconsistent. However, if the inconsistency only affects the redundant types, it doesn't matter. The container will still be able to resolve everything the current application requires.

Thus, a container self-test method is worthless.

Then how can the container configuration be tested?

Explicit Testing of Container Configuration #

Since a container self-test doesn't achieve the desired goal, how can we ensure that an application can be composed correctly?

One option is to write an automated integration test (not a unit test) for each service that the application requires. Still, if done manually, you run the risk of forgetting to write a test for a specific service. A better option is to come up with a convention so that you can identify all the services required by a specific application, and then write a convention-based test to verify that the container can resolve them all.

Will this guarantee that the application can be correctly composed?

No, it only guarantees that it can be composed - not that this composition is correct.

Even when a composed instance can be created for each and every service, many things may still be wrong:

  • Composition is just plain wrong:
    • Decorators may be missing
    • Decorators may have been added in the wrong order
    • The wrong services are injected into consumers (this is more likely to happen when you follow the Reused Abstractions Principle, since there will be multiple concrete implementations of each interface)
  • Configuration values like connection strings and such are incorrect - e.g. while a connection string is supplied to a constructor, it may not contain the correct values.
  • Even if everything is correctly composed, the run-time environment may prevent the application from working. As an example, even if an injected connection string is correct, there may not be any connection to the database due to network or security misconfiguration.

In short, a Subcutaneous Test or full System Test is the only way to verify that everything is correctly wired. However, if you have good test coverage at the unit test level, a series of Smoke Tests is all that you need at the System Test level because in general you have good reason to believe that all behavior is correct. The remaining question is whether all this correct behavior can be correctly connected, and that tends to be an all-or-nothing proposition.

Conclusion #

While it would be possible to write a set of convention-based integration tests to verify the configuration of a DI Container, the return of investment is too low since it doesn't remove the need for a set of Smoke Tests at the System Test level.


While you're correct that integration tests are not required, they can still provide value if they pin-point the problem more quickly.

A failing smoke test won't always tell you exactly what went wrong (while that's a failing of the test, it's not always that easy to fix). Rather than investigating, I'd prefer to have a failing integration test that means the smoke test won't even be run.

I find the most value comes from tests that try and resolve the root of the object graph from the bootstrapped container, i.e. anything I (or a framework) explicitly try and resolve at runtime.

Their being green doesn't necessarily mean the application is fine, but their being red means it's definitely broken. It is a duplication of testing, but so are the smoke tests and some of the unit tests (hopefully!).
2011-12-22 11:18 UTC
I am currently not quite sure whether an automated configuration test is really required or not.

The correct wireing is already tested by the DI Container itself. An error prone configuration will be obvious in the first developer or at least user test.
So, is this kind of test overhead, useful or even necessary?
I probably wouldn't do these kind tests.
2011-12-22 20:36 UTC
Mike Bridge
In my application, I'd like to assert that my composition root will correctly instantiate a decorator in one situation and not in another. I managed to miswire this on my first attempt and have been trying to figure out how to write a simple test to assert that I'd specified the dependencies correctly.

Would it be worthwhile strategy to unit test my container's configuration by mocking the container's resolver? I'd like to be able to run my registration on a container, then assert that the mocked resolver received the correct "Resolve" messages in the correct order. Currently I'm using validating this with an integration test, but I was thinking that this would be much simpler---if my container supports it.

2011-12-28 19:23 UTC
That sounds like a brittle test, because instead of testing the 'what' you'd be testing the 'how'.
2011-12-29 07:50 UTC
Mike Bridge
Thanks, that's probably a correct assessment. I was intending it to be an interaction test which asserts "given a certain input to my DI container initializer, my decorator should be created when the container instantiates my graph".

I'll go back and think a bit more about how I can test the resulting behaviour instead of the implementation.
2011-12-29 17:12 UTC
Nice post! We have a set of automated developer acceptance tests which start the system in an machine.specifications test, execute business commands, shuts down the system and does business asserts on the external subsystems which are stubbed away depending on the feature under test. With that we have an implicit container test: the system boots and behave as expected. If someone adds new ccomponents for that feature which are not properly registered the tests will fail.

2012-01-14 07:58 UTC
Hi Mark,

While I agree on the principle one question came to my mind related to the first part. I don't understand why resolving IQux is an issue because even in runtime it's not required and not registered?


2012-02-03 18:16 UTC
Thomas, the point of IQux is that, in the example, the container doesn't know about the IQux interface. A container self-test is only going to walk through all components to see if they can be resolved. However, such a test will never attempt to resolve IQux, since the container doesn't know about it.

So, if an application code base requires IQux, the application is going to fail even when the container self-test succeeds.
2012-02-03 19:30 UTC
Mark, I understand that. However even in the production scenario when a code base requires IQux at some point in the time (for example Baz requires IQux in the constructor), it should be registered in the container, otherwise it won't work. I think I should miss something.
2012-02-05 11:10 UTC
Yes - that's why a container self-test is worthless. The container doesn't know about the requirements of you application. All it can do is to test whether or not it's internally consistent. It doesn't tell you anything about its ability to resolve the object graphs your application is going to need.
2012-02-05 12:02 UTC
Rajkumar Srinivasan
Mark, Great post. In order to avoid violation of Resused Abstraction Principle, I infer from some of your other posts that we need to provide null object implementation for all interfaces or abstractions. I am just trying to confirm if my inference is correct.
2012-06-05 09:10 UTC
I'm not sure I can agree with that - that sounds a bit like Cargo Culting to me... The point of the RAP is that it tells us something about the degree of abstraction of our interfaces. If the sole purpose of adding a Null Object implementation is to adhere to the RAP, it may address the mechanics of the RAP, but not the spirit</i>.
2012-06-05 09:18 UTC

You have made a very basic and primitive argument to sell a complex but feasible process as pointless:

such a method is close to worthless

and without a working example of a better way you explain why you are right. I am an avid reader of your posts and often reference them but IMO this particular argument is not well reasoned.

Your opening argument explains why you may have an issue when using the Service Locator anti-pattern:

an application could require instances of the ICorge, Corge or Grault types which are completely external to the configuration, in which case resolution would fail.

Assertions such as the following would ideally be specified in a set of automated tests regardless of the method of composition

Decorators may be missing


Decorators may have been added in the wrong order

And I fail to see how Pure DI would make the following into lesser issues

Configuration values like connection strings and such are incorrect - e.g. while a connection string is supplied to a constructor, it may not contain the correct values


the run-time environment may prevent the application from working

My response was prompted by a statement made by you on stackoverflow. You are most highly regarded when it comes to .NET and DI and I feel statements such as "Some people hope that you can ask a DI Container to self-diagnose, but you can't." are very dangerous when they are out-of-date.

Simple Injector will diagnose a number of issues beyond "do I have a registration for X". I'm not claiming that these tests alone are full-proof validation of your configuration but they are set of built in tests for issues that can occur in any configuration (including a Pure configuration) and these combined tests are far from worthless ...

  • LifeStyle Mismatches: The component depends on a service with a lifestyle that is shorter than that of the component
  • Short Circuited Dependencies: The component depends on an unregistered concrete type and this concrete type has a lifestyle that is different than the lifestyle of an explicitly registered type that uses this concrete type as its implementation
  • Potential Single Responsibility Violations: The component depends on too many services
  • Container-registered Type: A concrete type that was not registered explicitly and was not resolved using unregistered type resolution, but was created by the container using the default lifestyle
  • Torn Lifestyle: Multiple registrations with the same lifestyle map to the same component
  • Ambiguous Lifestyles: Multiple registrations with the different lifestyles map to the same component
  • Disposable Transient Components: A registration has been made with the Transient lifestyle for a component that implements IDisposable

Your claim that "such a method is close to worthless" may be true for the majority of the available .NET DI Containers but it does not take Simple Injector's diagnostic services into consideration.

2015-12-11 09:15 UTC

Factory Overload

Monday, 19 December 2011 13:04:55 UTC

Recently I received a question from Kelly Sommers about good ways to refactor away from Factory Overload. Basically, she's working in a code base where there's an explosion of Abstract Factories which seems to be counter-productive. In this post I'll take a look at the example problem and propose a set of alternatives.

An Abstract Factory (and its close relative Product Trader) can serve as a solution to various challenges that come up when writing loosely coupled code (chapter 6 of my book describes the most common scenarios). However, introducing an Abstract Factory may be a leaky abstraction, so don't do it blindly. For example, an Abstract Factory is rarely the best approach to address lifetime management concerns. In other words, the Abstract Factory has to make sense as a pure model element.

That's not the case in the following example.

Problem Statement #

The question centers around a code base that integrates with a database closely guarded by DBA police. Apparently, every single database access must happen through a set of very fine-grained stored procedures.

For example, to update the first name of a user, a set of stored procedures exist to support this scenario, depending on the context of the current application user:

User type Stored procedure Parameter name
Admin update_admin_firstname adminFirstName
Guest update_guest_firstname guestFirstName
Regular update_regular_firstname regularFirstName
Restricted update_restricted_firstname restrictedFirstName

As this table demonstrates, not only is there a stored procedure for each user context, but the parameter name differs as well. However, in this particular case it seems as though there's a pattern to the names.

If this pattern is consistent, I think the easiest way to address these variations would be to algorithmically build the strings from a couple of templates.

However, this is not the route taken by Kelly's team, so I assume that things are more complicated than that; apparently, a templated approach is not viable, so for the rest of  this article I'm going to assume that it's necessary to write at least some code to address each case individually.

The current solution that Kelly's team has implemented is to use an Abstract Factory (Product Trader) to translate the user type into an appropriate IUserFirstNameModifier instance. From the consuming code, it looks like this:

var modifier = factory.Create(UserTypes.Admin);

where the factory variable is an instance of the IUserFirstNameModifierFactory interface. This is certainly loosely coupled, but looks like a leaky abstraction. Why is a factory needed? It seems that its single responsibility is to translate a UserTypes instance (an enum) into an IUserFirstNameModifier. There's a code smell buried here - try to spot it before you read on :)

Proposed Solution #

Kelly herself suggests an alternative involving a concrete Builder which can create instances of a single concrete UserFirstNameModifier with or without an implicit conversion:

// Implicit conversion.
UserFirstNameModifier modifier1 = 
// Without implicit conversion.
var modifier2 = builder

While this may seem to reduce the number of classes involved, it has several drawbacks:

  • First of all, the Fluent Builder pattern implies that you can forgo invoking any of the WithXyz methods (WithUserType) and just accept all the default values encapsulated in the builder. This again implies that there's a default user type, which may or may not make sense in that particular domain. Looking at Kelly's code, UserTypes is an enum (and thus has a default value), so if WithUserType isn't invoked, the Create method defaults to UserTypes.Admin. That's a bit too implicit for my taste.
  • Since all involved classes are now concrete, the proposed solution isn't extensibile (and by corollary hard to unit test).
  • The builder is essentially a big switch statement.

Both the current implementation and the proposed solution involves passing an enum as a method parameter to a different class. If you've read and memorized Refactoring you should by now have recognized both a code smell and the remedy.

Alternative 1a: Make UserType Polymorphic #

The code smell is Feature Envy and a possible refactoring is to replace the enum with a Strategy. In order to do that, an IUserType interface is introduced:

public interface IUserType
    IUserFirstNameModifer CreateUserFirstNameModifier();

Usage becomes as simple as this:

var modifier = userType.CreateUserFirstNameModifier();

Obviously, more methods can be added to IUserType to support other update operations, but care should be taken to avoid creating a Header Interface.

While this solution is much more object-oriented, I'm still not quite happy with it, because apparently, the context is a CQRS style architecture. Since an update operation is essentially a Command, then why model the implementation along the lines of a Query? Both Abstract Factory and Factory Method patterns represent Queries, so it seems redundant in this case. It should be possible to apply the Hollywood Principle here.

Alternative 1b: Tell, Don't Ask #

Why have the user type return an modifier? Why can't it perform the update itself? The IUserType interface should be changed to something like this:

public interface IUserType
    void CommitUserFirtName(string firstName);

This makes it easier for the consumer to commit the user's first name because it can be done directly on the IUserType instance instead of first creating the modifier.

It also makes it much easier to unit test the consumer because there's no longer a mix of Command and Queries within the same method. From Growing Object-Oriented Software we know that Queries should be modeled with Stubs and Commands with Mocks, and if you've ever tried mixing the two you know that it's a sort of interaction that should be minimized.

Alternative 2a: Distinguish by Type #

While I personally like alternative 1b best, it may not be practical in all situations, so it's always valuable to examine other alternatives.

The root cause of the problem is that there's a lot of stored procedures. I want to reiterate that I still think that the absolutely easiest solution would be to generate a SqlCommand from a string template, but given that this article assumes that this isn't possible or desirable, it follows that code must be written for each stored procedure.

Why not simply define an interface for each one? As an example, to update the user's first name in the context of being an ‘Admin' user, this Role Interface can be used:

public interface IUserFirstNameAdminModifier
    void Commit(string firstName);

Similar interfaces can be defined for the other user types, such as IUserFirstNameRestrictedModifier, IUserFirstNameGuestModifier and so on.

This is a very simple solution; it's easy to implement, but risks violating the Reused Abstractions Principle (RAP).

Alternative 2b: Distinguish by Generic Type #

The problem with introducing interfaces like IUserFirstNameAdminModifier, IUserFirstNameRestrictedModifier, IUserFirstNameGuestModifier etc. is that they differ only by name. The Commit method is the same for all these interfaces, so this seems to violate the RAP. It'd be better to merge all these interfaces into a single interface, which is what Kelly's team is currently doing. However, the problem with this is that the type carries no information about the role that the modifier is playing.

Another alternative is to turn the modifier interface into a generic interface like this:

public interface IUserFirstNameModifier<T> 
    where T : IUserType
    void Commit(string firstName);

The IUserType is a Marker Interface, so .NET purists are not going to like this solution, since the .NET Type Design Guidelines recommend against using Marker Interfaces. However, it's impossible to constrain a generic type argument against an attribute, so the party line solution is out of the question.

This solution ensures that consumers can now have dependencies on IUserFirstNameModifier<AdminUserType>, IUserFirstNameModifier<RestrictedUserType>, etc.

However, the need for a marker interface gives off another odeur.

Alternative 3: Distinguish by Role #

The problem at the heart of alternative 2 is that it attempts to use the type of the interfaces as an indicator of the roles that Services play. It's seems that making the type distinct works against the RAP, but when the RAP is applied, the type becomes ambiguous.

However, as Ted Neward points out in his excellent series on Multiparadigmatic .NET, the type is only one axis of variability among many. Perhaps, in this case, it may be much easier to use the name of the dependency to communicate its role instead of the type.

Given a single, ambiguous IUserFirstNameModifier interface (just as in the original problem statement), a consumer can distinguish between the various roles of modifiers by their names:

public partial class SomeConsumer
    private readonly IUserFirstNameModifier adminModifier;
    private readonly IUserFirstNameModifier guestModifier;
    public SomeConsumer(
        IUserFirstNameModifier adminModifier,
        IUserFirstNameModifier guestModifier)
        this.adminModifier = adminModifier;
        this.guestModifier = guestModifier;
    public void DoSomething()
        if (this.UseAdmin)

Now it's entirely up to the Composition Root to compose SomeConsumer with the correct modifiers, and while this can be done manually, it's an excellent case for a DI Container and a bit of Convention over Configuration.

Conclusion #

I'm sure that if I'd spent more time analyzing the problem I could have come up with more alternatives, but this post is becoming long enough already.

Of the alternatives I've suggested here, I prefer 1b or 3, depending on the exact requirements.


Nice work, Mark.

Personally I prefer 1b over 3. The if-statement in option 3 looks a bit suspicious to me as one day it might give rise to some maintenance if more user types have to be supported by the SomeConsumer type, so I am afraid it may violate the open/closed principle. Option 1b looks more straight forward to me.
2011-12-21 12:48 UTC
wow, those dba enforced sprocs and security measures are friction no developer should have to face

sorry, way OT
2011-12-21 17:52 UTC
Emanuel Pasat
I have a similar situation and it might be a good ocassion to clarify it.
I'm trying to implement a visitor pattern over a list of visitees object, constructed with a factory.

A visitee requires some external services, so, it might be resonable to have them injected already by IOC in the _factory (VisiteeFactory). isLast is an example of other contextual information, outside of dto, but required to create other visitee types.
Given that is a bounded context, how can I improve this design?

Pseudocode follows:

VisiteeAdapterBase Create(Dto dto, bool isLast);

VisiteeFactory : IVisiteeFactory
ctor VisiteeFactory(IExternalService service)

public VisiteeAdapterBase Create(Dto dto, bool isLast)
// lot of switches and ifs to determine the type of VisiteeAdapterBase
if (isLast && dto.Type == "1") {
return new Type1VisiteeAdapter(... dto props...);


ConsumerCtor(VisiteeFactory factory, List<Dto> dtoList)
// guards
_factory = factory;
_dtoList = dtoList;
foreach (var dto in _dtoList) {
// isLast is additional logic, outside of dto
var visiteeAdapter = _factory.Create(dto, isLast);

2011-12-22 10:06 UTC
Too many classes/interfaces, why not use a simple lookup table?
2012-01-09 06:48 UTC
Eber, let me quoute myself from this particular post: "I think the easiest way to address these variations would be to algorithmically build the strings from a couple of templates". A lookup table falls into that category, so I agree that such a thing would be easier if at all possible.

The whole premise of the rest of the post is that for some reason, it's more complicated than that...
2012-01-09 19:58 UTC

Polymorphic Consistency

Wednesday, 07 December 2011 08:40:21 UTC

Asynchronous message passing combined with eventual consistency makes it possible to build very scalable systems. However, sometimes eventual consistency isn't appropriate in parts of the system, while it's acceptable in other parts. How can a consistent architecture be defined to fit both ACID and eventual consistency? This article provides an answer.

The case of an online game #

Last week I visited Pixel Pandemic, a company that produces browser-based MMORPGs. Since each game world has lots of players who can all potentially interact with each other, scalability is very important.

In traditional line of business applications, eventual consistency is often an excellent fit because the application is a projection of the real world. My favorite example is an inventory system: it models what's going on in one or more physical warehouses, but the real world is the ultimate source of truth. A warehouse worker might accidentally drop and damage some of the goods, in which case the application must adjust after the fact.

In other words, the information contained within line of business applications tend to lag after the real world. It's impossible to guarantee that the application is always consistent with the real world, so eventual consistency is a natural fit.

That's not the case with an online game world. The game world itself is the source of truth, and it must be internally consistent at all times. As an example, in Zombie Pandemic, players fight against zombies and may take damage along the way. Players can heal themselves, but they would prefer (I gather) that the healing action takes place immediately, and not some time in the future where the character might be dead. Similarly, when a player hits a zombie, they'd prefer to apply the damage immediately. (However, I think that even here, eventual consistency might provide some interesting game mechanics, but that's another discussion.)

While discussing these matters with the nice people in Pixel Pandemic, it turned out that while some parts of the game world have to be internally consistent, it's perfectly acceptable to use eventual consistency in other cases. One example is the game's high score table. While a single player should have a consistent view of his or her own score, it's acceptable if the high score table lags a bit.

At this point it seemed clear that this particular online game could use an appropriate combination of ACID and eventual consistency, and I think this conclusion can be generalized. The question now becomes: how can a consistent architecture encompass both types of consistency?

Problem statement #

With the above example scenario in mind the problem statement can be generalized:

Given that an application should apply a mix of ACID and eventual consistency, how can a consistent architecture be defined?

Keep in mind that ACID consistency implies that all writes to a transactional resource must take place as a blocking method call. This seems to be at odds with the concept of asynchronous message passing that works so well with eventual consistency.

However, an application architecture where blocking ACID calls are fundamentally different than asynchronous message passing isn't really an architecture at all. Developers will have to decide up-front whether or not a given operation is or isn't synchronous, so the ‘architecture' offers little implementation guidance. The end result is likely to be a heterogeneous mix of Services, Repositories, Units of Work, Message Channels, etc. A uniform principle will be difficult to distill, and the whole thing threatens to devolve into Spaghetti Code.

The solution turns out to be not at all difficult, but it requires that we invert our thinking a bit. Most of us tend to think about synchronous code first. When we think about code performing synchronous work it seems difficult (perhaps even impossible) to retrofit asynchrony to that model. On the other hand, the converse isn't true.

Given an asynchronous API, it's trivial to provide a synchronous, blocking implementation.

Adopting an architecture based on asynchronous message passing (the Pipes and Filters architecture) enables both scenarios. Eventual consistency can be achieved by passing messages around on persistent queues, while ACID consistency can be achieved by handling a message in a blocking call that enlists a (potentially distributed) transaction.

An example seems to be in order here.

Example: keeping score #

In the online game world, each player accumulates a score based on his or her actions. From the perspective of the player, the score should always be consistent. When you defeat the zombie boss, you want to see the result in your score right away. That sounds an awful lot like the Player is an Aggregate Root and the score is part of that Entity. ACID consistency is warranted whenever the Player is updated.

On the other hand, each time a score changes it may influence the high score table, but this doesn't need to be ACID consistent; eventual consistency is fine in this case.

Once again, polymorphism comes to the rescue.

Imagine that the application has a GameEngine class that handles updates in the game. Using an injected IChannel<PointsChangedEvent> it can update the score for a player as simple as this:

/* Lots of other interesting things happen
    * here, like calculating the new score... */
var cmd =
    new ScoreChangedEvent(this.playerId, score);

The Send method returns void, so it's a good example of a naturally asynchronous API. However, the implementation must do two things:

  • Update the Player Aggregate Root in a transaction
  • Update the high score table (eventually)

That's two different types of consistency within the same method call.

The first step to enable this is to employ the trusty old Composite design pattern:

public class CompositeChannel<T> : IChannel<T>
    private readonly IEnumerable<IChannel<T>> channels;
    public CompositeChannel(params IChannel<T>[] channels)
        this.channels = channels;
    public void Send(T message)
        foreach (var c in this.channels)

With a Composite channel it's possible to compose a polymorphic mix of IChannel<T> implementations, some blocking and some asynchronous.

ACID write #

To update the Player Aggregate Root a simple Adapter writes the event to a persistent data store. This could be a relational database, a document database, a REST resource or something else - it doesn't really matter exactly which technology is used.

public class PlayerStoreChannel : 
    private readonly IPlayerStore store;
    public PlayerStoreChannel(IPlayerStore store)
        this.store = store;
    public void Send(ScoreChangedEvent message)
        this.store.Save(message.PlayerId, message);

The important thing to realize is that the IPlayerStore.Save method will be a blocking method call - perhaps wrapped in a distributed transaction. This ensures that updates to the Player Aggregate Root always leave the data store in a consistent state. Either the operation succeeds or it fails during the method call itself.

This takes care of the ACID consistent write, but the application must also update the high score table.

Asynchronous write #

Since eventual consistency is acceptable for the high score table, the message can be transmitted over a persistent queue to be picked up by a background process.

A generic class can server as an Adapter over an IQueue abstraction:

public class QueueChannel<T> : IChannel<T>
    private readonly IQueue queue;
    private readonly IMessageSerializer serializer;
    public QueueChannel(IQueue queue,
        IMessageSerializer serializer)
        this.queue = queue;
        this.serializer = serializer;
    public void Send(T message)

Obvously, the Enqueue method is another void method. In the case of a persistent queue, it'll block while the message is being written to the queue, but that will tend to be a fast operation.

Composing polymorphic consistency #

Now all the building blocks are available to compose both channel implementations into the GameEngine via the CompositeChannel. That might look like this:

var playerConnString = ConfigurationManager
var gameEngine = new GameEngine(
    new CompositeChannel<ScoreChangedEvent>(
        new PlayerStoreChannel(
            new DbPlayerStore(playerConnString)),
        new QueueChannel<ScoreChangedEvent>(
            new PersistentQueue("messageQueue"),                        
            new JsonSerializer())));

When the Send method is invoked on the channel, it'll first invoke a blocking call that ensures ACID consistency for the Player, followed by asynchronous message passing for eventual consistency in other parts of the application.

Conclusion #

Even when parts of an application must be implemented in a synchronous fashion to ensure ACID consistency, an architecture based on asynchronous message passing provides a flexible foundation that enables you to polymorphically mix both kinds of consistency in a single method call. From the perspective of the application layer, this provides a consistent and uniform architecture because all mutating actions are modeled as commands end events encapsulated in messages.


Thanks a lot for mentioning us here at Pixel Pandemic and for your insights Mark! I very much agree with your conclusion and at this point we're discussing an architectural switch to what you're outlining here (a form of polymorphic consistency) and an event sourcing model for our persistent storage needs.

We're working on ways to make as many aspects of our games as possible fit with an eventual consistency model by, e.g. simply by changing the way we communicate information about the virtual game world state to players (to put them in a frame of mind in which eventual consistency fits naturally with their perception of the game state).

Looking very much forward to meeting with you again soon and discussing more details!
2011-12-07 09:46 UTC
Could you also use the async ctp stuff to do it all in a single command, so that you are not blocking while waiting for the persistant store to do I/O, and then when it calls back push the message onto the message queue then return.. If you were using something like async controllers in mvc 4 it would mean you could do something like registering a user which saves them to the database, then pass this event information onto the persistant queue so a backend could pick it up and send emails, and do other tasks that are longer to execute.

await this.store.Save(message.PlayerId, message);

Keen to hear your thoughts
2011-12-08 13:38 UTC
Why not? You can combine the async functionality with the approach described above. It could make the application more efficient, since it would free up a thread while the first transaction is being completed.

However, while the async CTP makes it easier to write asynchronous code, it doesn't help with blocking calls. It may be more efficient, but not necessarily faster. You can't know whether or not a transaction has committed until it actually happens, so you still need to wait for that before you decide whether or not to proceed.

BTW, F# has had async support since its inception, so it's interesting to look towards what people are doing with that. Agents (the F# word for Actors) seem to fit into that model pretty well, and as far as I can tell, an Agent is simply an in-memory asynchronous worker process.
2011-12-08 14:07 UTC
Hi Mark, firstly: great post.

I do have a question, though. I can see how this works for all future commands, because they will all need to load the aggregate work to work on it and that will be ACID at all times. What I'm not sure about is how that translates to the query side of the coin - where the query is *not* the high-score table, but must be immediately available on-screen.

Even if hypothetical, imagine the screen has a typical Heads-Up-Display of relevant information - stuff like 'ammo', 'health' and 'current score'. These are view concerns and will go down the query arm of the CQRS implementation. For example, the view-model backing this HUD could be stored in document storage for the player. This is, then, subject to eventual consistency and is not ACID, right?

I'm clearly not 'getting' this bit of the puzzle at the moment, hopefully you can enlighten me.
2011-12-14 21:23 UTC
A HUD is exactly the sort of scenario that a must be implemented by a synchronous write. If you want to be sure that the persisted data is ACID consistent, it must be written as a synchronous, blocking operation. This means that once the query side comes along, it can simply read from the same persistent store because it's always up to date. That sort of persisted data isn't eventually consistent - it's atomically consistent.
2011-12-21 08:08 UTC

TDD improves reusability

Thursday, 10 November 2011 16:55:10 UTC

There's this meme going around that software reuse is a fallacy. Bollocks! The reuse is a fallacy meme is a fallacy :) To be fair, I'm not claiming that everything can and should be reused, but my claim is that all code produced by Test-Driven Development (TDD) is being reused. Here's why:

When tests are written first, they act as a kind of REPL. Tests tease out the API of the SUT, as well as its behavior. In this point in the development process, the tests serve as a feedback mechanism. Only later, when the tests and the SUT stabilize, will the tests be repurposed (dare I say ‘reused'?) as regression tests. In other words: over time, tests written during TDD have more than one role:

  1. Feedback mechanism
  2. Regression test

Each test plays one of these roles at a time, but not both.

While the purpose of TDD is to evolve the SUT, the process produces two types of artifacts:

  1. Tests
  2. Production code

Notice how the tests appear before the production code, which is an artifact of the test code.

The unit tests are the first client of the production API.

When the production code is composed into an application, that application becomes the second client, so it reuses the API. This is a very beneficial effect of TDD, and probably one of the main reasons why TDD, if done correctly, produces code of high quality.

A colleague once told me (when referring to scale-out) that the hardest step is to go from one server to two servers, and I've later found that principle to apply much more broadly. Generalizing from a single case to two distinct cases is often the hardest step, and it becomes much easier to generalize further from two to an arbitrary number of cases.

This explains why TDD is such an efficient process. Apart from the beneficial side effect of producing a regression test suite, it also ensures that at the time the API goes into production, it's already being shared (or reused) between at least two distinct clients. If, at a later time, it becomes necessary to add a third client, the hard part is already done.

TDD produces reusable code because the production application reuses the API which were realized by the tests.


I am a junior developer and I am doing TDD for a small project right now and I can only agree. My code looks much better because i really use it instead of making assumptions how it should be used (ADD - Assumption Driven Development)
2011-11-10 18:59 UTC
Hi all TDD fans.
If you are using NUnit for TDD you may find useful NUnit.Snippets NuGet package - "almost every assert is only few keystrokes away" TM ;)
2011-11-11 12:03 UTC
I think that you're equating *using* a class with reusing a class - the two aren't the same.
2011-11-16 23:04 UTC
Hello Mark!

Do you use some test coverage software?

Is there some free test coverage tools thats worth to use? )

2012-05-11 07:11 UTC
Or if you dont know free tools - maybe some commercial tools - but not very expensive ))

I have found for example "dotcover" from jetbrains - 140 its ok for the company )
2012-05-11 09:13 UTC
I rarely use code coverage tools. Since I develop code almost exclusively with TDD, I know my coverage is good.

I still occasionally use code coverage tools when I work in a team environment, so I have nothing against them. When I do, I just use the coverage tool which is built into Visual Studio. When used with the TestDriven.NET Visual Studio add-in it's quite friction-less.
2012-05-11 18:56 UTC


Tuesday, 08 November 2011 15:29:05 UTC

Now that my book about Dependency Injection is out, it's only fitting that I also invert my own dependencies by striking out as an independent consultant/advisor. In the future I'm hoping to combine my writing and speaking efforts, as well as my open source interests, with helping out clients write better code.

If you'd like to get help with Dependency Injection or Test-Driven Development, SOLID, API design, application architecture or one of the other topics I regularly cover here on my blog, I'm available as a consultant worldwide.

When it comes to Windows Azure, I'll be renewing my alliance with my penultimate employer Commentor, so you can also hire me as part of larger deal with Commentor.

In case you are wondering what happened to my employment with AppHarbor, I resigned from my position there because I couldn't make it work with all the other things I also would like to do. I still think AppHarbor is a very interesting project, and I wish my former employers the best of luck with their endeavor.

This has been a message from the blog's sponsor (myself). Soon, regular content will resume.


Well shux, I was waiting on pins and needles for some magic unicorn stuff from ya! I hear ya though, gotta have that liberty. :) I'm often in the same situation.

Best of luck to you, I'll be reading the blog as always.

BTW - Got the physical book finally, even though I'm no newb of IoC and such, I'd have loved a solid read when I was learning about the options back in the day. ;)

2011-11-08 16:11 UTC
Best of luck.

As with all other endevours you set your mind to, you will for sure also excel as a free agent.
2011-11-08 16:58 UTC
Flemming Laugesen
Congratulation my friend - looking forward to take advances of you expertice :)

2011-11-08 19:45 UTC
Congratulations on your decision, and the very best of luck, I'm sure you'll have heaps of succes.
2011-11-08 21:52 UTC
I wish you the best with your new adventure. I cannot thank you enough for all I learned from your book on Dependency Injection.
One of your Fans in USA,
2011-11-09 03:08 UTC
Congrats! Best of luck.
2011-11-09 09:00 UTC

SOLID concrete

Tuesday, 25 October 2011 15:01:15 UTC

Greg Young gave a talk at GOTO Aarhus 2011 titled Developers have a mental disorder, which was (semi-)humorously meant, but still addressed some very real concerns about the cognitive biases of software developers as a group. While I have no intention to provide a complete resume of the talk, Greg said one thing that made me think a bit (more) about SOLID code. To paraphrase, it went something like this:

Developers have a tendency to attempt to solve specific problems with general solutions. This leads to coupling and complexity. Instead of being general, code should be specific.

This sounds correct at first glance, but once again I think that SOLID code offers a solution. Due to the Single Responsibility Principle each SOLID concrete (pardon the pun) class will tend to very specifically address a very narrow problem.

Such a class may implement one (or more) general-purpose interface(s), but the concrete type is specific.

The difference between the generality of an interface and the specificity of a concrete type becomes more and more apparent the better a code base applies the Reused Abstractions Principle. This is best done by defining an API in terms of Role Interfaces, which makes it possible to define a few core abstractions that apply very broadly, while implementations are very specific.

As an example, consider AutoFixture's ISpecimenBuilder interface. This is a very central interface in AutoFixture (in fact, I don't even know just how many implementations it has, and I'm currently too lazy to count them). As an API, it has proven to be very generally useful, but each concrete implementation is still very specific, like the CurrentDateTimeGenerator shown here:

public class CurrentDateTimeGenerator : ISpecimenBuilder
    public object Create(object request, 
        ISpecimenContext context)
        if (request != typeof(DateTime))
            return new NoSpecimen(request);
        return DateTime.Now;

This is, literally, the entire implementation of the class. I hope we can agree that it's very specific.

In my opinion, SOLID is a set of principles that can help us keep an API general while each implementation is very specific.

In SOLID code all concrete types are specific.


I don't agree with the "Reused Abstractions Principle" article at all. Programming to interfaces provides many benefits even in cases where "one interface, multiple implementations" doesn't apply.

For one, ctor injection for dependencies makes them explicit and increases readability of a particular class (you should be able to get a general idea of what a class does by looking at what dependencies it has in its ctor). However, if you're taking in more than a handful of dependencies, that is an indication that your class needs refactoring. Yes, you could accept dependencies in the form of concrete classes, but in such cases you are voiding the other benefits of using interfaces.

As far as API writing goes, using interfaces with implementations that are internal is a way to guide a person though what is important in your API and what isn't. Offering up a bunch of instantiatable classes in an API adds to the mental overhead of learning your code - whereas only marking the "entry point" classes as public will guide people to what is important.

Further, as far as API writing goes, there are many instances where Class A may have a concrete dependency on Class B, but you wish to hide the methods that Class A uses to talk to Class B. In this case, you may create an interface (Interface B) with all of the public methods that you wish to expose on Class B and have Class B implement them, then add your "private" methods as simple, non-virtual, methods on Class B itself. Class A will have a property of type Interface B, which simply returns a private field of type Class B. Class A can now invoke specific methods on Class B that aren't accessible though the public API using it's private reference to the concrete Class B.

Finally, there are many instances where you want to expose only parts of a class to another class. Let's say that you have an event publisher. You would normally only want to expose the methods that have to do with publishing to other code, yet that same class may include facilities that allow you to register handler objects with it. Using interfaces, you can limit what other classes can and can't do when they accept your objects as dependencies.

These are instances of what things you can do with interfaces that make them a useful construct on their own - but in addition to all of that, you get the ability to swap out implementations without changing code. I know that often times implementations are never swapped out in production (rather, the concrete classes themselves are changed), which is why I mention this last, but in the rare cases where it has to be done, interfaces make this scenario possible.

My ultimate point is that interfaces don't always equal generality or abstraction. They are simply tools that we can use to make code explicit and readable, and allow us to have greater control over method/property accessibility.
2011-10-25 18:15 UTC
The RAP fills the same type of role as unit testing/TDD: theoretically, it's possible to write testable code without writing a single unit test against it, but how do you know?

It's the same thing with the RAP: how can you know that it's possible to exchange one implementation of an interface with another if you've never tried it? Keep in mind that Test Doubles don't count because you can always create a bogus implementation of any interface. You could have an interface that's one big leaky abstraction: even though it's an interface, you can never meaningfully replace that single implementation with any other meaningful production implementation.

Also: using an interface alone doesn't guarantee the Liskov Substitution Principle. However, by following the RAP, you get a strong indication that you can, indeed, replace one implementation with another without changing the correctness of the code.
2011-10-25 18:56 UTC
That was my point, though. You can use interfaces as a tool to solve problems that have nothing directly to do with substituting implementations. I think people see this as the only usecase for the language construct, which is sad. These people then turn around and claim that you shouldn't be using interfaces at all, except for cases in which substituting implementation is the goal. I think this attitude disregards many other proper uses for the construct; the most important I think is being able to hide implementation details in the public API.

If an interface does not satisfy RAP, it does not absolutely mean that interface is invalid. Take the Customer and CustomerImpl types specified in the linked article. Perhaps the Customer interface simply exposes a public, readonly, interface for querying information about a customer. The CustomerImpl class, instantiated and acted upon behind the scenes in the domain services, may specify specific details such as OR/mapping or other behavior that isn't intended to be accessible to client code. Although a slightly contrived example (I would prefer the query model to be event sourced, not mapped to a domain model mapped to an RDBMS), I think this use is valid and should not immediately be thrown out because it does not satisfy RAP.
2011-10-25 20:15 UTC
On his bio it says that Greg Young writes for Experts Exchange. Maybe he's the one with the mental disorder :P
2011-10-26 04:16 UTC
Nelson, I think we agree :) To me, the RAP is not an absolute rule, but just another guideline/metric. If none of my interfaces have multiple implementations, I start to worry about the quality of my abstractions, but I don't find it problematic if some of my interfaces have only a single implementation.

Your discussion about interfaces fit well with the Interface Segregation Principle and the concept of Role Interfaces, and I've also previously described how interfaces act as access modifiers.
2011-10-26 08:37 UTC

Checking for exactly one item in a sequence using C# and F#

Tuesday, 11 October 2011 14:36:03 UTC

Here's a programming issue that comes up from time to time. A method takes a sequence of items as input, like this:

public void Route(IEnumerable<string> args)

While the signature of the method may be given, the implementation may be concerned with finding out whether there is exactly one element in the sequence. (I'd argue that this would be a violation of the Liskov Substitution Principle, but that's another discussion.) By corollary, we might also be interested in the result sets on each side of that single element: no elements and multiple elements.

Let's assume that we're required to raise the appropriate event for each of these three cases.

Naïve approach in C# #

A naïve implementation would be something like this:

public void Route(IEnumerable<string> args)
    var countCategory = args.Count();
    switch (countCategory)
        case 0:
        case 1:

However, the problem with that is that IEnumerable<string> carries no guarantee that the sequence will ever end. In fact, there's a whole category of implementations that keep iterating forever - these are called Generators. If you pass a Generator to the above implementation, it will never return because the Count method will block forever.

Robust implementation in C# #

A better solution comes from the realization that we're only interested in knowing about which of the three categories the input matches: No elements, a single element, or multiple elements. The last case is covered if we find at least two elements. In other words, we don't have to read more than at most two elements to figure out the category. Here's a more robust solution:

public void Route(IEnumerable<string> args)
    var countCategory = args.Take(2).Count();
    switch (countCategory)
        case 0:
        case 1:

Notice the inclusion of the Take(2) method call, which is the only difference from the first attempt. This will give us at most two elements that we can then count with the Count method.

While this is better, it still annoys me that it's necessary with a secondary LINQ call (to the Single method) to extract that single element. Not that it's particularly inefficient, but it still feels like I'm repeating myself here.

(We could also have converted the Take(2) iterator into an array, which would have enabled us to query its Length property, as well as index into it to get the single value, but it basically amounts to the same work.)

Implementation in F# #

In F# we can implement the same functionality in a much more compact manner, taking advantage of pattern matching against native F# lists:

member this.Route args =
    let atMostTwo = args |> Seq.truncate 2 |> Seq.toList
    match atMostTwo with
    | [] -> onNoArgument.Trigger(Unit.Default)
    | [arg] -> onSingleArgument.Trigger(arg)
    | _ -> onMultipleArguments.Trigger(args)

The first thing happening here is that the input is being piped through a couple of functions. The truncate method does the same thing as the Take LINQ method does, and the toList method subsequently converts that sequence of at most two elements into a native F# list.

The beautiful thing about native F# lists is that they support pattern matching, so instead of first figuring out in which category the input belongs, and then subsequently extract the data in the single element case, we can match and forward the element in a single statement.

Why is this important? I don't know… it's just satisfying on an aesthetic level :)


string item = null;
int count = 0;

foreach(var current in args)
item = current;
i ++;

if (i == 2)

if (i == 1)
2011-10-11 14:42 UTC

Weakly-typed versus Strongly-typed Message Channels

Friday, 23 September 2011 09:08:53 UTC

Soon after I posted my previous blog post on message dispatching without Service Location I received an email from Jeff Saunders with some great observations. Jeff has been so kind to allow me to quote his email here on the blog, so here it is:

“I enjoyed your latest blog post about message dispatching. I have to ask, though: why do we want weakly-typed messages? Why can't we just inject an appropriate IConsumer<T> into our services - they know which messages they're going to send or receive.

“A really good example of this is ISubject<T> from Rx. It implements both IObserver<T> (a message consumer) and IObservable<T> (a message producer) and the default implementation Subject<T> routes messages directly from its IObserver side to its IObservable side.

“We can use this with DI quite nicely - I have written an example in .NET Pad: http://dotnetpad.net/ViewPaste/woTkGk6_GEq3P9xTVEJYZg#c9,c26,

“The good thing about this is that we now have access to all of the standard LINQ query operators and the new ones added in Rx, so we can use a select query to map messages between layers, for instance.

“This way we get all the benefits of a weakly-typed IChannel interface, with the added advantages of strong typing for our messages and composability using Rx.

“One potential benefit of weak typing that could be raised is that we can have just a single implementation for IChannel, instead of an ISubject<T> for each message type. I don't think this is really a benefit, though, as we may want different propagation behaviour for each message type - there are other implementations of ISubject<T> that call consumers asynchronously, and we could pass any IObservable<T> or IObserver<T> into a service for testing purposes.”

These are great observations and I think that Rx holds much promise in this space. Basically you can say that in CQRS-style architectures we're already pushing events (and commands) around, so why not build upon what the framework offers?

Even if you find the IObserver<T> interface a bit too clunky with its OnNext, OnError and OnCompleted methods compared to the strongly typed IConsumer<T> interface, the question still remains: why do we want weakly-typed messages?

We don't, necessarily. My previous post wasn't meant as a particular endorsement of a weakly typed messaging channel. It was more an observation that I've seen many variations of this IChannel interface:

public interface IChannel
    void Send<T>(T message);

The most important thing I wanted to point out was that while the generic type argument may create the illusion that this is a strongly typed method, this is all it is: an illusion. IChannel isn't strongly typed because you can invoke the Send method with any type of message - and the code will still compile. This is no different than the mechanical distinction between a Service Locator and an Abstract Factory.

Thus, when defining a channel interface I normally prefer to make this explicit and instead model it like this:

public interface IChannel
    void Send(object message);

This achieves exactly the same and is more honest.

Still, this doesn't really answer Jeff's question: is this preferable to one or more strongly typed IConsumer<T> dependencies?

Any high-level application entry point that relies on a weakly typed IChannel can get by with a single IChannel dependency. This is flexible, but (just like with Service Locator), it might hide that the client may have (or (d)evolve) too many responsibilities.

If, instead, the client would rely on strongly typed dependencies it becomes much easier to see if/when it violates the Single Responsibility Principle.

In conclusion, I'd tend to prefer strongly typed Datatype Channels instead of a single weakly typed channel, but one shouldn't underestimate the flexibility of a general-purpose channel either.


Thanks for the response, Mark! We are in full agreement.
2011-09-23 09:20 UTC

Page 51 of 66

"Our team wholeheartedly endorses Mark. His expert service provides tremendous value."
Hire me!