ploeh blog danish software design

Code duplication is often harmful - except when it isn't. Learn how to think about the trade-offs involved.

Good programmers know that code duplication should be avoided. There's a cost associated with duplicated code, so we have catchphrases like Don't Repeat Yourself (DRY) in order to remind ourselves that code duplication is evil.

It seems to me that some programmers see themselves as Terminators: out to eliminate all code duplication with extreme prejudice; sometimes, perhaps, without even considering the trade-offs involved. Every time you remove duplicated code, you add a level of indirection, and as you've probably heard before, all problems in computer science can be solved by another level of indirection, except for the problem of too many levels of indirection.

Removing code duplication is important, but it tends to add a cognitive overhead. Therefore, it's important to understand why code duplication is harmful - or rather: when it's harmful.

Rates of change #

Imagine that you copy a piece of code and paste it into ten other code bases, and then never touch that piece of code again. Is that harmful?

Probably not.

Here's one of my favourite examples. When protecting the invariants of objects, I always add Guard Clauses against nulls:

if (subject == null)
    throw new ArgumentNullException("subject");

In fact, I have a Visual Studio code snippet for this; I've been using this code snippet for years, which means that I have code like this Guard Clause duplicated all over my code bases. Most likely, there are thousands of examples of such Guard Clauses on my hard drive, with the only variation being the name of the parameter. I don't mind, because, in my experience, these two lines of code never change.

Yet many programmers see that as a violation of DRY, so instead, they introduce something like this:

Guard.AgainstNull(subject, "subject");

The end result of this is that you've slightly increased the cognitive overhead, but what have you gained? As far as I can tell: nothing. The code still has the same number of Guard Clauses. Instead of idiomatic if statements, they are now method calls, but it's hardly DRY when you have to repeat those calls to Guard.AgainstNull all over the place. You'd still be repeating yourself.

The point here is that DRY is a catchphrase, but shouldn't be an excuse for avoiding thinking explicitly about any given problem.

If the duplicated code is likely to change a lot, the cost of duplication is likely to be high, because you'll have to spend time making the same change in lots of different places - and if you forget one, you'll be introducing bugs in the system. If the duplicated code is unlikely to change, perhaps the cost is low. As with all other risk management, you conceptually multiply the risk of the adverse event happening with the cost of the damage associated with that event. If the product is low, don't bother addressing the risk.

The Rule of Three #

It's not a new observation that unconditional elimination of duplicated code can be harmful. The Rule of Three exists for this reason:

1. Write a piece of code.
2. Write the same piece of code again. Resist the urge to generalise.
3. Write the same piece of code again. Now you are allowed to consider generalising it.

Another reason is that even if the duplication is 'real' (and not coincidental), you may not have enough examples to enable you to make the correct refactoring. Often, even duplicated code comes with small variations:

• The logic is the same, but a string value differs.
• The logic is almost the same, but one duplicate performs an extra small step.
• The logic looks similar, but operates on two different types of object.
• etc.

If you refactor too prematurely, you may perform the wrong refactoring. Often, people introduce helper methods, and then when they realize that the axis of variability was not what they expected, they add more and more parameters to the helper method, and more and more complexity to its implementation. This leads to ripple effects. Ripple effects lead to thrashing. Thrashing leads to poor maintainability. Poor maintainability leads to low productivity.

This is, in my experience, the most important reason to follow the Rule of Three: wait, until you have more facts available to you. You don't have to take the rule literally either. You can wait until you have four, five, or six examples of the duplication, if the rate of change is low.

The parallel to statistics #

If you've ever taken a course in statistics, you would have learned that the less data you have, the less confidence you can have in any sort of analysis. Conversely, the more samples you have, the more confidence can you have if you are trying to find or verify some sort of correlation.

The same holds true for code duplication, I believe. The more samples you have of duplicated code, the better you understand what is truly duplicated, and what varies. The better you understand the axes of variability, the better a refactoring you can perform in order to get rid of the duplication.

Summary #

Code duplication is costly - but only if the code changes. The cost of code duplication, thus, is C*p, where C is the cost incurred, when you need to change the code, and p is the probability that you'll need to change the code. In my experience, for example, the Null Guard Clause in this article has a cost of duplication of 0, because the probability that I'll need to change it is 0.

There's a cost associated with removing duplication - particularly if you make the wrong refactoring. Thus, depending on the values of C and p, you may be better off allowing a bit of duplication, instead of trying to eradicate it as soon as you see it.

You may not be able to quantify C and p (I'm not), but you should be able to estimate whether these values are small or large. This should help you decide if you need to eliminate the duplication right away, or if you'd be better off waiting to see what happens.

Comments

if (subject1 == null)
{
    throw new ArgumentNullException("subject1");
}

if (subject2 == null)
{
    throw new ArgumentNullException("subject2");
}

if (subject3 == null)
{
    throw new ArgumentNullException("subject3");
}

if (subject4 == null)
{
    throw new ArgumentNullException("subject4");
}

if (subject5 == null)
{
    throw new ArgumentNullException("subject5");
}

_ = subject1 ?? throw new ArgumentNullException("subject1");
_ = subject2 ?? throw new ArgumentNullException("subject2");
_ = subject3 ?? throw new ArgumentNullException("subject3");
_ = subject4 ?? throw new ArgumentNullException("subject4");
_ = subject5 ?? throw new ArgumentNullException("subject5");
        

ArgumentNullException.ThrowIfNull(subject1);
ArgumentNullException.ThrowIfNull(subject2);
ArgumentNullException.ThrowIfNull(subject3);
ArgumentNullException.ThrowIfNull(subject4);
ArgumentNullException.ThrowIfNull(subject5);
        

Spencer Farley #

I love the call out of risk * probability. I use threat matrixes to guide decisions all the time.

I'd also like to elaborate that duplication is determined not just by similar code, but by similar forces that change the code. If code looks similar, but changes for different reasons, then centralizing will cause tension between the different cases.

Using the example above, guard clauses are unlikely to change together. Most likely, the programmmer changes a parameter and just the single guard clause is effected. Since the clauses don't change for the same reason, the effect of duplication is very low.

Different forces might effect all guard clauses and justify centralizing. For example, if a system wanted to change its global null handling strategy at debug-time versus production.

However, I often see semantically separate code struggling to share an implementation. I think header interfaces and conforming container-like library wrappers are companion smells of this scenario. For example, I frequently see multiple flows trying to wrap email notifications behind some generic email notification abstraction. The parent flow is still coupled to the idea of email, the two notifications still change separately, and there's an extra poor abstraction between us and sending emails.

2022-02-19 21:00 UTC

My latest Pluralsight course is now available. This time it's about fundamental programming techniques.

Most programmers I meet know about encapsulation and Object-Oriented Design (OOD) - at least, until I start drilling them on specifics. Apparently, the way most people have been taught OOD is at odds with my view on the subject. For a long time, I've wanted to teach OOD the way I think it should be taught. My perspective is chiefly based on Bertrand Meyer's Object-Oriented Software Construction and Robert C. Martin's SOLID principles (presented, among other sources, in Agile Principles, Patterns, and Practices in C#). My focus is on Command Query Separation, protection of invariants, and favouring Composition over Inheritance.

In my new Pluralsight course, I'm happy to be able to teach OOD the way I think it should be taught. The course is aimed at professional developers with a couple of years of experience, but it's based on decades of experience (mine and others'), so I hope that even seasoned programmers can learn a thing or two from watching it.

What you will learn #

How do you make your code easily usable for other people? By following the actionable, measurable rules laid forth by Command/Query Separation as well as Postel’s law.

Learn how to write maintainable software that can easily respond to changing requirements, using object-oriented design principles. First, you'll learn about the fundamental object-oriented design principle of Encapsulation, and then you'll learn about the five SOLID principles - also known as the principles of object-oriented design. There are plenty of code examples along the way; they are in C#, but written in such a way that they should be easily understandable to readers of Java, or other curly-brace-based languages.

Is OOD still relevant? #

Functional Programming is becoming increasingly popular, and regular readers of this blog will have noticed that I, myself, write a fair amount of F# code. In light of this, is OOD still relevant?

There's a lot of Object-Oriented code out there, or at least, code written in a nominally Object-Oriented language, and it's not going to go away any time soon. Getting better at maintaining and evolving Object-Oriented code is, in my opinion, still important.

Some of the principles I cover in this course are also relevant in Functional Programming.

Comments

public bool TryRead(int id, out string message)
{
    message = null;
    var path = this.GetFileName(id);
    if (!File.Exists(path))
        return false;
    message = File.ReadAllText(path);
    return true;
}

public bool TryRead(int id, out string message)
{
    
    var path = this.GetFileName(id);
    try
    {
        message = File.ReadAllText(path);
        return true;
    }
    catch
    {
        message = null;
        return false;
    }
}

A Drain is a filter abstraction over an Iterator, with the purpose of making out-of-process queries more efficient.

For some years now, the Reused Abstractions Principle has pushed me towards software architectures based upon fewer and fewer abstractions, where each abstraction, on the other hand, is reused over and over again.

In my Pluralsight course about A Functional Architecture with F#, I describe how to build an entire mainstream application based on only two abstractions:

• Iterator
• Observer

The problem #

It turns out that these two abstractions are enough to build an entire, mainstream system, but in practice, there's a performance problem. If you have only Iterators, you'll have to read all your data into memory, and then filter in memory. If you have lots of data on storage, this is obviously going to be prohibitively slow, so you'll need a way to select only a subset out of your persistent store.

This problem should be familiar to every student of software development. Pure abstractions tend not to survive contact with reality (there are examples in both Object-Oriented, Functional, and Logical or Relational programming), but we should still strive towards keeping abstractions as pure as possible.

One proposed solution to the Query Problem is to use something like IQueryable, but unfortunately, IQueryable is an extremely poor abstraction (and so are F# query expressions, too).

In my experience, the most important feature of IQueryable is the ability to filter before loading data; normally, you can perform projections in memory, but when you read from persistent storage, you need to select your desired subset before loading it into memory.

Inspired by talks by Bart De Smet, in my Pluralsight course, I define custom filter interfaces like:

type IReservations =
    inherit seq<Envelope<Reservation>>
    abstract Between : DateTime -> DateTime -> seq<Envelope<Reservation>>

or

type INotifications =
    inherit seq<Envelope<Notification>>
    abstract About : Guid -> seq<Envelope<Notification>>

Both of these interfaces derive from IEnumerable<T> and add a single extra method that defines a custom filter. Storage-aware implementations can implement this method by returning a new sequence of only those items on storage that match the filter. Such a method may

• make a SQL query against a database
• make a query against a document database
• read only some files from the file system
• etc.

Generalized interface #

The custom interfaces shown above follow a common template: the interface derives from IEnumerable<T> and adds a single 'filter' method, which filters the sequence based on the input argument(s). In the above examples, IReservations define a Between method with two arguments, while INotifications defines an About method with a single argument.

In order to generalize, it's necessary to find a common name for the interface and its single method, as well as deal with variations in method arguments.

All the really obvious names like Filter, Query, etc. are already 'taken', so I hit a thesaurus and settled on the name Drain. A Drain can potentially drain a sequence of elements to a smaller sequence.

When it comes to variations in input arguments, the solution is to use generics. The Between method that takes two arguments could also be modelled as a method taking a single tuple argument. Eventually, I've arrived at this general definition:

module Drain =
    type IDrainable<'a, 'b> =
        inherit seq<'a>
        abstract On : 'b -> seq<'a>
 
    let on x (d : IDrainable<'a, 'b>) = d.On x

As you can see, I decided to name the extra method On, as well as add an on function, which enables clients to use a Drain like this:

match tasks |> Drain.on id |> Seq.toList with

In the above example, tasks is defined as IDrainable<TaskRendition, string>, and id is a string, so the result of draining on the ID is a sequence of TaskRendition records.

Here's another example:

match statuses |> Drain.on(id, conversationId) |> Seq.toList with

Here, statuses is defined as IDrainable<string * Guid, string * string> - not the most well-designed instance, I admit: I should really introduce some well-named records instead of those tuples, but the point is that you can also drain on multiple values by using a tuple (or a record type) as the value on which to drain.

In-memory implementation #

One of the great features of Drains is that an in-memory implementation is easy, so you can add this function to the Drain module:

let ofSeq areEqual s =
    { new IDrainable<'a, 'b> with
        member this.On x = s |> Seq.filter (fun y -> areEqual y x)
        member this.GetEnumerator() = s.GetEnumerator()
        member this.GetEnumerator() = 
            (this :> 'a seq).GetEnumerator() :> System.Collections.IEnumerator }

This enables you to take any IEnumerable<T> (seq<'a>) and turn it into an in-memory Drain by supplying an equality function. Here's an example:

let private toDrainableTasks (tasks : TaskRendition seq) =
    tasks
    |> Drain.ofSeq (fun x y -> x.Id = y)

This little helper function takes a sequence of TaskRendition records and defines the equality function as a comparison on each TaskRendition record's Id property. The result is a drain that you can use to select one or more TaskRendition records based on their IDs.

I use this a lot for unit testing.

Empty Drains #

It's also easy to define an empty drain, by adding this value to the Drain module:

let empty<'a, 'b> = Seq.empty<'a> |> ofSeq (fun x (y : 'b) -> false)

Here's a usage example:

let mappedUsers = Drain.empty<UserMapped, string>

Again, this can be handy when unit testing.

Other implementations #

While the in-memory implementation is useful when unit testing, the entire purpose of the Drain abstraction is to enable various implementations to implement the On method to perform a custom selection against a well-known data source. As an example, you could imagine an implementation that translates the input arguments of the On method into a SQL query.

If you want to see examples of this, my Pluralsight course demonstrates how to implement IReservations and INotifications with various data stores - I trust you can extrapolate from those examples.

Summary #

You can base an entire mainstream application on the two abstractions of Iterator and Observer. However, the problem when it comes to Iterators is that conceptually, you'll need to iterate over all potentially relevant elements in your system - and that may be millions of records!

However impure it is to introduce a third interface into the mix, I still prefer to introduce a single generic interface, instead of multiple custom interfaces, because once you and your co-workers understand the Drain abstraction, the cognitive load is still quite low. A Drain is an Iterator with a twist, so in the end, you'll have a system built on 2½ abstractions.

P.S. 2018-06-20. While this article is a decent attempt to generalise the query side of a fundamentally object-oriented approach to software architecture, I've later realised that dependency injection, even when disguised as partial application, isn't functional. The problem is that querying a database, reading files, and so on, is essentially non-deterministic, even when no side effects are incurred. The functional approach is to altogether reject the notion of dependencies.

Comments

July-October 2014 I have some time available, if you'd like to hire me.

Since I became self-employed in 2011, I've been as busy as always, but it looks like I have some time available in the next months. If you'd like to hire me for small or big tasks, please contact me. See here for details.

Passive Attributes are Dependency Injection-friendly.

In my article about Dependency Injection-friendly frameworks, towards the end I touched on the importance of defining attributes without behaviour, but I didn't provide a constructive example of how to do this. In this article, I'll outline how to write a Dependency Injection-friendly attribute for use with ASP.NET Web API, but as far as I recall, you can do something similar with ASP.NET MVC.

Problem statement #

In ASP.NET Web API, you can adorn your Controllers and their methods with various Filter attributes, which is a way to implement cross-cutting concerns, such as authorization or error handling. The problem with this approach is that attribute instances are created by the run-time, so you can't use proper Dependency Injection (DI) patterns such as Constructor Injection. If an attribute defines behaviour (which many of the Web API attributes do), the most common attempt at writing loosely coupled code is to resort to a static Service Locator (an anti-pattern).

This again seems to lead framework designers towards attempting to make their frameworks 'DI-friendly' by introducing a Conforming Container (another anti-pattern).

The solution is simple: define attributes without behaviour.

Metering example #

Common examples of cross-cutting concerns are authentication, authorization, error handling, logging, and caching. In these days of multi-tenant on-line services, another example would be metering, so that you can bill each user based on consumption.

Imagine that you're writing an HTTP API where some actions must be metered, whereas others shouldn't. It might be nice to adorn the metered actions with an attribute to indicate this:

[Meter]
public IHttpActionResult Get(int id)

Metering is a good example of a cross-cutting concern with behaviour, because, in order to be useful, you'd need to store the metering records somewhere, so that you can bill your users based on these records.

A passive Meter attribute would simply look like this:

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class MeterAttribute : Attribute
{
}

In order to keep the example simple, this attribute defines no data, and can only be used on methods, but nothing prevents you from adding (primitive) values to it, or extend its usage to classes as well as methods.

As you can tell from the example, the MeterAttribute has no behaviour.

In order to implement a metering cross-cutting concern, you'll need to define an IActionFilter implementation, but that's a 'normal' class that can take dependencies:

public class MeteringFilter : IActionFilter
{
    private readonly IObserver<MeterRecord> observer;
 
    public MeteringFilter(IObserver<MeterRecord> observer)
    {
        if (observer == null)
            throw new ArgumentNullException("observer");
 
        this.observer = observer;
    }
 
    public Task<HttpResponseMessage> ExecuteActionFilterAsync(
        HttpActionContext actionContext,
        CancellationToken cancellationToken,
        Func<Task<HttpResponseMessage>> continuation)
    {
        var meterAttribute = actionContext
            .ActionDescriptor
            .GetCustomAttributes<MeterAttribute>()
            .SingleOrDefault();
 
        if (meterAttribute == null)
            return continuation();
 
        var operation = actionContext.ActionDescriptor.ActionName;
        var user = actionContext.RequestContext.Principal.Identity.Name;
        var started = DateTimeOffset.Now;
        return continuation().ContinueWith(t =>
            {
                var completed = DateTimeOffset.Now;
                var duration = completed - started;
                var record = new MeterRecord
                {
                    Operation = operation,
                    User = user,
                    Started = started,
                    Duration = duration
                };
                this.observer.OnNext(record);
                return t.Result;
            });
 
    }
 
    public bool AllowMultiple
    {
        get { return true; }
    }
}

This MeteringFilter class implements IActionFilter. It looks for the [Meter] attribute. If it doesn't find the attribute on the method, it immediately returns; otherwise, it starts collecting data about the invoked action:

1. From actionContext.ActionDescriptor it retrieves the name of the operation. If you try this out for yourself, you may find that ActionName alone doesn't provide enough information to uniquely identify the method - it basically just contains the value "Get". However, the actionContext contains enough information about the action that you can easily build up a better string; I just chose to skip doing that in order to keep the example simple.
2. From actionContext.RequestContext.Principal you can get information about the current user. In order to be useful, the user must be authenticated, but if you need to meter the usage of your service, you'll probably not allow anonymous access.
3. Before invoking the continuation, the MeteringFilter records the current time.
4. After the continuation has completed, the MeteringFilter again records the current time and calculates the duration.
5. Finally, it publishes a MeterRecord to an injected dependency.

Configuring the service #

MeteringFilter is a normal class with behaviour, which you can register as a cross-cutting concern in your Web API service as easily as this:

var filter = new MeteringFilter(observer);
config.Filters.Add(filter);

where observer is your preferred implementation of IObserver<MeterRecord>. This example illustrates the Pure DI approach, but if you rather prefer to resolve MeteringFilter with your DI Container of choice, you can obviously do this as well.

The above code typically goes into your Global.asax file, or at least a class directly or indirectly invoked from Application_Start. This constitutes (part of) the Composition Root of your service.

Summary #

Both ASP.NET Web API and ASP.NET MVC supports cross-cutting concerns in the shape of filters that you can add to the service. Such filters can look for passive attributes in order to decide whether or not to trigger. The advantage of this approach is that you can use normal Constructor Injection with these filters, which completely eliminates the need for a Service Locator or Conforming Container.

The programming model remains the same as with active attributes: if you want a particular cross-cutting concern to apply to a particular method or class, you adorn it with the appropriate attribute. Passive attributes have all the advantages of active attributes, but none of the disadvantages.

Comments

var meterAttribute = actionContext
    .ControllerContext
    .ControllerDescriptor
    .GetCustomAttributes<MeterAttribute>()
    .SingleOrDefault();

Adding a Raygun error handler to ASP.NET Web API is easy.

In my Pluralsight course on a Functional Architecture with F#, I show you how to add a global error handler to an ASP.NET Web API site. The error handler you see in the course just saves the error as a text file in Azure BLOB storage, which is fine for a demo. For production software, though, you may want something a bit more sophisticated, like Raygun.

Here's how to add a Raygun exception filter to an ASP.NET Web API site:

let raygunHandler = { new System.Web.Http.Filters.IExceptionFilter with 
    member this.AllowMultiple = true
    member this.ExecuteExceptionFilterAsync(actionExecutedContext, cancellationToken) =
        let raygunKey = CloudConfigurationManager.GetSetting "raygunKey"
        let raygunClient = Mindscape.Raygun4Net.RaygunClient raygunKey
 
        System.Threading.Tasks.Task.Factory.StartNew(
            fun () -> raygunClient.Send actionExecutedContext.Exception) }

This creates an Adapter from the ASP.NET Web API IExceptionFilter interface to a RaygunClient instance. As you can see, I use CloudConfigurationManager.GetSetting to get the Raygun API key from the configuration store.

The only remaining step is to add the error handler to an HttpConfiguration instance:

config.Filters.Add raygunHandler

That's it. Now you can use the Raygun service to manage your errors.

Pure DI is Dependency Injection without a DI Container.

TL;DR: the term Pure DI replaces the term Poor Man's DI.

This post essentially proposes a change of terminology. In my book about Dependency Injection (DI), I was careful to explain the principles and patterns of DI in the pure form, without involving DI Containers. Only in Part 4 do you get extensive coverage of various DI Containers, and even here, what you learn is how the DI principles and patterns map to the various containers.

DI is a set of principles and patterns; DI Containers are optional helper libraries.

However, when I wrote the book, I made a mistake (I probably made many, but here, I'll address a single, specific mistake): in the book, DI without DI Containers is called Poor Man's DI. There are reasons for that, but eventually, I've learned that Poor Man's DI is poor terminology (pun intended). The problem is that it sounds slightly derogatory, or at least unattractive; it also doesn't communicate the message that DI without a DI Container is, in many cases, better than DI with a DI Container - on the contrary, it sounds like it's not as good.

Apart from my book, I've attempted to describe the trade-off involved in going from Poor Man's DI to using a DI Container in various articles:

• When to use a DI Container
• Compile Time Lifetime Matching
• Succeeding with Dependency Injection

For these reasons, from now on, I'll retire the term Poor Man's DI, and instead start using the term Pure DI. Pure DI is when you use the DI principles and patterns, but not a DI Container; it's what I've been doing for the last 1½ years, as well as many years before I wrote my book.

P.S. 2018-04-15. Naming is hard. When I came up with the alternative name of Pure DI, I was already in the process of changing my focus to functional programming, but at that time, I'd still not realised how big a role referential transparency plays in strictly functional languages like Haskell, with its emphasis on pure functions.

The purity implied by Pure DI, unfortunately, has nothing to do with purity in the functional sense of the word. In fact, DI makes everything impure.

When I decided on the term Pure DI, I did it because it sounds vaguely like Poor Man's DI, so that it'd be easy to remember. Additionally, I picked the word pure because it can mean essence, and I thought that, in a sense, Pure DI can be viewed as the Platonic ideal of DI. Finally, pure sounds like something desirable, so because of those reasons, I believed that it'd be a good term. People seem to have picked it up, so in that sense I think that I chose the right word, but it can be confusing if viewed through the lens of functional programming terminology.

Pure DI has nothing to do with pure functions.

When using hand-coded object composition, the compiler can help you match service lifetimes.

In my previous post, you learned how easy it is to accidentally misconfigure a DI Container to produce Captive Dependencies, which are dependencies that are being kept around after they should have been released. This can lead to subtle or catastrophic bugs.

This problem is associated with DI Containers, because Container registration APIs let you register services out of order, and with any particular lifestyle you'd like:

var builder = new ContainerBuilder();
builder.RegisterType<ProductService>().SingleInstance();
builder.RegisterType<CommerceContext>().InstancePerDependency();
builder.RegisterType<SqlProductRepository>().As<IProductRepository>()
    .InstancePerDependency();
var container = builder.Build();

In this Autofac example, CommerceContext is registered before SqlProductRepository, even though SqlProductRepository is a 'higher-level' service, but ProductService is registered first, and it's even 'higher-level' than SqlProductRepository. A DI Container doesn't care; it'll figure it out.

The compiler doesn't care if the various lifetime configurations make sense. As you learned in my previous article, this particular configuration combination doesn't make sense, but the compiler can't help you.

Compiler assistance #

The overall message in my Poka-yoke Design article series is that you can often design your types in such a way that they are less forgiving of programming mistakes; this enables the compiler to give you feedback faster than you could otherwise have gotten feedback.

If, instead of using a DI Container, you'd simply hand-code the required object composition (also called Poor Man's DI in my book, but now called Pure DI), the compiler will make it much harder for you to mismatch object lifetimes. Not impossible, but more difficult.

As an example, consider a web-based Composition Root. Here, the particular IHttpControllerActivator interface belongs to ASP.NET Web API, but it could be any Composition Root:

public class SomeCompositionRoot : IHttpControllerActivator
{
    // Singleton-scoped services are declared here...
    private readonly SomeThreadSafeService singleton;
 
    public SomeCompositionRoot()
    {
        // ... and (Singleton-scoped services) are initialised here.
        this.singleton = new SomeThreadSafeService();
    }
 
    public IHttpController Create(
        HttpRequestMessage request,
        HttpControllerDescriptor controllerDescriptor,
        Type controllerType)
    {
        // Per-Request-scoped services are declared and initialized here
        var perRequestService = new SomeThreadUnsafeService();
 
        if(controllerType == typeof(FooController))
        {
            // Transient services are created and directly injected into
            // FooController here:
            return new FooController(
                new SomeServiceThatMustBeTransient(),
                new SomeServiceThatMustBeTransient());
        }
 
        if(controllerType == typeof(BarController))
        {
            // Transient service is created and directly injected into
            // BarController here, but Per-Request-scoped services or
            // Singleton-scoped services can be used too.
            return new BarController(
                this.singleton,
                perRequestService,
                perRequestService,
                new SomeServiceThatMustBeTransient());
        }
 
        throw new ArgumentException("Unexpected type!", "controllerType");
    }
}

Notice the following:

• There's only going to be a single instance of the SomeCompositionRoot class around, so any object you assign to a readonly field is effectively going to be a Singleton.
• The Create method is invoked for each request, so if you create objects at the beginning of the Create method, you can reuse them as much as you'd like, but only within that single request. This means that even if you have a service that isn't thread-safe, it's safe to create it at this time. In the example, the BarController depends on two arguments where the Per-Request Service fits, and the instance can be reused. This may seem contrived, but isn't at all if SomeThreadUnsafeService implements more that one (Role) interface.
• If you need to make a service truly Transient (i.e. it must not be reused at all), you can create it within the constructor of its client. You see an example of this when returning the FooController instance: this example is contrived, but it makes the point: for some unfathomable reason, FooController needs two instances of the same type, but the SomeServiceThatMustBeTransient class must never be shared. It's actually quite rare to have this requirement, but it's easy enough to meet it, if you encounter it.

Commerce example #

In the previous article, you saw how easy it is to misconfigure a ProductService, because you'd like it to be a Singleton. When you hand-code the composition, it becomes much easier to spot the mistake. You may start like this:

public class CommerceCompositionRoot : IHttpControllerActivator
{
    private readonly ProductService productService;
 
    public CommerceCompositionRoot()
    {
        this.productService = new ProductService();
    }
 
    public IHttpController Create(
        HttpRequestMessage request,
        HttpControllerDescriptor controllerDescriptor,
        Type controllerType)
    {
        // Implementation follows here...
    }
}

Fortunately, that doesn't even compile, because ProductService doesn't have a parameterless constructor. With a DI Container, you could define ProductService as a Singleton without a compilation error:

var container = new StandardKernel();
container.Bind<ProductService>().ToSelf().InSingletonScope();

If you attempt to do the same with hand-coded composition, it doesn't compile. This is an excellent example of Poka-Yoke Design: design your system in such a way that the compiler can give you as much feedback as possible.

Intellisense will tell you that ProductService has dependencies, so your next step may be this:

public CommerceCompositionRoot()
{
    this.productService = 
        new ProductService(
            new SqlProductRepository(
                new CommerceContext())); // Alarm bell!
}

This will compile, but at this point, an alarm bell should go off. You know that you mustn't share CommerceContext across threads, but you're currently creating a single instance. Now it's much clearer that you're on your way to doing something wrong. In the end, you realise, simply by trial and error, that you can't make any part of the ProductService sub-graph a class field, because the leaf node (CommerceContext) isn't thread-safe.

Armed with that knowledge, the next step is to create the entire object graph in the Create method, because that's the only safe implementation left:

public IHttpController Create(
    HttpRequestMessage request,
    HttpControllerDescriptor controllerDescriptor,
    Type controllerType)
{
    if(controllerType == typeof(HomeController))
    {
        return new HomeController(
            new ProductService(
                new SqlProductRepository(
                    new CommerceContext())));
    }
 
    // Handle other controller types here...
 
    throw new ArgumentException("Unexpected type!", "controllerType");
}

In this example, you create the object graph in a single statement, theoretically giving all services the Transient lifestyle. In practice, there's no difference between the Per Request and the Transient lifestyle as long as there's only a single instance of each service for each object graph.

Concluding remarks #

Some time ago, I wrote an article on when to use a DI Container. In that article, I attempted to explain how going from Pure DI (hand-coded composition) to a DI Container meant loss of compile-time safety, but I may have made an insufficient job of providing enough examples of this effect. The Captive Dependency configuration error, and this article together, describe one such effect: with Pure DI, lifetime matching is compiler-assisted, but if you refactor to use a DI Container, you lose the compiler's help.

Since I wrote the article on when to use a DI Container, I've only strengthened my preference for Pure DI. Unless I'm writing a very complex code base that could benefit from Convention over Configuration, I don't use a DI Container, but since I explicitly architect my systems to be non-complex these days, I haven't used a DI Container in production code for more than 1½ years.

Comments

public class AutofacControllerFactory : DefaultControllerFactory
{
    private readonly IContainer container;
    private Dictionary<IController, ILifetimeScope> scopes = new Dictionary<IController, ILifetimeScope>();

    public AutofacControllerFactory()
    {
        var builder = new ContainerBuilder();
        RegisterRootTypes(builder);

        this.container = builder.Build();
    }

    private void RegisterRootTypes(ContainerBuilder builder)
    {
        builder.RegisterType<ProductService>().SingleInstance();

        builder.RegisterAssemblyTypes(GetType().Assembly)
            .Where(t => t.Name.Contains("Controller"))
            .InstancePerLifetimeScope();
    }
                
    protected internal override IController GetControllerInstance(RequestContext requestContext, Type controllerType)
    {
        var requestScope = container.BeginLifetimeScope(RegisterRequestTypes);
        var controller = (IController)requestScope.Resolve(controllerType);
        scopes[controller] = requestScope;
        return controller;
    }

    private void RegisterRequestTypes(ContainerBuilder builder)
    {
        builder.RegisterType<CommerceContext>().InstancePerDependency();
        builder.RegisterType<SqlProductRepository>().As<IProductRepository>()
            .InstancePerDependency();
    }

    public override void ReleaseController(IController controller)
    {
        scopes[controller].Dispose();
        scopes.Remove(controller);
    }
}
          

A Captive Dependency is a dependency with an incorrectly configured lifetime. It's a typical and dangerous DI Container configuration error.

This post is the sixth in a series about Poka-yoke Design.

When you use a Dependency Injection (DI) Container, you should configure it according to the Register Resolve Release pattern. One aspect of configuration is to manage the lifetime of various services. If you're not careful, though, you may misconfigure lifetimes in such a way that a longer-lived service holds a shorter-lived service captive - often with subtle, but disastrous results. You could call this misconfiguration a Captive Dependency.

A major step in applying DI is to compose object graphs, and service lifetimes in object graphs are hierarchical:

This figure illustrates the configured and effective lifetimes of an object graph. Node A1 should have a Transient lifetime, which is certainly possible. A new instance of C1 should be created Per Request (if the object graph is part of a web application), which is also possible, because A1 has a shorter lifetime than Per Request. Similarly, only a single instance of B3 should ever be created, which is also possible, because the various instances of C1 can reuse the same B3 instance.

The A2 node also has a Singleton lifetime, which means that only a single instance should exist of this object. Because A2 holds references to B1 and A3, these two object are also effectively Singletons. It doesn't matter how you'd like the lifetimes of B1 and A3 to be: the fact is that the single instance of A2 holds on to its injected instances of B1 and A3 means that these instances are going to stick around as long as A2. This effect is transitive, so A2 also causes B2 to have an effective Singleton lifetime.

This can be problematic if, for example, B1, A3, or B2 aren't thread-safe.

Commerce example #

This may make more sense if you see this in a more concrete setting than just an object graph with A1, A2, B1, etc. nodes, so consider the introductory example from my book. It has a ProductService, which depends on an IProductRepository interface (actually, in the book, the Repository is an Abstract Base Class):

public class ProductService
{
    private readonly IProductRepository repository;
 
    public ProductService(IProductRepository repository)
    {
        this.repository = repository;
    }
 
    // Other members go here...
}

One implementation of IProductRepository is SqlProductRepository, which itself depends on an Entity Framework context:

public class SqlProductRepository : IProductRepository
{
    private readonly CommerceContext context;
 
    public SqlProductRepository(CommerceContext context)
    {
        this.context = context;
    }
 
    // IProductRepository members go here...
}

The CommerceContext class derives from the Entity Framework DbContext class, which, last time I looked, isn't thread-safe. Thus, when used in a web application, it's very important to create a new instance of the CommerceContext class for every request, because otherwise you may experience errors. What's worse is that these errors will be threading errors, so you'll not discover them when you test your web application on your development machine, but when in production, you'll have multiple concurrent requests, and then the application will crash (or perhaps 'just' lose data, which is even worse).

(As a side note I should point out that I've used neither Entity Framework nor the Repository pattern for years now, but the example explains the problem well, in a context familiar to most people.)

The ProductService class is a stateless service, and therefore thread-safe, so it's an excellent candidate for the Singleton lifestyle. However, as it turns out, that's not going to work.

NInject example #

If you want to configure ProductService and its dependencies using Ninject, you might accidentally do something like this:

var container = new StandardKernel();
container.Bind<ProductService>().ToSelf().InSingletonScope();
container.Bind<IProductRepository>().To<SqlProductRepository>();

With Ninject you don't need to register concrete types, so there's no reason to register the CommerceContext class; it wouldn't be necessary to register the ProductService either, if it wasn't for the fact that you'd like it to have the Singleton lifestyle. Ninject's default lifestyle is Transient, so that's the lifestyle of both SqlProductRepository and CommerceContext.

As you've probably already predicted, the Singleton lifestyle of ProductService captures both the direct dependency IProductRepository, and the indirect dependency CommerceContext:

var actual1 = container.Get<ProductService>();
var actual2 = container.Get<ProductService>();
 
// You'd want this assertion to pass, but it fails
Assert.NotEqual(actual1.Repository, actual2.Repository);

The repositories are the same because actual1 and actual2 are the same instance, so naturally, their constituent components are also the same.

This is problematic because CommerceContext (deriving from DbContext) isn't thread-safe, so if you resolve ProductService from multiple concurrent requests (which you could easily do in a web application), you'll have a problem.

The immediate fix is to make this entire sub-graph Transient:

var container = new StandardKernel();
container.Bind<ProductService>().ToSelf().InTransientScope();
container.Bind<IProductRepository>().To<SqlProductRepository>();

Actually, since Transient is the default, stating the lifetime is redundant, and can be omitted:

var container = new StandardKernel();
container.Bind<ProductService>().ToSelf();
container.Bind<IProductRepository>().To<SqlProductRepository>();

Finally, since you don't have to register concrete types with Ninject, you can completely omit the ProductService registration:

var container = new StandardKernel();
container.Bind<IProductRepository>().To<SqlProductRepository>();

This works:

var actual1 = container.Get<ProductService>();
var actual2 = container.Get<ProductService>();
 
Assert.NotEqual(actual1.Repository, actual2.Repository);

While the Captive Dependency error is intrinsically tied to using a DI Container, it's by no means particular to Ninject.

Autofac example #

It would be unfair to leave you with the impression that this problem is a problem with Ninject; it's not. All DI Containers I know of have this problem. Autofac is just another example.

Again, you'd like ProductService to have the Singleton lifestyle, because it's thread-safe, and it would be more efficient that way:

var builder = new ContainerBuilder();
builder.RegisterType<ProductService>().SingleInstance();
builder.RegisterType<SqlProductRepository>().As<IProductRepository>();
builder.RegisterType<CommerceContext>();
var container = builder.Build();

Like Ninject, the default lifestyle for Autofac is Transient, so you don't have to explicitly configure the lifetimes of SqlProductRepository or CommerceContext. On the other hand, Autofac requires you to register all services in use, even when they're concrete classes; this is the reason you see a registration statement for CommerceContext as well.

The problem is exactly the same as with Ninject:

var actual1 = container.Resolve<ProductService>();
var actual2 = container.Resolve<ProductService>();
 
// You'd want this assertion to pass, but it fails
Assert.NotEqual(actual1.Repository, actual2.Repository);

The reason is the same as before, as is the solution:

var builder = new ContainerBuilder();
builder.RegisterType<ProductService>();
builder.RegisterType<SqlProductRepository>().As<IProductRepository>();
builder.RegisterType<CommerceContext>();
var container = builder.Build();
 
var actual1 = container.Resolve<ProductService>();
var actual2 = container.Resolve<ProductService>();
 
Assert.NotEqual(actual1.Repository, actual2.Repository);

Notice that, because the default lifetime is Transient, you don't have to state it while registering any of the services.

Concluding remarks #

You can re-create this problem with any major DI Container. The problem isn't associated with any particular DI Container, but simply the fact that there are trade-offs associated with using a DI Container, and one of the trade-offs is a reduction in compile-time feedback. The way typical DI Container registration APIs work, they can't easily detect this lifetime configuration mismatch.

It's been a while since I last did a full survey of the .NET DI Container landscape, and back then (when I wrote my book), no containers could detect this problem. Since then, I believe Castle Windsor has got some Captive Dependency detection built in, but I admit that I'm not up to speed; other containers may have this feature as well.

When I wrote my book some years ago, I considered including a description of the Captive Dependency configuration error, but for various reasons, it never made it into the book:

• As far as I recall, it was Krzysztof Koźmic who originally made me aware of this problem. In emails, we debated various ideas for a name, but we couldn't really settle on something catchy. Since I don't like to describe something I can't name, it never really made it into the book.
• One of the major goals of the book was to explain DI as a set of principles and patterns decoupled from DI Containers. The Captive Dependency problem is specifically associated with DI Containers, so it didn't really fit into the book.

In a follow-up post to this, I'll demonstrate why you don't have the same problem when you hand-code your object graphs.

Comments

var container = new Container();
container.RegisterSingle<ProductService>();
container.Register<IProductRepository, SqlProductRepository>();
container.Register<CommerceContext>();

container.Verify();
var results = Analyzer.Analyze(container);
Assert.That(results[0].Description, Is.StringContaining("CaptiveDependency"));
		

ASP.NET vNext includes a Dependency Injection API. This post offers feedback on the currently available code.

As part of Microsoft's new openness, the ASP.NET team have made the next version of ASP.NET available on GitHub. Obviously, it's not yet done, but I suppose that the reasons for this move is to get early feedback, as well as perhaps take contributions. This is an extremely positive move for the ASP.NET team, and I'm very grateful that they have done this, because it enables me to provide early feedback, and offer my help.

It looks like one of the proposed new features of the next version of ASP.NET is a library or API simply titled Dependency Injection. In this post, I will provide feedback to the team on that particular sub-project, in the form of an open blog post. The contents of this blog post is also cross-posted to the official ASP.NET vNext forum.

Dependency Injection support #

The details on the motivation for the Dependency Injection library are sparse, but I assume that the purpose is provide 'Dependency Injection support' to ASP.NET. If so, that motivation is laudable, because Dependency Injection (DI) is the proper way to write loosely coupled code when using Object-Oriented Design.

Some parts of the ASP.NET family already have DI support; personally, I'm most familiar with ASP.NET MVC and ASP.NET Web API. Other parts have proven rather hostile towards DI - most notably ASP.NET Web Forms. The problem with Web Forms is that Constructor Injection is impossible, because the Web Forms framework doesn't provide a hook for creating new Page objects.

My interpretation #

As far as I can tell, the current ASP.NET Dependency Injection code defines an interface for creating objects:

public interface ITypeActivator
{
    object CreateInstance(
        IServiceProvider services,
        Type instanceType,
        params object[] parameters);
}

In addition to this central interface, there are other interfaces that enable you to configure a 'service collection', and then there are Adapters for

• Autofac
• Ninject
• StructureMap
• Unity
• Caste Windsor

My recommendations #

It's an excellent idea to add 'Dependency Injection support' to ASP.NET, for the few places where it's not already present. However, as I've previously explained, a Conforming Container isn't the right solution. The right solution is to put the necessary extensibility points into the framework:

• ASP.NET MVC already has a good extensibility point in the IControllerFactory interface. I recommend keeping this interface, and other interfaces in MVC that play a similar role.
• ASP.NET Web API already has a good extensibility point in the IHttpControllerActivator interface. I recommend keeping this interface, and other interfaces in Web API that play a similar role.
• ASP.NET Web Forms have no extensibility point that enables you to create custom Page objects. I recommend adding an IPageFactory interface, as described in my article about DI-Friendly frameworks. Other object types related to Web Forms, such as Object Data Sources, suffer from the same shortcoming, and should have similar factory interfaces.
• There may be other parts of ASP.NET with which I'm not particularly familiar (SignalR?), but they should all follow the same pattern of defining Abstract Factories for user classes, in the cases where these don't already exist.

"perfection is attained not when there is nothing more to add, but when there is nothing more to remove." - Antoine de Saint Exupéry