ploeh blog danish software design

Use a piece of middleware to enrich a Data Transfer Object. An ASP.NET Core example.

When developing true REST APIs, you should use hypermedia controls (i.e. links) to guide clients to the resources they need. I've always felt that the code that generates these links tends to make otherwise readable Controller methods unreadable.

I'm currently experimenting with generating links as a cross-cutting concern. So far, I like it very much.

The code shown here is part of the sample code base that accompanies my book Code That Fits in Your Head.

Links from home #

Consider an online restaurant reservation system. When you make a GET request against the home resource (which is the only published URL for the API), you should receive a representation like this:

{
  "links": [
    {
      "rel": "urn:reservations",
      "href": "http://localhost:53568/reservations"
    },
    {
      "rel": "urn:year",
      "href": "http://localhost:53568/calendar/2020"
    },
    {
      "rel": "urn:month",
      "href": "http://localhost:53568/calendar/2020/8"
    },
    {
      "rel": "urn:day",
      "href": "http://localhost:53568/calendar/2020/8/13"
    }
  ]
}

As you can tell, my example just runs on my local development machine, but I'm sure that you can see past that. There's three calendar links that clients can use to GET the restaurant's calendar for the current day, month, or year. Clients can use these resources to present a user with a date picker or a similar user interface so that it's possible to pick a date for a reservation.

When a client wants to make a reservation, it can use the URL identified by the rel (relationship type) "urn:reservations" to make a POST request.

Link generation as a Controller responsibility #

I first wrote the code that generates these links directly in the Controller class that serves the home resource. It looked like this:

public IActionResult Get()
{
    var links = new List<LinkDto>();
    links.Add(Url.LinkToReservations());
    if (enableCalendar)
    {
        var now = DateTime.Now;
        links.Add(Url.LinkToYear(now.Year));
        links.Add(Url.LinkToMonth(now.Year, now.Month));
        links.Add(Url.LinkToDay(now.Year, now.Month, now.Day));
    }
    return Ok(new HomeDto { Links = links.ToArray() });
}

That doesn't look too bad, but 90% of the code is exclusively concerned with generating links. (enableCalendar, by the way, is a feature flag.) That seems acceptable in this special case, because there's really nothing else the home resource has to do. For other resources, the Controller code might contain some composition code as well, and then all the link code starts to look like noise that makes it harder to understand the actual purpose of the Controller method. You'll see an example of a non-trivial Controller method later in this article.

It seemed to me that enriching a Data Transfer Object (DTO) with links ought to be a cross-cutting concern.

LinksFilter #

In ASP.NET Core, you can implement cross-cutting concerns with a type of middleware called IAsyncActionFilter. I added one called LinksFilter:

internal class LinksFilter : IAsyncActionFilter
{
    private readonly bool enableCalendar;
 
    public IUrlHelperFactory UrlHelperFactory { get; }
 
    public LinksFilter(
        IUrlHelperFactory urlHelperFactory,
        CalendarFlag calendarFlag)
    {
        UrlHelperFactory = urlHelperFactory;
        enableCalendar = calendarFlag.Enabled;
    }
 
    public async Task OnActionExecutionAsync(
        ActionExecutingContext context,
        ActionExecutionDelegate next)
    {
        var ctxAfter = await next().ConfigureAwait(false);
        if (!(ctxAfter.Result is OkObjectResult ok))
            return;
 
        var url = UrlHelperFactory.GetUrlHelper(ctxAfter);
        switch (ok.Value)
        {
            case HomeDto homeDto:
                AddLinks(homeDto, url);
                break;
            case CalendarDto calendarDto:
                AddLinks(calendarDto, url);
                break;
            default:
                break;
        }
    }
 
    // ...

There's only one method to implement. If you want to run some code after the Controllers have had their chance, you invoke the next delegate to get the resulting context. It should contain the response to be returned. If Result isn't an OkObjectResult there's no content to enrich with links, so the method just returns.

Otherwise, it switches on the type of the ok.Value and passes the DTO to an appropriate helper method. Here's the AddLinks overload for HomeDto:

private void AddLinks(HomeDto dto, IUrlHelper url)
{
    if (enableCalendar)
    {
        var now = DateTime.Now;
        dto.Links = new[]
        {
            url.LinkToReservations(),
            url.LinkToYear(now.Year),
            url.LinkToMonth(now.Year, now.Month),
            url.LinkToDay(now.Year, now.Month, now.Day)
        };
    }
    else
    {
        dto.Links = new[] { url.LinkToReservations() };
    }
}

You can probably recognise the implemented behaviour from before, where it was implemented in the Get method. That method now looks like this:

public ActionResult Get()
{
    return new OkObjectResult(new HomeDto());
}

That's clearly much simpler, but you probably think that little has been achieved. After all, doesn't this just move some code from one place to another?

Yes, that's the case in this particular example, but I wanted to start with an example that was so simple that it highlights how to move the code to a filter. Consider, then, the following example.

A calendar resource #

The online reservation system enables clients to navigate its calendar to look up dates and time slots. A representation might look like this:

{
  "links": [
    {
      "rel": "previous",
      "href": "http://localhost:53568/calendar/2020/8/12"
    },
    {
      "rel": "next",
      "href": "http://localhost:53568/calendar/2020/8/14"
    }
  ],
  "year": 2020,
  "month": 8,
  "day": 13,
  "days": [
    {
      "links": [
        {
          "rel": "urn:year",
          "href": "http://localhost:53568/calendar/2020"
        },
        {
          "rel": "urn:month",
          "href": "http://localhost:53568/calendar/2020/8"
        },
        {
          "rel": "urn:day",
          "href": "http://localhost:53568/calendar/2020/8/13"
        }
      ],
      "date": "2020-08-13",
      "entries": [
        {
          "time": "18:00:00",
          "maximumPartySize": 10
        },
        {
          "time": "18:15:00",
          "maximumPartySize": 10
        },
        {
          "time": "18:30:00",
          "maximumPartySize": 10
        },
        {
          "time": "18:45:00",
          "maximumPartySize": 10
        },
        {
          "time": "19:00:00",
          "maximumPartySize": 10
        },
        {
          "time": "19:15:00",
          "maximumPartySize": 10
        },
        {
          "time": "19:30:00",
          "maximumPartySize": 10
        },
        {
          "time": "19:45:00",
          "maximumPartySize": 10
        },
        {
          "time": "20:00:00",
          "maximumPartySize": 10
        },
        {
          "time": "20:15:00",
          "maximumPartySize": 10
        },
        {
          "time": "20:30:00",
          "maximumPartySize": 10
        },
        {
          "time": "20:45:00",
          "maximumPartySize": 10
        },
        {
          "time": "21:00:00",
          "maximumPartySize": 10
        }
      ]
    }
  ]
}

This is a JSON representation of the calendar for August 13, 2020. The data it contains is the identification of the date, as well as a series of entries that lists the largest reservation the restaurant can accept for each time slot.

Apart from the data, the representation also contains links. There's a general collection of links that currently holds only next and previous. In addition to that, each day has its own array of links. In the above example, only a single day is represented, so the days array contains only a single object. For a month calendar (navigatable via the urn:month link), there'd be between 28 and 31 days, each with its own links array.

Generating all these links is a complex undertaking all by itself, so separation of concerns is a boon.

Calendar links #

As you can see in the above LinksFilter, it branches on the type of value wrapped in an OkObjectResult. If the type is CalendarDto, it calls the appropriate AddLinks overload:

private static void AddLinks(CalendarDto dto, IUrlHelper url)
{
    var period = dto.ToPeriod();
    var previous = period.Accept(new PreviousPeriodVisitor());
    var next = period.Accept(new NextPeriodVisitor());
 
    dto.Links = new[]
    {
        url.LinkToPeriod(previous, "previous"),
        url.LinkToPeriod(next, "next")
    };
 
    if (dto.Days is { })
        foreach (var day in dto.Days)
            AddLinks(day, url);
}

It both generates the previous and next links on the dto, as well as the links for each day. While I'm not going to bore you with more of that code, you can tell, I hope, that the AddLinks method calls other helper methods and classes. The point is that link generation involves more than just a few lines of code.

You already saw that in the first example (related to HomeDto). The question is whether there's still some significant code left in the Controller class?

Calendar resource #

The CalendarController class defines three overloads of Get - one for a single day, one for a month, and one for an entire year. Each of them looks like this:

public async Task<ActionResult> Get(int year, int month)
{
    var period = Period.Month(year, month);
    var days = await MakeDays(period).ConfigureAwait(false);
    return new OkObjectResult(
        new CalendarDto
        {
            Year = year,
            Month = month,
            Days = days
        });
}

It doesn't look as though much is going on, but at least you can see that it returns a CalendarDto object.

While the method looks simple, it's not. Significant work happens in the MakeDays helper method:

private async Task<DayDto[]> MakeDays(IPeriod period)
{
    var firstTick = period.Accept(new FirstTickVisitor());
    var lastTick = period.Accept(new LastTickVisitor());
    var reservations = await Repository
        .ReadReservations(firstTick, lastTick).ConfigureAwait(false);
 
    var days = period.Accept(new DaysVisitor())
        .Select(d => MakeDay(d, reservations))
        .ToArray();
    return days;
}

After having read relevant reservations from the database, it applies complex business logic to allocate them and thereby being able to report on remaining capacity for each time slot.

Not having to worry about link generation while doing all that work seems like a benefit.

Filter registration #

You must tell the ASP.NET Core framework about any filters that you add. You can do that in the Startup class' ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers(opts => opts.Filters.Add<LinksFilter>());
 
    // ...

When registered, the filter executes for each HTTP request. When the object represents a 200 OK result, the filter populates the DTOs with links.

Conclusion #

By treating RESTful link generation as a cross-cutting concern, you can separate if from the logic of generating the data structure that represents the resource. That's not the only way to do it. You could also write a simple function that populates DTOs, and call it directly from each Controller action.

What I like about using a filter is that I don't have to remember to do that. Once the filter is registered, it'll populate all the DTOs it knows about, regardless of which Controller generated them.

Comments

public class LinkDescriptor
{
    public string Rel { get; set; }
    public string Href { get; set; }
    public string Resource { get; set; }
}

public ActionResult Get()
{
    return new OkObjectResult(new HomeDto());
}

public static void RegisterLinks(LinkSubscriber subscriber)
{
    subscriber
        .Add(
            resource: "/Home",
            rel: "urn:reservations",
            href: "/reservations")
        .Add(
            resource: "/Home",
            rel: "urn:year",
            href: "/calendar/2020")
        .Add(
            resource: "/Home",
            rel: "urn:month",
            href: "/calendar/2020/8")
        .Add(
            resource: "/Home",
            rel: "urn:day",
            href: "/calendar/2020/8/13");
}

public static class MvcBuilderExtensions
{
    public static IMvcBuilder AddRestLinks(this IMvcBuilder mvcBuilder)
    {
        var subscriber = new LinkSubscriber();

        var linkRegistrationMethods = GetLinkRegistrationMethods(mvcBuilder.Services);

        PopulateLinks(subscriber, linkRegistrationMethods);

        mvcBuilder.Services.AddSingleton<IEnumerable<LinkDescriptor>>(subscriber.LinkDescriptors);

        return mvcBuilder;
    }

    private static List<MethodInfo> GetLinkRegistrationMethods(IServiceCollection services)
    {
        return typeof(MvcBuilderExtensions).Assembly.ExportedTypes
            .Where(tp => typeof(ControllerBase).IsAssignableFrom(tp))
            .Select(tp => tp.GetMethod("RegisterLinks", new[] { typeof(LinkSubscriber) }))
            .Where(mi => mi != null)
            .ToList();
    }

    private static void PopulateLinks(LinkSubscriber subscriber, List<MethodInfo> linkRegistrationMethods)
    {
        foreach (var method in linkRegistrationMethods)
        {
            method.Invoke(null, new[] { subscriber });
        }
    }
}

public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers(conf => conf.Filters.Add<LinksFilter>())
            .AddRestLinks();
}

public class LinksFilter : IAsyncActionFilter
{
    private readonly IEnumerable<LinkDescriptor> links;

    public LinksFilter(IEnumerable<LinkDescriptor> links)
    {
        this.links = links;
    }

    public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
    {
        var relatedLinks = links
            .Where(lk => context.HttpContext.Request.Path.Value.ToLower() == lk.Resource);

        if (relatedLinks.Any())
            await ManipulateResponseAsync(context, next, relatedLinks);
    }

    private async Task ManipulateResponseAsync(ActionExecutingContext context, ActionExecutionDelegate next, IEnumerable<LinkDescriptor> relatedLinks)
    {
        var ctxAfter = await next().ConfigureAwait(false);

        if (!(ctxAfter.Result is ObjectResult objRes))
            return;

        var expandoResult = new ExpandoObject();

        FillExpandoWithResultProperties(expandoResult, objRes.Value);

        FillExpandoWithLinks(expandoResult, relatedLinks);

        objRes.Value = expandoResult;
    }

    private void FillExpandoWithResultProperties(ExpandoObject resultExpando, object value)
    {
        var properties = value.GetType().GetProperties();

        foreach (var property in properties)
        {
            resultExpando.TryAdd(property.Name, property.GetValue(value));
        }
    }

    private void FillExpandoWithLinks(ExpandoObject resultExpando, IEnumerable<LinkDescriptor> relatedLinks)
    {
        var linksToAdd = relatedLinks.Select(lk => new { Rel = lk.Rel, Href = lk.Href });

        resultExpando.TryAdd("Links", linksToAdd);
    }
}

Unit testing considered harmful? I think not.

Once in a while, some article, video, or podcast makes the rounds on social media, arguing that unit testing is bad, overrated, harmful, or the like. I'm not going to link to any specific resources, because this post isn't an attack on any particular piece of work. Many of these are sophisticated, thoughtful, and make good points, but still arrive at the wrong conclusion.

The line of reasoning tends to be to show examples of bad unit tests and conclude that, based on the examples, unit tests are bad.

The power of examples is great, but clearly, this is a logical fallacy.

Symbolisation #

In case it isn't clear that the argument is invalid, I'll demonstrate it using the techniques I've learned from Howard Pospesel's introduction to predicate logic.

We can begin by symbolising the natural-language arguments into well-formed formulas. I'll keep this as simple as possible:

∃xBx ⊢ ∀xBx
(Domain: unit tests; Bx = x is bad)

Basically, Bx states that x is bad, where x is a unit test. ∃xBx is a statement that there exists a unit test x for which x is bad; i.e. bad unit tests exist. The statement ∀xBx claims that for all unit tests x, x is bad; i.e. all unit tests are bad. The turnstile symbol ⊢ in the middle indicates that the antecedent on the left proves the consequent to the right.

Translated back to natural language, the claim is this: Because bad unit tests exist, all unit tests are bad.

You can trivially prove this sequent invalid.

Logical fallacy #

One way to prove the sequent invalid is to use a truth tree:

Briefly, the way this works is that the statements on the left-hand side represent truth, while the ones to the right are false. By placing the antecedent on the left, but the consequent on the right, you're basically assuming the sequent to be wrong. This is is also the way you prove correct sequents true; if the conclusion is assumed false, a logical truth should lead to a contradiction. If it doesn't, the sequent is invalid. That's what happens here.

The tree remains open, which means that the original sequent is invalid. It's a logical fallacy.

Counter examples #

You probably already knew that. All it takes to counter a universal assertion such as all unit tests are bad is to produce a counter example. One is sufficient, because if just a single good unit test exists, it can't be true that all are bad.

Most of the think pieces that argue that unit testing is bad do so by showing examples of bad unit tests. These tests typically involve lots of mocks and stubs; they tend to test the interaction between internal components instead of the components themselves, or the system as a whole. I agree that this often leads to fragile tests.

While I still spend my testing energy according to the Test Pyramid, I don't write unit tests like that. I rarely use dynamic mock libraries. Instead, I push impure actions to the boundary of the system and write most of the application code as pure functions, which are intrinsically testable. No test-induced damage there.

I follow up with boundary tests that demonstrate that the functions are integrated into a working system. That's just another layer in the Test Pyramid, but smaller. You don't need that many integration tests when you have a foundation of good unit tests.

While I'm currently working on a larger body of work that showcases this approach, this blog already has examples of this.

Conclusion #

You often hear or see the claim that unit tests are bad. The supporting argument is that a particular (popular, I admit) style of unit testing is bad.

If the person making this claim only knows of that single style of unit testing, it's natural to jump to the conclusion that all unit testing must be bad.

That's not the case. I write most of my unit tests in a style dissimilar from the interaction-heavy, mocks-and-stubs-based style that most people use. These test have a low maintenance burden and don't cause test-induced damage.

Unit testing is fine.

A use case for the Immutable Fluent Builder design pattern variant.

The Fluent Builder design pattern is popular in object-oriented programming. Most programmers use the mutable variant, while I favour the immutable alternative. The advantages of Immutable Fluent Builders, however, may not be immediately clear.

"I never thought of someone reusing a configured builder (soulds like too big class/SRP violation)."

It's possible that I'm wrong, but in my my recent article on Builder isomorphisms I focused on the pattern variations themselves, to the point where a convincing example wasn't my top priority.

I recently encountered a good use case for an Immutable Fluent Builder.

The code shown here is part of the sample code base that accompanies my book Code That Fits in Your Head.

Build links #

I was developing a REST API and wanted to generate some links like these:

{
  "links": [
    {
      "rel": "urn:reservations",
      "href": "http://localhost:53568/reservations"
    },
    {
      "rel": "urn:year",
      "href": "http://localhost:53568/calendar/2020"
    },
    {
      "rel": "urn:month",
      "href": "http://localhost:53568/calendar/2020/7"
    },
    {
      "rel": "urn:day",
      "href": "http://localhost:53568/calendar/2020/7/7"
    }
  ]
}

As I recently described, the ASP.NET Core Action API is tricky, and since there was some repetition, I was looking for a way to reduce the code duplication. At first I just thought I'd make a few private helper methods, but then it occurred to me that an Immutable Fluent Builder as an Adapter to the Action API might offer a fertile alternative.

UrlBuilder class #

The various Action overloads all accept null arguments, so there's effectively no clear invariants to enforce on that dimension. While I wanted an Immutable Fluent Builder, I made all the fields nullable.

public sealed class UrlBuilder
{
    private readonly string? action;
    private readonly string? controller;
    private readonly object? values;
 
    public UrlBuilder()
    {
    }
 
    private UrlBuilder(string? action, string? controller, object? values)
    {
        this.action = action;
        this.controller = controller;
        this.values = values;
    }
 
    // ...

I also gave the UrlBuilder class a public constructor and a private copy constructor. That's the standard way I implement that pattern.

Most of the modification methods are straightforward:

public UrlBuilder WithAction(string newAction)
{
    return new UrlBuilder(newAction, controller, values);
}
 
public UrlBuilder WithValues(object newValues)
{
    return new UrlBuilder(action, controller, newValues);
}

I wanted to encapsulate the suffix-handling behaviour I recently described in the appropriate method:

public UrlBuilder WithController(string newController)
{
    if (newController is null)
        throw new ArgumentNullException(nameof(newController));
 
    const string controllerSuffix = "controller";
 
    var index = newController.LastIndexOf(
        controllerSuffix,
        StringComparison.OrdinalIgnoreCase);
    if (0 <= index)
        newController = newController.Remove(index);
    return new UrlBuilder(action, newController, values);
}

The WithController method handles both the case where newController is suffixed by "Controller" and the case where it isn't. I also wrote unit tests to verify that the implementation works as intended.

Finally, a Builder should have a method to build the desired object:

public Uri BuildAbsolute(IUrlHelper url)
{
    if (url is null)
        throw new ArgumentNullException(nameof(url));
 
    var actionUrl = url.Action(
        action,
        controller,
        values,
        url.ActionContext.HttpContext.Request.Scheme,
        url.ActionContext.HttpContext.Request.Host.ToUriComponent());
    return new Uri(actionUrl);
}

One could imagine also defining a BuildRelative method, but I didn't need it.

Generating links #

Each of the objects shown above are represented by a simple Data Transfer Object:

public class LinkDto
{
    public string? Rel { get; set; }
    public string? Href { get; set; }
}

My next step was to define an extension method on Uri, so that I could turn a URL into a link:

internal static LinkDto Link(this Uri uri, string rel)
{
    return new LinkDto { Rel = rel, Href = uri.ToString() };
}

With that function I could now write code like this:

private LinkDto CreateYearLink()
{
    return new UrlBuilder()
        .WithAction(nameof(CalendarController.Get))
        .WithController(nameof(CalendarController))
        .WithValues(new { year = DateTime.Now.Year })
        .BuildAbsolute(Url)
        .Link("urn:year");
}

It's acceptable, but verbose. This only creates the urn:year link; to create the urn:month and urn:day links, I needed similar code. Only the WithValues method calls differed. The calls to WithAction and WithController were identical.

Shared Immutable Builder #

Since UrlBuilder is immutable, I can trivially define a shared instance:

private readonly static UrlBuilder calendar =
    new UrlBuilder()
        .WithAction(nameof(CalendarController.Get))
        .WithController(nameof(CalendarController));

This enabled me to write more succinct methods for each of the relationship types:

internal static LinkDto LinkToYear(this IUrlHelper url, int year)
{
    return calendar.WithValues(new { year }).BuildAbsolute(url).Link("urn:year");
}
 
internal static LinkDto LinkToMonth(this IUrlHelper url, int year, int month)
{
    return calendar.WithValues(new { year, month }).BuildAbsolute(url).Link("urn:month");
}
 
internal static LinkDto LinkToDay(this IUrlHelper url, int year, int month, int day)
{
    return calendar.WithValues(new { year, month, day }).BuildAbsolute(url).Link("urn:day");
}

This is possible exactly because UrlBuilder is immutable. Had the Builder been mutable, such sharing would have created an aliasing bug, as I previously described. Immutability enables reuse.

Conclusion #

I got my first taste of functional programming around 2010. Since then, when I'm not programming in F# or Haskell, I've steadily worked on identifying good ways to enjoy the benefits of functional programming in C#.

Immutability is a fairly low-hanging fruit. It requires more boilerplate code, but apart from that, it's easy to make classes immutable in C#, and Visual Studio has plenty of refactorings that make it easier.

Immutability is one of those features you're unlikely to realise that you're missing. When it's not there, you work around it, but when it's there, it simplifies many tasks.

The example you've seen in this article relates to the Fluent Builder pattern. At first glance, it seems as though a mutable Fluent Builder has the same capabilities as a corresponding Immutable Fluent Builder. You can, however, build upon shared Immutable Fluent Builders, which you can't with mutable Fluent Builders.

How to generate links to other resources in a refactoring-friendly way.

I recently spent a couple of hours yak-shaving, and despite much Googling couldn't find any help on the internet. I'm surprised that the following problem turned out to be so difficult to figure out, so it may just be that I'm ignorant or that my web search skills failed me that day. On the other hand, if this really is as difficult as I found it, perhaps this article can save some other poor soul an hour or two.

The code shown here is part of the sample code base that accompanies my book Code That Fits in Your Head.

I was developing a REST API and wanted to generate some links like these:

{
  "links": [
    {
      "rel": "urn:reservations",
      "href": "http://localhost:53568/reservations"
    },
    {
      "rel": "urn:year",
      "href": "http://localhost:53568/calendar/2020"
    },
    {
      "rel": "urn:month",
      "href": "http://localhost:53568/calendar/2020/7"
    },
    {
      "rel": "urn:day",
      "href": "http://localhost:53568/calendar/2020/7/7"
    }
  ]
}

Like previous incarnations of the framework, ASP.NET Core 3 has an API for generating links to a method on a Controller. I just couldn't get it to work.

Using nameof #

I wanted to generate a URL like http://localhost:53568/calendar/2020 in a refactoring-friendly way. While ASP.NET wants you to define HTTP resources as methods (actions) on Controllers, the various Action overloads want you to identify these actions and Controllers as strings. What happens if someone renames one of those methods or Controller classes?

That's what the C# nameof keyword is for.

I naively called the Action method like this:

var href = Url.Action(
    nameof(CalendarController.Get),
    nameof(CalendarController),
    new { year = DateTime.Now.Year },
    Url.ActionContext.HttpContext.Request.Scheme,
    Url.ActionContext.HttpContext.Request.Host.ToUriComponent());

Looks good, doesn't it?

I thought so, but it didn't work. In the time-honoured tradition of mainstream programming languages, the method just silently fails to return a value and instead returns null. That's not helpful. What might be the problem? No clue is provided. It just doesn't work.

Strip off the suffix #

It turns out that the Action method expects the controller argument to not contain the Controller suffix. Not surprisingly, nameof(CalendarController) becomes the string "CalendarController", which doesn't work.

It took me some time to figure out that I was supposed to pass a string like "Calendar". That works!

As a first pass at the problem, then, I changed my code to this:

var controllerName = nameof(CalendarController);
var controller = controllerName.Remove(
    controllerName.LastIndexOf(
        "Controller",
        StringComparison.Ordinal));
 
var href = Url.Action(
    nameof(CalendarController.Get),
    controller,
    new { year = DateTime.Now.Year },
    Url.ActionContext.HttpContext.Request.Scheme,
    Url.ActionContext.HttpContext.Request.Host.ToUriComponent());

That also works, and is more refactoring-friendly. You can rename both the Controller class and the method, and the link should still work.

Conclusion #

The UrlHelperExtensions.Action methods expect the controller to be the 'semantic' name of the Controller, if you will - not the actual class name. If you're calling it with values produced with the nameof keyword, you'll have to strip the Controller suffix away.

Is task asynchronous programming a substitute for the IO container? An article for C# programmers.

This article is part of an article series about the IO container in C#. In the previous articles, you've seen how a type like IO<T> can be used to distinguish between pure functions and impure actions. While it's an effective and elegant solution to the problem, it depends on a convention: that all impure actions return IO objects, which are opaque to pure functions.

In reality, .NET base class library methods don't do that, and it's unrealistic that this is ever going to happen. It'd require a breaking reset of the entire .NET ecosystem to introduce this design.

A comparable reset did, however, happen a few years ago.

TAP reset #

Microsoft introduced the task asynchronous programming (TAP) model some years ago. Operations that involve I/O got a new return type. Not IO<T>, but Task<T>.

The .NET framework team began a long process of adding asynchronous alternatives to existing APIs that involve I/O. Not as breaking changes, but by adding new, asynchronous methods side-by-side with older methods. ExecuteReaderAsync as an alternative to ExecuteReader, ReadAllLinesAsync side by side with ReadAllLines, and so on.

Modern APIs exclusively with asynchronous methods appeared. For example, the HttpClient class only affords asynchronous I/O-based operations.

The TAP reset was further strengthened by the move from .NET to .NET Core. Some frameworks, most notably ASP.NET, were redesigned on a fundamentally asynchronous core.

In 2020, most I/O operations in .NET are easily recognisable, because they return Task<T>.

Task as a surrogate IO #

I/O operations are impure. Either you're receiving input from outside the running process, which is consistently non-deterministic, or you're writing to an external resource, which implies a side effect. It might seem natural to think of Task<T> as a replacement for IO<T>. Szymon Pobiega had a similar idea in 2016, and I investigated his idea in an article. This was based on F#'s Async<'a> container, which is equivalent to Task<T> - except when it comes to referential transparency.

Unfortunately, Task<T> is far from a perfect replacement of IO<T>, because the .NET base class library (BCL) still contains plenty of impure actions that 'look' pure. Examples include Console.WriteLine, the parameterless Random constructor, Guid.NewGuid, and DateTime.Now (arguably a candidate for the worst-designed API in the BCL). None of those methods return tasks, which they ought to if tasks should serve as easily recognisable signifiers of impurity.

Still, you could write asynchronous Adapters over such APIs. Your Console Adapter might present this API:

public static class Console
{
    public static Task<string> ReadLine();
    public static Task WriteLine(string value);
}

Moreover, the Clock API might look like this:

public static class Clock
{
    public static Task<DateTime> GetLocalTime();
}

Modern versions of C# enable you to write asynchronous entry points, so the hello world example shown in this article series becomes:

static async Task Main(string[] args)
{
    await Console.WriteLine("What's your name?");
    var name = await Console.ReadLine();
    var now = await Clock.GetLocalTime();
 
    var greeting = Greeter.Greet(now, name);
 
    await Console.WriteLine(greeting);
}

That's nice idiomatic C# code, so what's not to like?

No referential transparency #

The above Main example is probably as good as it's going to get in C#. I've nothing against that style of C# programming, but you shouldn't believe that this gives you compile-time checking of referential transparency. It doesn't.

Consider a simple function like this, written using the IO container shown in previous articles:

public static string AmIEvil()
{
    Console.WriteLine("Side effect!");
    return "No, I'm not.";
}

Is this method referentially transparent? Surprisingly, despite the apparent side effect, it is. The reason becomes clearer if you write the code so that it explicitly ignores the return value:

public static string AmIEvil()
{
    IO<Unit> _ = Console.WriteLine("Side effect!");
    return "No, I'm not.";
}

The Console.WriteLine method returns an object that represents a computation that might take place. This IO<Unit> object, however, never escapes the method, and thus never runs. The side effect never takes place, which means that the method is referentially transparent. You can replace AmIEvil() with its return value "No, I'm not.", and your program would behave exactly the same.

Consider what happens when you replace IO with Task:

public static string AmIEvil()
{
    Task _ = Console.WriteLine("Side effect!");
    return "Yes, I am.";
}

Is this method a pure function? No, it's not. The problem is that the most common way that .NET libraries return tasks is that the task is already running when it's returned. This is also the case here. As soon as you call this version of Console.WriteLine, the task starts running on a background thread. Even though you ignore the task and return a plain string, the side effect sooner or later takes place. You can't replace a call to AmIEvil() with its return value. If you did, the side effect wouldn't happen, and that would change the behaviour of your program.

Contrary to IO, tasks don't guarantee referential transparency.

Conclusion #

While it'd be technically possible to make C# distinguish between pure and impure code at compile time, it'd require such a breaking reset to the entire .NET ecosystem that it's unrealistic to hope for. It seems, though, that there's enough overlap with the design of IO<T> and task asynchronous programming that the latter might fill that role.

Unfortunately it doesn't, because it fails to guarantee referential transparency. It's better than nothing, though. Most C# programmers have now learned that while Task objects come with a Result property, you shouldn't use it. Instead, you should write your entire program using async and await. That, at least, takes you halfway towards where you want to be.

The compiler, on the other hand, doesn't help you when it comes to those impure actions that look pure. Neither does it protect you against asynchronous side effects. Diligence, code reviews, and programming discipline are still required if you want to separate pure functions from impure actions.

Comments

public static Task<DateTime> GetTime() => new Task<DateTime>(() => DateTime.Now);

public static async Task<TResult> SelectMany<T, TResult>(this Task<T> source
	, Func<T, Task<TResult>> project) {
	T t = await source.GetResultAsync();
	Task<TResult>? ret = project(t);
	return await ret.GetResultAsync();
}

/// <summary> Asynchronously runs the <paramref name="task"/> to Completion </summary>
/// <returns><see cref="Task{T}.Result"/></returns>
public static async Task<T> GetResultAsync<T>(this Task<T> task) {
	switch (task.Status) {
		case TaskStatus.Created: await Task.Run(task.RunSynchronously); return task.Result;
		case TaskStatus.WaitingForActivation:
		case TaskStatus.WaitingToRun:
		case TaskStatus.Running:
		case TaskStatus.WaitingForChildrenToComplete: return await task;
		case TaskStatus.Canceled:
		case TaskStatus.Faulted:
		case TaskStatus.RanToCompletion: return task.Result; //let the Exceptions fly...
		default: throw new ArgumentOutOfRangeException();
	}
}

How to close database connections to SQL Server during integration testing teardown.

Whenever I need to run integration tests that involve SQL Server, I have a standard approach that I've evolved since 2007. It involves

1. setting up a SQL Server Express database before each test
2. running the test
3. tearing down the database

One problem with that approach is that SQL Server doesn't allow you to delete a database if it has existing connections.

Turn off connection pooling #

I usually solve the problem by turning off connection pooling. For an integration test suite, this is fine. I usually use integration testing to verify functionality - not performance.

Turning off connection pooling is easily done by setting the flag to false in the connection string:

Server=(LocalDB)\MSSQLLocalDB;Database=Booking;Integrated Security=true;Pooling=false

This means that when you get to the teardown phase of the test, you can issue a DDL statement to the master database:

IF EXISTS (SELECT name
    FROM master.dbo.sysdatabases
    WHERE name = N'Booking')
DROP DATABASE [Booking]

When connection pooling is turned off, no other connections are open when you attempt to do that, and the database (here named Booking) is deleted.

Forcibly close other connections #

Recently, however, I ran into a testing scenario where connection pooling had to be turned on. When you turn on connection pooling, however, the above DROP DATABASE statement fails because at least one connection from the pool is still connected to the database.

To solve that issue, I forcibly close other connections during teardown:

IF EXISTS (SELECT name
    FROM master.dbo.sysdatabases
    WHERE name = N'Booking')

BEGIN
    -- This closes existing connections:
    ALTER DATABASE [Booking]
    SET SINGLE_USER WITH ROLLBACK IMMEDIATE

    DROP DATABASE [Booking]
END

Surprisingly, turning on connection pooling like this makes the integration tests slower. I suppose it's because throwing other connections off the database involves a bit of negotiation between the server and the clients, and that takes some time.

While slower, it does enable you to run the integration tests with connection pooling turned on. When you need it, you need it.

Summary #

You can run integration tests against a SQL Server Express database. People do it in various ways, but I've found that setting up and tearing down a pristine database for each test case is a robust and maintainable solution to the problem.

SQL Server will not, however, allow you to delete a database if other connections exist. The easiest and fastest solution to that problem is to turn off connection pooling.

Sometimes, you can't do that, so instead, you can expand your database teardown script so that it closes existing connections before it deletes the database.

Comments

Implementation details of the C# IO container.

This article is part of an article series about the IO container in C#. In the previous articles, you've seen how a type like IO<T> can be used to distinguish between pure functions and impure actions.

The point of the article series is to illustrate the concept that Haskell uses to impose the functional interaction law at compile time. The implementation details really aren't important. Still, I believe that I know my readership well enough that a substantial fraction would be left unsatisfied if I didn't share the implementation details.

I consider this an appendix to the article series. It's not really what this is all about, but here it is, nonetheless.

Constructor #

Based on the public API already published, the constructor implementation hardly comes as a surprise.

private readonly Func<T> item;
 
public IO(Func<T> item)
{
    this.item = item;
}

The IO<T> class is little more than a wrapper around a lazily evaluated function, with the restriction that while you can put a Func<T> object into an IO object, you can never get it out again. Thus, the item is a private class field instead of a public property.

SelectMany #

The SelectMany method is a little more tricky:

public IO<TResult> SelectMany<TResult>(Func<T, IO<TResult>> selector)
{
    return new IO<TResult>(() => selector(item()).item());
}

To guarantee referential transparency, we don't want the method to trigger evaluation of the lazy value, so the selector has to run inside a new lazy computation. This produces a lazy IO value that the method then has to unwrap inside another lazy computation. Such a translation from Func<IO<TResult>> to a new IO object with a Func<TResult> inside it is reminiscent of what in Haskell is known as a traversal.

UnsafePerformIO #

Finally, the UnsafePerformIO method isn't part of the API, but as explained in the previous article, this is the special method that the hypothetical parallel-world framework calls on the IO<Unit> returned by Main methods.

internal T UnsafePerformIO()
{
    return item();
}

Since only the framework is supposed to call this method, it's internal by design. The only thing it does is to force evaluation of the lazy item.

Conclusion #

Most of the implementation of IO<T> is straightforward, with the single exception of SelectMany, which has to jump through a few hoops to keep the behaviour lazy until it's activated.

Once more I want to point out that the purpose of this article series is to explain how a type system like Haskell's guarantees referential transparency. C# could do the same, but it'd require that all impure actions in all libraries in the entire .NET ecosystem were to return IO values. That's not going to happen, but something similar already has happened. Read on in the next article.

Next: Task asynchronous programming as an IO surrogate.

Comments

-- output: xx
main = 
  let io = putStr "x"
  in do { io ; io }

// output: x
static IO<Unit> MainIO(string[] args) {
  var io = Console.Write("x");
  return from _1 in io
         from _2 in io
         select Unit.Instance;
}

static IO<Unit> Main(string[] args)
{
    IO<DateTime> getTime = Clock.GetLocalTime();
    return
        getTime.SelectMany(t1 =>
        Console.WriteLine(t1.Ticks.ToString()).Select(u1 =>
        {
            Thread.Sleep(2);
            return Unit.Instance;
        }).SelectMany(u2 =>
        getTime).SelectMany(t2 =>
        Console.WriteLine(t2.Ticks.ToString())));
}

public sealed class IO<T>
{
    private readonly Func<T> item;
 
    public IO(Func<T> item)
    {
        this.item = item;
    }
 
    public IO<TResult> SelectMany<TResult>(Func<T, IO<TResult>> selector)
    {
        return new IO<TResult>(() => selector(item()).item());
    }
 
    internal T UnsafePerformIO()
    {
        return item();
    }
}

public sealed class IO<T>
{
    private readonly Lazy<T> item;
 
    public IO(Lazy<T> item)
    {
        this.item = item;
    }
 
    public IO<TResult> SelectMany<TResult>(Func<T, IO<TResult>> selector)
    {
        var res = new Lazy<IO<TResult>>(() => selector(item.Value));
        return new IO<TResult>(
            new Lazy<TResult>(() => res.Value.item.Value));
    }
 
    internal T UnsafePerformIO()
    {
        return item.Value;
    }
}

How the IO container guarantees referential integrity. An article for object-oriented programmers.

This article is part of an article series about the IO container in C#. In a previous article you got a basic introduction to the IO<T> container I use to explain how a type system like Haskell's distinguishes between pure functions and impure actions.

The whole point of the IO container is to effectuate the functional interaction law: a pure function mustn't be able to invoke an impure activity. This rule follows from referential transparency.

The practical way to think about it is to consider the two rules of pure functions:

• Pure functions must be deterministic
• Pure functions may have no side effects

Determinism #

Like in the previous articles in this series, you must imagine that you're living in a parallel universe where all impure library functions return IO<T> objects. By elimination, then, methods that return naked values must be pure functions.

Consider the Greet function from the previous article. Since it returns a plain string, you can infer that it must be a pure function. What prevents it from doing something impure?

What if you thought that passing now as an argument is a silly design. Couldn't you just call Clock.GetLocalTime from the method?

Well, yes, in fact you can:

public static string Greet(DateTime now, string name)
{
    IO<DateTime> now1 = Clock.GetLocalTime();
 
    var greeting = "Hello";
    if (IsMorning(now))
        greeting = "Good morning";
    if (IsAfternoon(now))
        greeting = "Good afternoon";
    if (IsEvening(now))
        greeting = "Good evening";
 
    if (string.IsNullOrWhiteSpace(name))
        return $"{greeting}.";
 
    return $"{greeting}, {name.Trim()}.";
}

This compiles, but is only the first refactoring step you have in mind. Next, you want to extract the DateTime from now1 so that you can get rid of the now parameter. Alas, you now run into an insuperable barrier. How do you get the DateTime out of the IO?

You can't. By design.

While you can call the GetLocalTime method, you can't use the return value. The only way you can use it is by composing it with SelectMany, but that still accomplishes nothing unless you return the resulting IO object. If you do that, though, you've now turned the entire Greet method into an impure action.

You can't perform any non-deterministic behaviour inside a pure function.

Side effects #

How does IO<T> protect against side effects? In the Greet method, couldn't you just write to the console, like the following?

public static string Greet(DateTime now, string name)
{
    Console.WriteLine("Side effect!");
 
    var greeting = "Hello";
    if (IsMorning(now))
        greeting = "Good morning";
    if (IsAfternoon(now))
        greeting = "Good afternoon";
    if (IsEvening(now))
        greeting = "Good evening";
 
    if (string.IsNullOrWhiteSpace(name))
        return $"{greeting}.";
 
    return $"{greeting}, {name.Trim()}.";
}

This also compiles, despite our best efforts. That's unfortunate, but, as you'll see in a moment, it doesn't violate referential transparency.

In Haskell or F# equivalent code would make the compiler complain. Those compilers don't have special knowledge about IO, but they can see that an action returns a value. F# generates a compiler warning if you ignore a return value. In Haskell the story is a bit different, but the result is the same. Those compilers complain because you try to ignore the return value.

You can get around the issue using the language's wildcard pattern. This tells the compiler that you're actively ignoring the result of the action. You can do the same in C#:

public static string Greet(DateTime now, string name)
{
    IO<Unit> _ = Console.WriteLine("Side effect!");
 
    var greeting = "Hello";
    if (IsMorning(now))
        greeting = "Good morning";
    if (IsAfternoon(now))
        greeting = "Good afternoon";
    if (IsEvening(now))
        greeting = "Good evening";
 
    if (string.IsNullOrWhiteSpace(name))
        return $"{greeting}.";
 
    return $"{greeting}, {name.Trim()}.";
}

The situation is now similar to the above treatment of non-determinism. While there's no value of interest in an IO<Unit>, the fact that there's an object at all is a hint. Like Lazy<T>, that value isn't a result. It's a placeholder for a computation.

If there's a way to make the C# compiler complain about ignored return values, I'm not aware of it, so I don't know if we can get closer to how Haskell works than this. Regardless, keep in mind that I'm not trying to encourage you to write C# like this; I'm only trying to explain how Haskell enforces referential transparency at the type level.

Referential transparency #

While the above examples compile without warnings in C#, both are still referentially transparent!

This may surprise you. Particularly the second example that includes Console.WriteLine looks like it has a side effect, and thus violates referential transparency.

Keep in mind, though, what referential transparency means. It means that you can replace a particular function call with its value. For example, you should be able to replace the function call Greeter.Greet(new DateTime(2020, 6, 4, 12, 10, 0), "Nearth") with its return value "Hello, Nearth.", or the function call Greeter.Greet(new DateTime(2020, 6, 4, 7, 30, 0), "Bru") with "Good morning, Bru.", without changing the behaviour of the software. This property still holds.

Even when the Greet method includes the Console.WriteLine call, that side effect never happens.

The reason is that an IO object represents a potential computation that may take place (also known as a thunk). Notice that the IO<T> constructor takes a Func as argument. It's basically just a lazily evaluated function wrapped in such a way that you can't force evaluation.

Instead, you should imagine that after Main returns its IO<Unit> object, the parallel-universe .NET framework executes it.

The framework supplies command-line arguments to the Main method. Once the method returns an IO<Unit> object, the framework executes it with a special method that only the framework can invoke. Any other IO values that may have been created (e.g. the above Console.WriteLine) never gets executed, because they're not included in the return value.

Conclusion #

The IO container makes sure that pure functions maintain referential transparency. The underlying assumption that makes all of this work is that all impure actions return IO objects. That's not how the .NET framework works, but that's how Haskell works. Since the IO container is opaque, pure functions can't see the contents of IO boxes.

Program entry points are all impure. The return value of the entry point must be IO<Unit>. The hypothetical parallel-universe framework executes the IO value returned by Main.

Haskell is a lazily evaluated language. That's an important piece of the puzzle, so I've modelled the IO<T> example so that it only wraps lazily evaluated Func values. That emulates Haskell's behaviour in C#. In the next article, you'll see how I wrote the C# code that supports these articles.

Next: Implementation of the C# IO container.

Comments

public static string Greet(DateTime now, string name)
{
    _ = Console.WriteLine("Side effect!");
	
    var greeting = "Hello";
    if (IsMorning(now))
        greeting = "Good morning";
    if (IsAfternoon(now))
        greeting = "Good afternoon";
    if (IsEvening(now))
        greeting = "Good evening";
	
    if (string.IsNullOrWhiteSpace(name))
        return $"{greeting}.";
	
    return $"{greeting}, {name.Trim()}.";
}

How to make use of the C# IO container less ugly.

This article is part of an article series about the IO container in C#. In the previous article you saw a basic C# hello world program using IO<T> to explicitly distinguish between pure functions and impure actions. The code wasn't as pretty as you could hope for. In this article, you'll see how to improve the aesthetics a bit.

The code isn't going to be perfect, but I think it'll be better.

Sugared version #

The IO<T> container is an imitation of the Haskell IO type. In Haskell, IO is a monad. This isn't a monad tutorial, and I hope that you're able to read the article without a deep understanding of monads. I only mention this because when you compose monadic values with each other, you'll sometimes have to write some 'noisy' code - even in Haskell. To alleviate that pain, Haskell offers syntactic sugar in the form of so-called do notation.

Likewise, F# comes with computation expressions, which also gives you syntactic sugar over monads.

C#, too, comes with syntactic sugar over monads. This is query syntax, but it's not as powerful as Haskell do notation or F# computation expressions. It's powerful enough, though, to enable you to improve the Main method from the previous article:

static IO<Unit> Main(string[] args)
{
    return from    _ in Console.WriteLine("What's your name?")
           from name in Console.ReadLine()
           from  now in Clock.GetLocalTime()
 
           let greeting = Greeter.Greet(now, name)
 
           from  res in Console.WriteLine(greeting)
           select res;
}

If you use C# query syntax at all, you may think of it as exclusively the realm of object-relational mapping, but in fact it works for any monad. There's no data access going on here - just the interleaving of pure and impure code (in an impureim sandwich, even).

Infrastructure #

For the above code to compile, you must add a pair of methods to the IO<T> API. You can write them as extension methods if you like, but here I've written them as instance methods on IO<T>.

When you have multiple from keywords in the same query expression, you must supply a particular overload of SelectMany. This is an oddity of the implementation of the query syntax language feature in C#. You don't have to do anything similar to that in F# or Haskell.

public IO<TResult> SelectMany<U, TResult>(Func<T, IO<U>> k, Func<T, U, TResult> s)
{
    return SelectMany(x => k(x).SelectMany(y => new IO<TResult>(() => s(x, y))));
}

Once you've implemented such overloads a couple of times, they're more tedious than challenging to write. They always follow the same template. First use SelectMany with k, and then SelectMany again with s. The only marginally stimulating part of the implementation is figuring out how to wrap the return value from s.

You're also going to need Select as shown in the article about IO as a functor.

Conclusion #

C#'s query syntax offers limited syntactic sugar over functors and monads. Compared with F# and Haskell, the syntax is odd and its functionality limited. The most galling lacuna is that you can't branch (e.g. use if or switch) inside query expressions.

The point of these articles is (still) not to endorse this style of programming. While the code I show in this article series is working C# code that runs and passes its tests, I'm pretending that all impure actions in C# return IO results. To be clear, the Console class this code interacts with isn't the Console class from the base class library. It's a class that pretends to be such a class from a parallel universe.

So far in these articles, you've seen how to compose impure actions with pure functions. What I haven't covered yet is the motivation for it all. We want the compiler to enforce the functional interaction law: a pure function shouldn't be able to invoke an impure action. That's the topic for the next article.

Next: Referential transparency of IO.

The IO container forms a functor. An article for object-oriented programmers.

This article is an instalment in an article series about functors. Previous articles have covered Maybe, Lazy, and other functors. This article provides another example.

Functor #

In a recent article, I gave an example of what IO might look like in C#. The IO<T> container already has sufficient API to make it a functor. All it needs is a Select method:

public IO<TResult> Select<TResult>(Func<T, TResult> selector)
{
    return SelectMany(x => new IO<TResult>(() => selector(x)));
}

This is an instance method on IO<T>, but you can also write it as an extension method, if that's more to your liking.

When you call selector(x), the return value is an object of the type TResult. The SelectMany method, however, wants you to return an object of the type IO<TResult>, so you use the IO constructor to wrap that return value.

Haskell #

The C# IO<T> container is an illustration of how Haskell's IO type works. It should come as no surprise to Haskellers that IO is a functor. In fact, it's a monad, and all monads are also functors.

The C# IO<T> API is based around a constructor and the SelectMany method. The constructor wraps a plain T value in IO<T>, so that corresponds to Haskell's return method. The SelectMany method corresponds to Haskell's monadic bind operator >>=. When you have lawful return and >>= implementations, you can have a Monad instance. When you have a Monad instance, you not only can have Functor and Applicative instances, you must have them.

Conclusion #

IO forms a functor, among other abstractions. In C#, this manifests as a proper implementation of a Select method.

Next: Monomorphic functors.

Comments