ploeh blog danish software design

When you use Semantic Versioning with Continuous Deployment, version numbers must be checked into source control systems by programmers.

If you aren't already using Semantic Versioning, you should. It makes it much easier to figure out how to version your releases. Even if you're 'just' building software for your internal organization, or a single customer, you should still care about versioning of the software you release. Instead of an ad-hoc versioning scheme, Semantic Versioning offers a set of easy-to-understand rules about when to increment which version number.

In short, you

• increment the patch version (e.g. from 2.3.4 to 2.3.5) when you only release bug fixes and the like
• increment the minor version (e.g. from 1.3.2 to 1.4.0) when you add new features
• increment the major version (e.g. from 3.2.9 to 4.0.0) when you introduce breaking changes

Continuous Deployment #

While Semantic Versioning is great, it requires a bit of consideration when combined with Continuous Deployment. Every time you deploy a new version, you should increment the version number.

Continuous Delivery and Continuous Deployment rely on automation. A code check-in triggers an automated build, which is subsequently processed by a Deployment Pipeline, and potentially released to end-users. Each released (or releasable) build should have a unique version.

Traditionally, Build Servers have had the responsibility of incrementing version numbers - typically by incrementing a build number, like this:

1. 3.7.11.942
2. 3.7.12.958
3. 3.7.13.959
4. 3.7.14.979
5. 3.7.15.987

Unfortunately, this versioning scheme is wrong if you combine Semantic Versioning with Continuous Deployment. Even if you throw away the fourth build number, you're left with a sequence like this:

1. 3.7.11 (bug fix)
2. 3.7.12 (partial new feature, hidden behind a Feature Toggle.)
3. 3.7.13 (performance improvement)
4. 3.7.14 (completed feature initiated in 3.7.12)
5. 3.7.15 (breaking changes in public API)

Semantic Versioning might look like this:

1. 3.7.11 (bug fix)
2. 3.7.12 (partial new feature, hidden behind a Feature Toggle.)
3. 3.7.13 (performance improvement)
4. 3.8.0 (completed feature initiated in 3.7.12)
5. 4.0.0 (breaking changes in public API)

Versioning is a programmer decision #

With Continuous Deployment, every time you integrate code (check in, merge, rebase, whatever), you produce a version of the software that will be deployed. This means that every time you integrate, something or somebody should assign a new version to the software.

The rules of Semantic Versioning require explicit decisions to be made. Only the development team understands what a particular commit contains. Is it a fix? Is it a new feature? Is it a breaking change? A Build Server doesn't know how to answer these questions, but you do.

A few years ago, I changed the delivery scheme for my open source project AutoFixture to use Semantic Versioning with Continuous Deployment. When I did that, I realised that I could no longer rely on a Build Server for controlling the version. Instead, I would have to explicitly control the versioning as part of the commit process.

Because AutoFixture is a .NET project, I decided to use the version assignment mechanism already present in the framework: The [AssemblyVersion] and [AssemblyFileVersion] attributes that you typically put in AssemblyInfo files.

The version control system used for AutoFixture is Git, so it works like this in practice:

1. A programmer adds one or more commits to a branch.
2. The programmer sends a pull request.
3. I pull down the commits from the pull request.
4. I increment all the version attributes in all the AssemblyInfo files, and commit that change.
5. I push the commits to master.
6. The Build Server picks up the new commits, and the Deployment Pipeline kicks in.

After more than two years of experience with this way of controlling software versions, I'm consistently using this approach for all my open source software, as well as the internal software we create in Grean.

Summary #

If you want to use Continuous Deployment (or Delivery) with Semantic Versioning, the assignment of a new version number is a programmer decision. Only a human understands when a commit constitutes a bug fix, a new feature, or a breaking change. The new version number must be committed to the version control system, so that whomever or whatever compiles and/or releases the software will always use the same version number for the same version of the source code.

The version number is kept in the source control system, together with the source code. It's not the realm of a Build Server.

Comments

while (a release request is in progress)
    if (app has previously been 'released' to some repository AND has subsequently changed in any way)
        increment PATCH...
            unless (all issues and features in next milestone have been marked closed, subsequent to last release)
                increment MINOR and reset PATCH to zero...
                    unless (unit tests that predate release have changed OR dependent application unit tests are failing OR some other determination of breaking change) 
                        increment MAJOR and reset MINOR and PATCH to zero...

If you apply the Dependency Inversion Principle to Layered Architecture, you end up with Ports and Adapters.

One of my readers, Giorgio Sala, asks me:

In his book "Implementing DDD" mr Vernon talks a lot about the Ports and Adapter architecture as a next level step of the Layered architecture. I would like to know your thinking about it.

The short answer is that this is more or less the architecture I describe in my book, although in the book, I never explicitly call it out by that name.

Layered architecture #

In my book, I describe the common pitfalls of a typical layered architecture. For example, in chapter 2, I analyse a typical approach to layered architecture; it's an example of what not to do. Paraphrased from the book's figure 2.13, the erroneous implementation creates this dependency graph:

The arrows show the direction of dependencies; i.e. the User Interface library depends on the Domain library, which in turn depends on the Data Access library. This violates the Dependency Inversion Principle (DIP), because the Domain library depends on the Data Access library, and the DIP says that:

Abstractions should not depend upon details. Details should depend upon abstractions.

- Agile Principles, Patterns, and Practices in C#, p. 154

Later in chapter 2, and throughout the rest of my book, I demonstrate how to invert the dependencies. Paraphrased, figure 2.12 looks like this:

This is almost the same figure as the previous, but notice that the direction of dependency has changed, so that the Data Access library now depends on the Domain library, instead of the other way around. This is the DIP applied: the details (UI, Data Access) depend on the abstractions (the Domain Model).

Onion layers #

The example from chapter 2 in my book is obviously simplified, with only three libraries involved. Imagine a generalized architecture following the DIP:

While there are many more libraries, notice that all dependencies still point inwards. If you're still thinking in terms of layers, you can draw concentric layers around the boxes:

These concentric layers resemble the layers of an onion, so it's not surprising that Jeffrey Palermo calls this type of architecture for Onion Architecture.

The DIP still applies, so dependencies can only go in one direction. However, it would seem that I've put the UI components (the orange boxes) and the Data Access components (the blue boxes) in the same layer. (Additionally, I've added some yellow boxes that might symbolise unit tests.) This may seem unfamiliar, but actually makes sense, because the components in the outer layer are all at the boundaries of the application. Some boundaries (such as UI, RESTful APIs, message systems, etc.) face outward (to the internet, extranets, etc.), while other boundaries (e.g. databases, file systems, dependent web services, etc.) face inward (to the OS, database servers, etc.).

As the diagram implies, components can depend on other components within the same layer, but does that mean that UI components can talk directly to Data Access components?

Hexagonal architecture #

While traditional Layered Architecture is no longer the latest fad, it doesn't mean that all of its principles are wrong. It's still not a good idea to allow UI components to depend directly on the Data Access layer; it would couple such components together, and you might accidentally bypass important business logic.

You have probably noticed that I've grouped the orange, yellow, and blue boxes into separate clusters. This is because I still want to apply the old rule that UI components must not depend on Data Access components, and vice versa. Therefore, I introduce bulkheads between these groups:

Although it may seem a bit accidental that I end up with exactly six sections (three of them empty), it does nicely introduce Alistair Cockburn's closely related concept of Hexagonal Architecture:

You may feel that I cheated a bit in order to make my diagram hexagonal, but that's okay, because there's really nothing inherently hexagonal about Hexagonal Architecture; it's not a particularly descriptive name. Instead, I prefer the alternative name Ports and Adapters.

Ports and Adapters #

The only thing still bothering me with the above diagram is that the dependency hierarchy is too deep (at least conceptually). When the diagram consisted of concentric circles, it had three (onion) layers. The hexagonal dependency graph above still has those intermediary (grey) components, but as I've previously attempted to explain, the flatter the dependency hierarchy, the better.

The last step, then, is to flatten the dependency hierarchy of the inner hexagon:

The components in the inner hexagon have few or no dependencies on each other, while components in the outer hexagon act as Adapters between the inner components, and the application boundaries: its ports.

Summary #

In my book, I never explicitly named the architecture I describe, but essentially, it is the Ports and Adapters architecture. There are other possible application architectures than the variations described here, and some of them still work well with Dependency Injection, but the main architectural emphasis in Dependency Injection in .NET is Ports and Adapters, because I judged it to be the least foreign for the majority of the book's readers.

The reason I never explicitly called attention to Ports and Adapters or Onion Architecture in my book is that I only became aware of these pattern names as I wrote the book. At that time, I didn't feel confident that what I did matched those patterns, but the more I've learned, the more I've become convinced that this was what I'd been doing all along. This just confirms that Ports and Adapters is a bona fide pattern, because one of the attributes of patterns is that they materialize independently in different environments, and are then subsequently discovered as patterns.

Unfortunately, I've had to cancel my speaking engagement at NDC London 2013.

Ever since I was accepted as a speaker at NDC London 2013, I've really looked forward to it. Unfortunately, due to serious illness in my family, I've decided to cancel my appearance at the conference. This has been a difficult decision to make, but now that I've made it, I can feel that it's the right decision, even though it pains me.

I hope to be able to return to NDC another time, both in Oslo and in London.

If you visit my Lanyrd profile, you will see that while I have removed myself from NDC London, I'm still scheduled to speak at the Warm Crocodile Developer Conference in January 2014. I still hope to be able to attend and speak here. Not only is it realistic to hope that my family situation is better in January, but because the conference is in my home town, it also means that it puts less of a strain on my family. This may change, though...

Albedo is a .NET library targeted at making Reflection programming more consistent, using a common set of abstractions and utilities.

For the last five weeks, Nikos Baxevanis, Adam Chester, and I have been working on a new, small Open Source project: Albedo.

It's a .NET library targeted at making Reflection programming more consistent, using a common set of abstractions and utilities. The project site may actually contain more details than you'd care to read, but here's a small code sample to either whet your appetite, or scare you away:

PropertyInfo pi = from v in new Properties<Version>()
                  select v.Minor;
var version = new Version(2, 7);
var visitor = new ValueCollectingVisitor(version);
 
var actual = new PropertyInfoElement(pi).Accept(visitor);
 
Assert.Equal(version.Minor, actual.Value.OfType<int>().First());

Albedo is available via NuGet.

When unit testing, use Mocks for Commands, and Stubs for Queries.

A few years ago, I helped an organization adopt Test-Driven Development. One question that kept coming up was when to use Stubs, and when to use Mocks. (First, you need to understand the conceptual difference between Mocks and Stubs, so go read xUnit Test Patterns, and then resume reading this blog post once you're done. If you need a shorter introduction, you can read my MSDN Magazine article on the subject.)

After having answered the question on when to use what, a couple of times, I arrived at this simple rule, based on the language of Command Query Separation (CQS):

• Use Mocks for Commands
• Use Stubs for Queries

This discovery made me really happy, and I planned to blog about it. Then, soon after, I saw the exact same advice in GOOS (ch. 24, p. 278):

Allow Queries; Expect Commands

A couple of years later, it's clear to me that this rule may be public knowledge, but is certainly isn't common knowledge. This is, without comparison, the most common mistake I see people make, when I review unit tests. Thus, although it has already been said in GOOS, I'll repeat it here.

Use Mocks for Commands; Stubs for Queries

(Anti-)Example: Mocking Queries #

In this example, you'll see how easily things can go wrong when you use Mocks for Queries. Assume that you want to test SomeController, that uses an IUserRepository:

public interface IUserRepository
{
    User Read(int userId);
 
    void Create(int userId);
}

In the first test, you want to verify that when you invoke SomeController.GetUser, it returns the user from the injected Repository, so you write this unit test:

[Fact]
public void GetUserReturnsCorrectResult()
{
    var expected = new User();
    var td = new Mock<IUserRepository>();
    td.Setup(r => r.Read(It.IsAny<int>())).Returns(expected);
    var sut = new SomeController(td.Object);
 
    var actual = sut.GetUser(1234);
 
    Assert.Equal(expected, actual);
}

This test uses Moq, but it could have used another dynamic mock library as well, or even hand-written Test Doubles.

The most remarkable characteristic of this test is the unconditional return of a value from the Test Double, as implemented by the use of It.IsAny<int>(). Why you'd do this is a small mystery to me, but I see it time and again when I review unit tests. This is the source of many problems.

Using a technique called Devil's Advocate, when reviewing, I usually tell the author that I can make the test pass with this (obviously degenerate) implementation:

public User GetUser(int userId)
{
    return this.userRepository.Read(0);
}

In my experience, this will most likely prompt you to add another test:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserCallsRepositoryWithCorrectValue(int userId)
{
    var td = new Mock<IUserRepository>();
    var sut = new SomeController(td.Object);
 
    sut.GetUser(userId);
 
    td.Verify(r => r.Read(userId));
}

Jolly good attempt, sport! Regrettably, it doesn't protect you against my Devil's Advocate technique; I can implement the GetUser method like this:

public User GetUser(int userId)
{
    this.userRepository.Read(userId);
    return this.userRepository.Read(0);
}

This is so obviously wrong that you'd be likely to argue that no-one in their right mind would do something like that. While I agree, the fact that this (obviously incorrect) implementation passes all unit tests should inform you about the quality of your unit tests.

Strict Mocks are not the solution #

The problem is that the second test method (GetUserCallsRepositoryWithCorrectValue) attempts to use a Mock to verify a Query. This is wrong, as indicated by GOOS.

In desperation, some people attempt to resolve their conundrum by tightening the constraints of the Mock. This road leads towards Strict Mocks, and there lies madness. Soon, you'll learn the reason for this.

First, you may attempt to constrain the number of times the Read method can be invoked:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserCallsRepositoryWithCorrectValue(int userId)
{
    var td = new Mock<IUserRepository>();
    var sut = new SomeController(td.Object);
 
    sut.GetUser(userId);
 
    td.Verify(r => r.Read(userId), Times.Once());
}

Notice the addition of the Times.Once() clause in the last line. This instructs Moq that Read(userId) can only be invoked once, and that an exception should be thrown if it's called more than once. Surprisingly, the Devil's Advocate implementation still passes all tests:

public User GetUser(int userId)
{
    this.userRepository.Read(userId);
    return this.userRepository.Read(0);
}

While this may seem surprising, the reason it passes is that the assertion only states that Read(userId) should be invoked exactly once; it doesn't state anything about Read(0), as long as userId isn't 0.

Second, you attempt to resolve that problem by stating that no matter the input, the Read method must be invoked exactly once:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserCallsRepositoryWithCorrectValue(int userId)
{
    var td = new Mock<IUserRepository>();
    var sut = new SomeController(td.Object);
 
    sut.GetUser(userId);
 
    td.Verify(r => r.Read(It.IsAny<int>()), Times.Once());
}

Notice that the input constraint is loosened to It.IsAny<int>(); combined with Times.Once(), it should ensure that the Read method is invoked exactly once. It does, but the devil is still mocking you (pun intended):

public User GetUser(int userId)
{
    return this.userRepository.Read(0);
}

This is the first degenerate implementation I added, so now you're back where you began: it passes all tests. The new test case (that uses a Mock against a Query) has added no value at all.

Third, in utter desperation, you turn to Strict Mocks:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserCallsRepositoryWithCorrectValue(int userId)
{
    var td = new Mock<IUserRepository>(MockBehavior.Strict);
    td.Setup(r => r.Read(userId)).Returns(new User());
    var sut = new SomeController(td.Object);
 
    sut.GetUser(userId);
 
    td.Verify();
}

Notice the use of MockBehavior.Strict in the Mock constructor, as well as the explicit Setup in the Fixture Setup phase.

Finally, this reigns in the devil; I have no other recourse than to implement the Read method correctly:

public User GetUser(int userId)
{
    return this.userRepository.Read(userId);
}

If the Devil's Advocate technique indicates that a faulty implementation implies a bad test suite, then a correct implementation must indicate a good test suite, right?

Not quite, because the use of a Strict Mock makes your tests much less maintainable. You should already know this, but I'll show you an example.

Assume that you have a rule that if the User returned by the Read method has an ID of 0, it means that the user doesn't exist, and should be created. (There are various problems with this design, most notably that it violates CQS, but that's another story...)

In order to verify that a non-existing User is created during reading, you add this unit test:

[Fact]
public void UserIsSavedIfItDoesNotExist()
{
    var td = new Mock<IUserRepository>();
    td.Setup(r => r.Read(1234)).Returns(new User { Id = 0 });
    var sut = new SomeController(td.Object);
 
    sut.GetUser(1234);
 
    td.Verify(r => r.Create(1234));
}

This is a fine test that verifies a Command (Create) with a Mock. This implementation passes the test:

public User GetUser(int userId)
{
    var u = this.userRepository.Read(userId);
    if (u.Id == 0)
        this.userRepository.Create(1234);
    return u;
}

Alas! Although this implementation passes the new test, it breaks an existing test! Can you guess which one? Yes: the test that verifies a Query with a Strict Mock. It breaks because it explicitly states that the only method call allowed on the Repository is the Read method.

You can resolve the problem by editing the test:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserCallsRepositoryWithCorrectValue(int userId)
{
    var td = new Mock<IUserRepository>(MockBehavior.Strict);
    td.Setup(r => r.Read(userId)).Returns(new User { Id = userId });
    var sut = new SomeController(td.Object);
 
    sut.GetUser(userId);
 
    td.Verify();
}

Notice that userId was added to the returned User in the Setup for the Strict Mock.

Having to edit existing tests is a genuine unit test smell. Not only does it add an unproductive overhead (imagine that many tests break instead of a single one), but it also decreases the trustworthiness of your test suite.

Data Flow Verification #

The solution is really simple: use a conditional Stub to verify the data flow through the SUT. Here's the single test you need to arrive at a correct implementation:

[Theory]
[InlineData(1234)]
[InlineData(9876)]
public void GetUserReturnsCorrectValue(int userId)
{
    var expected = new User();
    var td = new Mock<IUserRepository>();
    td.Setup(r => r.Read(userId)).Returns(expected);
    var sut = new SomeController(td.Object);
 
    var actual = sut.GetUser(userId);
 
    Assert.Equal(expected, actual);
}

Even when I attempt to use the Devil's Advocate technique, there's no way I can provide a faulty implementation. The method must invoke Read(userId), because otherwise it can't return expected. If it invokes the Read method with any other value, it may get a User instance, but not expected.

The Data Flow Verification test uses a (conditional) Stub to verify a Query. The distinction may seem subtle, but it's important. The Stub doesn't require that Read(userId) is invoked, but is configured in such a way that if (and only if) Read(userId) is invoked, it'll return expected.

With the Stub verifying the Query (GetUserReturnsCorrectValue), and the other test that uses a Mock to verify a Command (UserIsSavedIfItDoesNotExist), the final implementation is:

public User GetUser(int userId)
{
    var u = this.userRepository.Read(userId);
    if (u.Id == 0)
        this.userRepository.Create(1234);
    return u;
}

These two tests correctly specify the behaviour of the system in a terse and maintainable way.

Conclusion #

Data Flow Verification is remarkably simple, so I'm continually taken aback that people seem to go out of their way to avoid it. Use it to verify Queries with Stubs, and keep the Mocks with the Commands.

Comments

In F#, Discriminated Unions provide a good alternative to overloading.

When you're learning a new programming language, you may experience these phases:

1. [language x] is the coolest thing ever :D
2. [language x] sucks, because it can't do [xyz], like [language y], which I know really well :(
3. Now I finally understand that in [language x] I don't need [xyz], because it has different idioms o_O

From C#, I'm used to method overloading as a design technique to provide an API with easy-to-learn default methods, and more complicated, but more flexible, methods - all in the same family of methods. Alas, in F#, function overloading isn't possible. (Well, method overloading is still possible, because you can create .NET classes in F#, but if you're creating a module with free-standing functions, you can't have two functions with the same name, but different parameters.)

This really bothered me until I realized that F# has other language constructs that enable you to approach that problem differently. Discriminated Unions is one such language construct.

Multiple related functions #

Recently, I was working with a little module that enabled me to list a range of dates. As an example, I wanted to be able to list all dates in a given year, or all dates in a given month.

My first attempt looked like this:

module Dates =
    let InitInfinite (date : DateTime) =
        date |> Seq.unfold (fun d -> Some(d, d.AddDays 1.0))
 
    let InYear year =
        DateTime(year, 1, 1)
        |> InitInfinite
        |> Seq.takeWhile (fun d -> d.Year = year)
 
    let InMonth year month =
        DateTime(year, month, 1)
        |> InitInfinite
        |> Seq.takeWhile (fun d -> d.Month = month)

These functions did what I wanted them to do, but I found the names awkward. The InYear and InMonth functions are closely related, so I would have liked to call them both In, as I would have been able to do in C#. That would have enabled me to write code such as:

let actual = Dates.In year

and

let actual = Dates.In year month

where year and month are integer values. Alas, that's not possible, so instead I had to settle for the slightly more clumsy InYear:

let actual = Dates.InYear year

and InMonth:

let actual = Dates.InMonth year month

That is, until I realized that I could model this better with a Discriminated Union.

One function #

After a day or so, I had one of those small revelations described in Domain-Driven Design: implicitly, I was working with a concept of a period. Time to make the implicit concept explicit:

type Period =
    | Year of int
    | Month of int * int
 
module Dates =
    let InitInfinite (date : DateTime) =
        date |> Seq.unfold (fun d -> Some(d, d.AddDays 1.0))
 
    let private InYear year =
        DateTime(year, 1, 1)
        |> InitInfinite
        |> Seq.takeWhile (fun d -> d.Year = year)
 
    let private InMonth year month =
        DateTime(year, month, 1)
        |> InitInfinite
        |> Seq.takeWhile (fun d -> d.Month = month)
 
    let In period =
        match period with
        | Year(y) -> InYear y
        | Month(y, m) -> InMonth y m

Notice that I defined a Period Discriminated Union outside the module, because it enables me to write client code like:

let actual = Dates.In(Year(year))

and

let actual = Dates.In(Month(year, month))

This syntax requires slightly more characters than the previous alternative, but is (subjectively) more elegant.

If you prefer, you can refactor the In function to:

let In = function
    | Year(y) -> InYear y
    | Month(y, m) -> InMonth y m

You may have noticed that this implementation still relies on the two private functions InYear and InMonth, but it's easy to refactor the In function to a single function:

let In period =
    let generate dt predicate =
        dt |> InitInfinite |> Seq.takeWhile predicate
    match period with
    | Year(y) -> generate (DateTime(y, 1, 1)) (fun d -> d.Year = y)
    | Month(y, m) -> generate (DateTime(y, m, 1)) (fun d -> d.Month = m)

As you can see, the introduction of a Period Discriminated Union enabled me to express the API in a way that closely resembles what I originally envisioned.

A richer API #

Once I made the change to a Discriminated Union, I discovered that I could make my API richer. Soon, I had this Dates module:

type Period =
    | Year of int
    | Month of int * int
    | Day of int * int * int
 
module Dates =
    let InitInfinite (date : DateTime) =
        date |> Seq.unfold (fun d -> Some(d, d.AddDays 1.0))
 
    let In period =
        let generate dt predicate =
            dt |> InitInfinite |> Seq.takeWhile predicate
        match period with
        | Year(y) -> generate (DateTime(y, 1, 1)) (fun d -> d.Year = y)
        | Month(y, m) -> generate (DateTime(y, m, 1)) (fun d -> d.Month = m)
        | Day(y, m, d) -> DateTime(y, m, d) |> Seq.singleton
 
    let BoundariesIn period =
        let getBoundaries firstTick (forward : DateTime -> DateTime) =
            let lastTick = forward(firstTick).AddTicks -1L
            (firstTick, lastTick)
        match period with
        | Year(y) -> getBoundaries (DateTime(y, 1, 1)) (fun d -> d.AddYears 1)
        | Month(y, m) -> getBoundaries (DateTime(y, m, 1)) (fun d -> d.AddMonths 1)
        | Day(y, m, d) -> getBoundaries (DateTime(y, m, d)) (fun d -> d.AddDays 1.0)

Notice that I added a Day case. Originally, I didn't think it would be valuable, as Dates.In(Day(year, month, day)) seemed like quite a convoluted way of saying DateTime(year, month, day). However, it turned out that the In abstraction was valuable, also with a single day - simply because it's an abstraction.

Additionally, I then discovered the utility of a function called BoundariesIn, which gives me the boundaries of a Period - that is, the very first and last tick of the Period.

Summary #

It's easy to become frustrated while learning a new programming language. In F#, I was annoyed by the lack of function overloading, until I realized that a single function taking a Discriminated Union might actually be a richer idiom.

With F#, it's easy to create DTOs for use with the ASP.NET Web API, using record types.

When writing HTTP (or RESTful) web services with the ASP.NET Web API, the most normal approach is to define Data Transfer Objects (DTOs), which represents the data structures that go on the wire in the form of JSON or XML. While I usually call these boundary objects for renditions (a bit punny), a more normal terminology is, indeed, DTOs.

To enable the default .NET serializers (and particularly, deserializers) to do their magic, these DTOs must be mutable and have a default constructor. Not particularly something that seems to fit nicely with F#.

Until recently, I've been declaring such DTOs like this in F#:

type HomeRendition() =
    [<DefaultValue>] val mutable Message : string
    [<DefaultValue>] val mutable Time : string

However, then I discovered the [<CLIMutable>] attribute. This effectively turn a standard F# record type into something that can be serialized with the normal .NET serializers. This means that it's possible to redefine the above HomeRendition like this:

[<CLIMutable>]
type HomeRendition = {
    Message : string
    Time : string }

This is much better, because this looks like a proper immutable record type from the perspective of the F# compiler. However, from a CLI perspective, the HomeRendition class has a default constructor, and mutable properties.

A DTO defined like this still works with the ASP.NET Web API, although by default, the serialization looks a bit strange:

<HomeRendition>
  <Message_x0040_>
    This message is served by a pure F# ASP.NET Web API implementation.
  </Message_x0040_>
  <Time_x0040_>2013-10-15T23:32:42.6725088+02:00</Time_x0040_>
</HomeRendition>

The reason for that is that the [<CLIMutable>] attribute causes the record type to be compile with auto-generated internal mutable fields, and these are named by appending an @ character - in this case, the field names become Message@ and Time@. Since the unicode value for the @ character is x0040, these field names become Message_x0040_ and Time_x0040_.

Wait a minute! Did I say internal fields? Yes, I did. Then why are the internal fields being serialized instead of the public properties? Well, what can I say, other than the DataContractSerializer once again proves to be a rather poor choice of default serializer. Yet another reason to use a sane XML serializer.

No doubt, one of my readers can come up with a good solution for the DataContractSerializer too, but since I always switch my Web API services to use a proper XML serializer, I don't really care:

GlobalConfiguration.Configuration.Formatters.XmlFormatter.UseXmlSerializer <- true

Now all is well again:

<HomeRendition>
  <Message>
    This message is served by a pure F# ASP.NET Web API implementation.
  </Message>
  <Time>2013-10-15T23:50:06.9025322+02:00</Time>
</HomeRendition>

That's it: much easier, and more robust, Web API DTOs with F# record types. Just apply the [<CLIMutable>] attribute.

Comments

config.Formatters.JsonFormatter.SerializerSettings.ContractResolver <-
    Newtonsoft.Json.Serialization.CamelCasePropertyNamesContractResolver()

type HomeRecord = {
    Message : string
    Time : string
}

type InputModel = {
    Message : string
}

type HomeController() =
    inherit ApiController()
    member this.Get() =
        this.Ok({ Message = "Hello from F#!"; Time = DateTime.Now.ToString() })
    member this.Post(input : InputModel) =
        this.Ok(input.Message)

You can run a validation task for every commit in a Git branch in order to verify the integrity of a Git branch's history.

Just like the Soviet Union, I occasionally prefer to rewrite history... although I limit myself to rewriting the history of one or more Git branches. When I do that, I'd like to verify that I didn't break anything after an interactive rebase. It actually turns out that this can easily happen. Some commits are left in a state where they either don't compile, or tests fail. Git is happy, but the code isn't. To deal with such situations, I wanted a tool that could verify each and every commit in a branch like this:

1. Check out a commit.
2. Compile the code.
3. Run all tests.
4. Repeat for the next commit, or stop if there's a failure.

Motivation #

If you don't care about my motivation for doing this, you can skip this section and move on to the solution. However, some people might like to point out to me that I shouldn't rewrite the history of my Git repositories - and particularly not of the master branch. I agree, for all repositories where I actually collaborate with other programmers.

However, I also use Git locally to produce teaching materials. As an example, if you download the example code for my Pluralsight courses, you'll see that it's all available as Git repositories. I think that's an added benefit for the student, because not only can you see the final result, but you can also see all the steps that lead me there. I even occasionally write commit messages longer than a single line, explaining my thinking as I check in code.

Like everyone else, I'm fallible, so often, when I prepare such materials, I end up taking some detours that I feel will confuse students more than it will help. Thus, I have a need to be able to edit my Git repositories. Right now, as an example, I'm working with a repository with 120 commits on the master branch, and I need to make some changes in the beginning of the history.

Solution #

With much help from Darrell Hamilton, who ended up providing this gist, I was able to piece together this bash script:

#!/bin/sh
COMMITS=$(git --git-dir=BookingApi/.git --work-tree=BookingApi log --oneline  --reverse | cut -d " " -f 1)
CODE=0

git --git-dir=BookingApi/.git --work-tree=BookingApi reset --hard
git --git-dir=BookingApi/.git --work-tree=BookingApi clean -xdf
 
for COMMIT in $COMMITS
do
    git --git-dir=BookingApi/.git --work-tree=BookingApi checkout $COMMIT
    # run-tests          
    ./build.sh

    if [ $? -eq 0 ]
    then
        echo $COMMIT - passed
    else
        echo $COMMIT - failed
        exit
    fi

    git --git-dir=BookingApi/.git --work-tree=BookingApi reset --hard
    git --git-dir=BookingApi/.git --work-tree=BookingApi clean -xdf
done
 
git --git-dir=BookingApi/.git --work-tree=BookingApi checkout master

As you can tell, it follows the algorithm outlined above. First, it does a git log to get all the commits in the branch, and then it loops through all of them, one by one. For each commit, it compiles and runs all tests by calling out to a separate build.sh script. If the build and test step succeeds, it cleans up the working directory and moves on to the next step. If the verification step fails, it stops, so that I can examine the problem.

(The reason for the use of --git-dir and --work-tree is that I need to run the script from outside the Git repository itself; otherwise, the git clean -xdf step would delete the script files!)

This has turned out to work beautifully, and has already caught quite a number of commits that Git could happily rebase, but afterwards either couldn't compile, or had failing tests.

Running those tests in a tight loop has also finally provided some work for my multi-core processor:

Each run through those 120 commits takes about 20 minutes, so, even though today we have fast compilers, once again, we have an excuse for slacking off.

Comments

My book Dependency Injection in .NET has received a Jolt Productivity Award!

It turns out that my book Dependency Injection in .NET has received a Jolt Productivity Award 2013 in the Best Books category. This is a completely unexpected (but obviously very pleasant) surprise :D

The motivation for giving the award states that it:

"provide[s] a brilliant and remarkably readable explanation of DI"

and concludes that

"It's the definitive guide to DI and the first place architects and implementers should turn for a truly deep and complete understanding of the pattern — regardless of whether .NET is part of their toolkit.""

Although it's already been selling well, the book is still available either directly from Manning, or via Amazon.com - and, in case you didn't know, every time you buy a Manning book, you automatically also get the electronic version, which (in the case of my book), includes a Kindle version, as well as a PDF. You can also skip the physical copy and buy only the electronic version, but this option is (AFAIK) only available from Manning.

What an honor!

Note to future self: how I got Karma running from bash in Windows.

Yesterday, I spent quite a bit of time getting Karma to run from bash on my Windows 7 (x64) laptop. Since, a month from now, I'll have absolutely no recollection of how I achieved this, I'm writing it down here for the benefit of my future self, as well as any other people who may find this valuable.

Add karma to PATH #

My first problem was that 'karma' wasn't a recognized command. Although I'd used Chocolatey to install node.js, used npm to install Karma, and 'karma' was a recognized command from PowerShell, bash didn't recognize it. While I don't yet know if the following is the 100 % correct solution, I managed to make bash recognize karma by adding my karma directory to my bash path:

PATH=$PATH:~/appdata/roaming/npm/node_modules/karma/bin

This works in the session itself, but will be forgotten by bash unless you add it to ~/.bash_profile. BTW, that file doesn't exist by default in a new Git Bash installation on Windows, so you'll have to add it yourself.

Chrome path #

My next problem was that, while karma could be found, I got the error message that it couldn't find Chrome, and that I should set the CHROME_BIN environment variable. After much experimentation, I managed to make it work by setting:

CHROME_BIN=/c/Program\ Files\ \(x86\)/Google/Chrome/Application/chrome.exe
export CHROME_BIN

That looks really simple, so why did I waste so much time shaving that yak? Well, setting the environment variable was one thing, but it didn't work before I also figured out to export it.

Once again, I added these lines to my .bash_profile file. Now I can run Karma from bash on Windows.