Parametrised unit tests in Haskell by Mark Seemann
Here's a way to write parametrised unit tests in Haskell.
Sometimes you'd like to execute the same (unit) test for a number of test cases. The only thing that varies is the input values, and the expected outcome. The actual test code is the same for all test cases. Among object-oriented programmers, this is known as a parametrised test.
When I recently searched the web for how to do parametrised tests in Haskell, I could only find articles that talked about property-based testing, mostly with QuickCheck. I normally prefer property-based testing, but sometimes, I'd rather like to run a test with some deterministic test cases that are visible and readable in the code itself.
Here's one way I found that I could do that in Haskell.
Testing date and time adjustments in C# #
In an earlier article, I discussed how to model date and time adjustments as a monoid. The example code was written in C#, and I used a few tests to demonstrate that the composition of adjustments work as intended:
[Theory] [InlineData("2017-01-31T07:45:55+2", "2017-02-28T07:00:00Z")] [InlineData("2017-02-06T10:03:02+1", "2017-03-06T09:03:02Z")] [InlineData("2017-02-09T04:20:00Z" , "2017-03-09T09:00:00Z")] [InlineData("2017-02-12T16:02:11Z" , "2017-03-10T16:02:11Z")] [InlineData("2017-03-14T13:48:29-1", "2017-04-13T14:48:29Z")] public void AccumulatedAdjustReturnsCorrectResult( string dtS, string expectedS) { var dt = DateTimeOffset.Parse(dtS); var sut = DateTimeOffsetAdjustment.Accumulate( new NextMonthAdjustment(), new BusinessHoursAdjustment(), new DutchBankDayAdjustment(), new UtcAdjustment()); var actual = sut.Adjust(dt); Assert.Equal(DateTimeOffset.Parse(expectedS), actual); }
The above parametrised test uses xUnit.net (particularly its Theory feature) to execute the same test code for five test cases. Here's another example:
[Theory] [InlineData("2017-10-02T06:59:04Z", "2017-10-02T09:00:00Z")] [InlineData("2017-10-02T09:42:41Z", "2017-10-02T09:42:41Z")] [InlineData("2017-10-02T19:01:32Z", "2017-10-03T09:00:00Z")] public void AdjustReturnsCorrectResult(string dts, string expectedS) { var dt = DateTimeOffset.Parse(dts); var sut = new BusinessHoursAdjustment(); var actual = sut.Adjust(dt); Assert.Equal(DateTimeOffset.Parse(expectedS), actual); }
This one covers the three code paths through BusinessHoursAdjustment.Adjust.
These tests are similar, so they share some good and bad qualities.
On the positive side, I like how readable such tests are. The test code is only a few lines of code, and the test cases (input, and expected output) are in close proximity to the code. Unless you're on a phone with a small screen, you can easily see all of it at once.
For a problem like this, I felt that I preferred examples rather than using property-based testing. If the date and time is this, then the adjusted result should be that, and so on. When we read code, we tend to prefer examples. Good documentation often contains examples, and for those of us who consider tests documentation, including examples in tests should advance that cause.
On the negative side, tests like these still contain noise. Most of it relates to the problem that xUnit.net tests aren't first-class values. These tests actually ought to take a DateTimeOffset value as input, and compare it to another, expected DateTimeOffset value. Unfortunately, DateTimeOffset values aren't constants, so you can't use them in attributes, like the [InlineData] attribute.
There are other workarounds than the one I ultimately chose, but none that are as simple (that I'm aware of). Strings are constants, so you can put formatted date and time strings in the [InlineData] attributes, but the cost of doing this is two calls to DateTimeOffset.Parse. Perhaps this isn't a big price, you think, but it does make me wish for something prettier.
Comparing date and time #
In order to port the above tests to Haskell, I used Stack to create a new project with HUnit as the unit testing library.
The Haskell equivalent to DateTimeOffset is called ZonedTime. One problem with ZonedTime values is that you can't readily compare them; the type isn't an Eq instance. There are good reasons for that, but for the purposes of unit testing, I wanted to be able to compare them, so I defined this helper data type:
newtype ZonedTimeEq = ZT ZonedTime deriving (Show) instance Eq ZonedTimeEq where ZT (ZonedTime lt1 tz1) == ZT (ZonedTime lt2 tz2) = lt1 == lt2 && tz1 == tz2
This enables me to compare two ZonedTimeEq values, which are only considered equal if they represent the same date, the same time, and the same time zone.
Test Utility #
I also added a little function for creating ZonedTime values:
zt (y, mth, d) (h, m, s) tz = ZonedTime (LocalTime (fromGregorian y mth d) (TimeOfDay h m s)) (hoursToTimeZone tz)
The motivation is simply that, as you can tell, creating a ZonedTime value requires a verbose expression. Clearly, the ZonedTime API is flexible, but in order to define some test cases, I found it advantageous to trade readability for flexibility. The zt function enables me to compactly define some ZonedTime values for my test cases.
Testing business hours in Haskell #
In HUnit, a test is either a Test, a list of Test values, or an impure assertion. For a parametrised test, a [Test] sounded promising. At the beginning, I struggled with finding a readable way to express the tests. I wanted to be able to start with a list of test cases (inputs and expected outputs), and then fmap them to an executable test. At first, the readability goal seemed elusive, until I realised that I can also use do notation with lists (since they're monads):
adjustToBusinessHoursReturnsCorrectResult :: [Test] adjustToBusinessHoursReturnsCorrectResult = do (dt, expected) <- [ (zt (2017, 10, 2) (6, 59, 4) 0, zt (2017, 10, 2) (9, 0, 0) 0), (zt (2017, 10, 2) (9, 42, 41) 0, zt (2017, 10, 2) (9, 42, 41) 0), (zt (2017, 10, 2) (19, 1, 32) 0, zt (2017, 10, 3) (9, 0, 0) 0) ] let actual = adjustToBusinessHours dt return $ ZT expected ~=? ZT actual
This is the same test as the above C# test named AdjustReturnsCorrectResult, and it's about the same size as well. Since the test is written using do notation, you can take a list of test cases and operate on each test case at a time. Although the test creates a list of tuples, the <- arrow pulls each (ZonedTime, ZonedTime) tuple out of the list and binds it to (dt, expected).
This test literally consists of only three expressions, so according to my normal heuristic for test formatting, I don't even need white space to indicate the three phases of the AAA pattern. The first expression sets up the test case (dt, expected).
The next expression exercises the System Under Test - in this case the adjustToBusinessHours function. That's simply a function call.
The third expression verifies the result. It uses HUnit's ~=? operator to compare the expected and the actual values. Since ZonedTime isn't an Eq instance, both values are converted to ZonedTimeEq values. The ~=? operator returns a Test value, and since the entire test takes place inside a do block, you must return it. Since this particular do block takes place inside the list monad, the type of adjustToBusinessHoursReturnsCorrectResult is [Test]. I added the type annotation for the benefit of you, dear reader, but technically, it's not required.
Testing the composed function #
Translating the AccumulatedAdjustReturnsCorrectResult C# test to Haskell follows the same recipe:
composedAdjustReturnsCorrectResult :: [Test] composedAdjustReturnsCorrectResult = do (dt, expected) <- [ (zt (2017, 1, 31) ( 7, 45, 55) 2 , zt (2017, 2, 28) ( 7, 0, 0) 0), (zt (2017, 2, 6) (10, 3, 2) 1 , zt (2017, 3, 6) ( 9, 3, 2) 0), (zt (2017, 2, 9) ( 4, 20, 0) 0 , zt (2017, 3, 9) ( 9, 0, 0) 0), (zt (2017, 2, 12) (16, 2, 11) 0 , zt (2017, 3, 10) (16, 2, 11) 0), (zt (2017, 3, 14) (13, 48, 29) (-1), zt (2017, 4, 13) (14, 48, 29) 0) ] let adjustments = reverse [adjustToNextMonth, adjustToBusinessHours, adjustToDutchBankDay, adjustToUtc] let adjust = appEndo $ mconcat $ Endo <$> adjustments let actual = adjust dt return $ ZT expected ~=? ZT actual
The only notable difference is that this unit test consists of five expressions, so according to my formatting heuristic, I inserted some blank lines in order to make it easier to distinguish the three AAA phases from each other.
Running tests #
I also wrote a third test called adjustToDutchBankDayReturnsCorrectResult, but that one is so similar to the two you've already seen that I see no point showing it here. In order to run all three tests, I define the tests' main function as such:
main = defaultMain $ hUnitTestToTests $ TestList [ "adjustToBusinessHours returns correct result" ~: adjustToBusinessHoursReturnsCorrectResult, "adjustToDutchBankDay returns correct result" ~: adjustToDutchBankDayReturnsCorrectResult, "Composed adjust returns correct result" ~: composedAdjustReturnsCorrectResult ]
This uses defaultMain from test-framework, and hUnitTestToTests from test-framework-hunit.
I'm not happy about the duplication of text and test names, and the maintenance burden implied by having to explicitly add every test function to the test list. It's too easy to forget to add a test after you've written it. In the next article, I'll demonstrate an alternative way to compose the tests so that duplication is reduced.
Conclusion #
Since HUnit tests are first-class values, you can manipulate and compose them just like any other value. That includes passing them around in lists and binding them with do notation. Once you figure out how to write parametrised tests with HUnit, it's easy, readable, and elegant.
The overall configuration of the test runner, however, leaves a bit to be desired, so that's the topic for the next article.
Next: Inlined HUnit test lists.
