Simple holidays by Mark Seemann
A story about arriving at the simplest solution that could possibly work.
The Zen of Python states: Simple is better than complex. If I've met a programmer who disagrees with that, I'm not aware of it. It's hardly a controversial assertion, but what does 'simplicity' mean? Can you even identify a simple solution?
I often see software developers defaulting to complex solutions, because a simpler solution isn't immediately obvious. In retrospect, a simple solution often is obvious, but only once you've found it. Until then, it's elusive.
I'd like to share a story in which I arrived at a simple solution after several false starts. I hope it can be an inspiration.
Dutch holidays #
Recently, I had to write some code that takes into account Dutch holidays. (In order to address any confusion that could arise from this: No, I'm not Dutch, I'm Danish, but currently, I'm doing some work on a system targeting a market in the Netherlands.) Specifically, given a date, I had to find the latest possible Dutch bank day on or before that date.
For normal week days (Monday to Friday), it's easy, because such a date is already a bank date. In other words, in that case, you can simply return the input date. Also, in normal weeks, given a Saturday or Sunday, you should return the preceding Friday. The problem is, however, that some Fridays are holidays, and therefore not bank days.
Like many other countries, the Netherlands have complicated rules for determining official holidays. Here are some tricky parts:
- Some holidays always fall on the same date. One example is Bevrijdingsdag (Liberation Day), which always falls on May 5. This holiday celebrates a historic event (the end of World War II in Europe), so if you wanted to calculate bank holidays in the past, you'd have to figure out in which year this became a public holiday. Surely, at least, it must have been 1945 or later.
- Some holidays fall on specific week days. The most prominent example is Easter, where Goede Vrijdag (Good Friday) always (as the name implies) falls on a Friday. Which Friday exactly can be calculated using a complicated algorithm.
- One holiday (Koningsdag (King's Day)) celebrates the king's birthday. The date is determined by the currently reigning monarch's birthday, and it's called Queen's Day when the reigning monarch is a queen. Obviously, the exact date changes depending on who's king or queen, and you can't predict when it's going to change. And what will happen if the current monarch abdicates or dies before his or her birthday, but after the new monarch's birthday? Does that mean that there will be no such holiday that year? Or what about the converse? Could there be two such holidays if a monarch abdicates after his or her birthday, and the new monarch's birthday falls later the same year?
Figuring out if a date is a bank day, then, is what you might call an 'interesting' problem. How would you solve it? Before you read on, take a moment to consider how you'd attempt to solve the problem. If you will, you can consider the test cases immediately below to get a better sense of the problem.
Test cases #
Here's a small set of test cases that I wrote in order to describe the problem:
| Test case | Input date | Expected output |
|---|---|---|
| Monday | 2017-03-06 | 2017-03-06 |
| Tuesday | 2017-03-07 | 2017-03-07 |
| Wednesday | 2017-03-08 | 2017-03-08 |
| Thursday | 2017-03-09 | 2017-03-09 |
| Friday | 2017-03-10 | 2017-03-10 |
| Saturday | 2017-03-11 | 2017-03-10 |
| Sunday | 2017-03-12 | 2017-03-10 |
| Good Friday | 2017-04-14 | 2017-04-13 |
| Saturday after Good Friday | 2017-04-15 | 2017-04-13 |
| Sunday after Good Friday | 2017-04-16 | 2017-04-13 |
| Easter Monday | 2017-04-17 | 2017-04-13 |
| Ascension Day - Thursday | 2017-05-25 | 2017-05-24 |
| Whit Monday | 2110-05-26 | 2110-05-23 |
| Liberation Day | 9713-05-05 | 9713-05-04 |
Option: query a third-party service #
How would you solve the problem? The first solution that occurred to me was to use a third-party service. My guess is that most developers would consider this option. After all, it's essentially third-part data. The official holidays are determined by a third party, in this case the Dutch state. Surely, some Dutch official organisation would publish the list of official holidays somewhere. Perhaps, if you're lucky, there's even an on-line service you can query in order to download the list of holidays in some machine-readable format.
There are, however, problems with this alternative: if you query such a service each time you need to find an appropriate bank date, how are you going to handle network errors? What if the third-part service is (temporarily) unavailable?
Since I'm trying to figure out bank dates, you may already have guessed that I'm handling money, so it's not desirable to simple throw an exception and say that a caller would have to try again later. This could lead to loss of revenue.
Querying a third-party service every time you need to figure out a Dutch bank holiday is out of the question for that reason. It's also likely to be inefficient.
Option: cache third-party data #
Public holidays rarely change, so your next attempt could be a variation of the previous. Use third-party data, but instead of querying a third-party service every time you need the information, cache it.
The problem with caching is that you're not guaranteed that the data you seek is in the cache. At application start, caches are usually empty. You'd have to rely on making one good query to the third-party data source in order to put the data in the cache. Only if that succeeds can you use the cache. This, again, leaves you vulnerable to the normal failure modes of distributed computing. If you can't reach the third-party data source, you have nothing to put in the cache.
This can be a problem at application start, or when the cache data expires.
Using a cache reduces the risk that the data is unavailable, but it doesn't eliminate it. It also adds complexity in the form of a cache that has to be configured and managed. Granted, you can use a reusable cache library or service to minimise that cost, so it may not be a big deal. Still, when making a decision about application architecture, I think it helps to explicitly identify advantages and disadvantages.
Using a cache felt better to me, but I still wasn't happy. Too many things could go wrong.
Option: persistent cache #
An incremental improvement on the previous option would be to write the cache data to persistent storage. This takes care of the issue with the cache being empty at application start-up. You can even deal with cache expiry by keep using stale data if you can't reach the 'official' source of the data.
It leaves me a bit concerned, though, because if you allow the system to continue working with stale data, perhaps the application could enter a state where the data never updates. This could happen if the official data source moves, or changes format. In such a case, your application would keep trying to refresh the cache, and it would permanently fail. It would permanently run with stale data. Would you ever discover that problem?
My concern is that the application could silently fail. You could counter that by logging a warning somewhere, but that would introduce a permanent burden on the team responsible for operating the application. This isn't impossible, but it does constitute an extra complexity. This alternative still didn't feel good to me.
Option: cron #
Because I wasn't happy with any of the above alternatives, I started looking for different approaches to the problem. For a short while, I considered using a .NET implementation of cron, with a crontab file. As far as I can tell, though there's no easy way to define Easter using cron, so I quickly abandoned that line of inquiry.
Option: Nager.Date #
I wasn't entirely done with idea of calculating holidays on the fly. While calculating Easter is complicated, it can be done; there is a well-defined algorithm for it. Whenever I run into a general problem like this, I assume that someone has already done the required work, and this is also the case here. I quickly found an open source library called Nager.Date; I'm sure that there are other alternatives, but Nager.Date looks like it's more than good enough.
Such a library would be able to calculate all holidays for a given year, based on the algorithms embedded in it. That looked really promising.
And yet... again, I was concerned. Official holidays are, as we've already established, politically decided. Using an algorithmic approach is fundamentally wrong, because that's not really how the holidays are determined. Holidays are defined by decree; it just so happens that some of the decrees take the form of an algorithm (such as Easter).
What would happen if the Dutch state decides to add a new holiday? Or to take one away? Of when a new monarch is crowned? In order to handle such changes, we'd now have to hope that Nager.Date would be updated. We could try to make that more likely to happen by sending a pull request, but we'd still be vulnerable to a third party. What if the maintainer of Nager.Date is on vacation?
Even if you can get a change into a library like Nager.Date, how is the algorithmic approach going to deal with historic dates? If the monarch changes, you can update the library, but does it correctly handle dates in the past, where the King's Day was different?
Using an algorithm to determine a holiday seemed promising, but ultimately, I decided that I didn't like this option either.
Option: configuration file #
My main concern about using an algorithm is that it'd make it difficult to handle arbitrary changes and exceptional cases. If we'd use a configuration file, on the other hand, we could always edit the configuration file in order to add or remove holidays for a given year.
In essence, I was envisioning a configuration file that simply contained a list of holidays for each year.
That sounds fairly simple and maintainable, but from where should the data come?
You could probably download a list of official holidays for the next few years, like 2017, 2018, 2019, and so on, but the list would be finite, and probably not cover more than a few years into the future.
What if, for example, I'd only be able to find an official list that goes to 2020? What will happen, then, when our application enters 2021? To the rest of the code base, it'd look like there were no holidays in 2021.
At this time we can expect that new official lists have been published, so a programmer could obtain such a list and update the configuration file when it's time. This, unfortunately, is easy to forget. Four years in the future, perhaps none of the original programmers are left. It's more than likely that no one will remember to do this.
Option: algorithm-generated configuration file #
The problem that the configuration data could 'run out' can be addressed by initialising the configuration file with data generated algorithmically. You could, for example, ask Nager.Date to generate all the holidays for the next many years. In fact, the year 9999 is the maximum year handled by .NET's System.DateTime, so you could ask it to generate all the holidays until 9999.
That sounds like a lot, but it's only about half a megabyte of data...
This solves the problem of 'running out' of holiday data, but still enables you to edit the holiday data when it changes in the future. For example, if the King's Day changes in 2031, you can change all the King's Day values from 2031 onward, while retaining the correct values for the previous years.
This seems promising...
Option: hard-coded holidays #
I almost decided to use the previous, configuration-based solution, and I was getting ready to invent a configuration file format, and a reader for it, and so on. Then I recalled Mike Hadlow's article about the configuration complexity clock.
I'm fairly certain that the only people who would be editing a hypothetical holiday configuration file would be programmers. In that case, why put the configuration in a proprietary format? Why deal with the hassle of reading and parsing such a file? Why not put the data in code?
That's what I decided to do.
It's not a perfect solution. It's still necessary to go and change that code file when the holiday rules change. For example, when the King's Day changes, you'd have to edit the file.
Still, it's the simplest solution I could come up with. It has no moving parts, and uses a 'best effort' approach in order to guarantee that holidays will always be present. If you can come up with a better alternative, please leave a comment.
Data generation #
Nager.Date seemed useful for generating the initial set of holidays, so I wrote a small F# script that generated the necessary C# code snippets:
#r @"packages/Nager.Date.1.3.0/lib/net45/Nager.Date.dll" open System.IO open Nager.Date let formatHoliday (h : Model.PublicHoliday) = let y, m, d = h.Date.Year, h.Date.Month, h.Date.Day sprintf "new DateTime(%i, %2i, %2i), // %s/%s" y m d h.Name h.LocalName let holidays = [2017..9999] |> Seq.collect (fun y -> DateSystem.GetPublicHoliday (CountryCode.NL, y)) |> Seq.map formatHoliday File.WriteAllLines (__SOURCE_DIRECTORY__ + "/dutch-holidays.txt", holidays)
This script simply asks Nager.Date to calculate all Dutch holidays for the years 2017 to 9999, format them as C# code snippets, and write the lines to a text file. The size of that file is 4 MB, because the auto-generated code comments also take up some space.
First implementation attempt #
The next thing I did was to copy the text from dutch-holidays.txt to a C# code file, which I had already prepared with a class and a few methods that would query my generated data. The result looked like this:
public static class DateTimeExtensions { public static DateTimeOffset AdjustToLatestPrecedingDutchBankDay( this DateTimeOffset value) { var candidate = value; while (!(IsDutchBankDay(candidate.DateTime))) candidate = candidate.AddDays(-1); return candidate; } private static bool IsDutchBankDay(DateTime date) { if (date.DayOfWeek == DayOfWeek.Saturday) return false; if (date.DayOfWeek == DayOfWeek.Sunday) return false; if (dutchHolidays.Contains(date.Date)) return false; return true; } #region Dutch holidays private static DateTime[] dutchHolidays = { new DateTime(2017, 1, 1), // New Year's Day/Nieuwjaarsdag new DateTime(2017, 4, 14), // Good Friday/Goede Vrijdag new DateTime(2017, 4, 17), // Easter Monday/ Pasen new DateTime(2017, 4, 27), // King's Day/Koningsdag new DateTime(2017, 5, 5), // Liberation Day/Bevrijdingsdag new DateTime(2017, 5, 25), // Ascension Day/Hemelvaartsdag new DateTime(2017, 6, 5), // Whit Monday/Pinksteren new DateTime(2017, 12, 25), // Christmas Day/Eerste kerstdag new DateTime(2017, 12, 26), // St. Stephen's Day/Tweede kerstdag new DateTime(2018, 1, 1), // New Year's Day/Nieuwjaarsdag new DateTime(2018, 3, 30), // Good Friday/Goede Vrijdag // Lots and lots of dates... new DateTime(9999, 5, 6), // Ascension Day/Hemelvaartsdag new DateTime(9999, 5, 17), // Whit Monday/Pinksteren new DateTime(9999, 12, 25), // Christmas Day/Eerste kerstdag new DateTime(9999, 12, 26), // St. Stephen's Day/Tweede kerstdag }; #endregion }
My old computer isn't happy about having to compile 71,918 lines of C# in a single file, but it's doable, and as far as I can tell, Visual Studio caches the result of compilation, so as long as I don't change the file, there's little adverse effect.
Unit tests #
In order to verify that the implementation works, I wrote this parametrised test:
public class DateTimeExtensionsTests { [Theory] [InlineData("2017-03-06", "2017-03-06")] // Monday [InlineData("2017-03-07", "2017-03-07")] // Tuesday [InlineData("2017-03-08", "2017-03-08")] // Wednesday [InlineData("2017-03-09", "2017-03-09")] // Thursday [InlineData("2017-03-10", "2017-03-10")] // Friday [InlineData("2017-03-11", "2017-03-10")] // Saturday [InlineData("2017-03-12", "2017-03-10")] // Sunday [InlineData("2017-04-14", "2017-04-13")] // Good Friday [InlineData("2017-04-15", "2017-04-13")] // Saturday after Good Friday [InlineData("2017-04-16", "2017-04-13")] // Sunday after Good Friday [InlineData("2017-04-17", "2017-04-13")] // Easter Monday [InlineData("2017-05-25", "2017-05-24")] // Ascension Day - Thursday [InlineData("2110-05-26", "2110-05-23")] // Whit Monday [InlineData("9713-05-05", "9713-05-04")] // Liberation Day public void AdjustToLatestPrecedingDutchBankDayReturnsCorrectResult( string sutS, string expectedS) { var sut = DateTimeOffset.Parse(sutS); var actual = sut.AdjustToLatestPrecedingDutchBankDay(); Assert.Equal(DateTimeOffset.Parse(expectedS), actual); } }
All test cases pass. This works in [Theory], but unfortunately, it turns out, it doesn't work in practice.
When used in an ASP.NET Web API application, AdjustToLatestPrecedingDutchBankDay throws a StackOverflowException. It took me a while to figure out why, but it turns out that the stack size is smaller in IIS than when you run a 'normal' .NET process, such as an automated test.
System.DateTime is a value type, and as far as I can tell, it uses some stack space during initialisation. When the DateTimeExtensions class is first used, the static dutchHolidays array is initialised, and that uses enough stack space to exhaust the stack when running in IIS.
Final implementation #
The stack space problem seems to be related to DateTime initialisation. If I store a similar number of 64-bit integers in an array, it seems that there's no problem.
First, I had to modify the formatHoliday function:
let formatHoliday (h : Model.PublicHoliday) = let t, y, m, d = h.Date.Ticks, h.Date.Year, h.Date.Month, h.Date.Day sprintf "%19iL, // %i-%02i-%02i, %s/%s" t y m d h.Name h.LocalName
This enabled me to generate a new file with C# code fragments, but now containing ticks instead of DateTime values. Copying those C# fragments into my file gave me this:
public static class DateTimeExtensions { public static DateTimeOffset AdjustToLatestPrecedingDutchBankDay( this DateTimeOffset value) { var candidate = value; while (!(IsDutchBankDay(candidate.DateTime))) candidate = candidate.AddDays(-1); return candidate; } private static bool IsDutchBankDay(DateTime date) { if (date.DayOfWeek == DayOfWeek.Saturday) return false; if (date.DayOfWeek == DayOfWeek.Sunday) return false; if (dutchHolidays.Contains(date.Date.Ticks)) return false; return true; } #region Dutch holidays private static long[] dutchHolidays = { 636188256000000000L, // 2017-01-01, New Year's Day/Nieuwjaarsdag 636277248000000000L, // 2017-04-14, Good Friday/Goede Vrijdag 636279840000000000L, // 2017-04-17, Easter Monday/ Pasen 636288480000000000L, // 2017-04-27, King's Day/Koningsdag 636295392000000000L, // 2017-05-05, Liberation Day/Bevrijdingsdag 636312672000000000L, // 2017-05-25, Ascension Day/Hemelvaartsdag 636322176000000000L, // 2017-06-05, Whit Monday/Pinksteren 636497568000000000L, // 2017-12-25, Christmas Day/Eerste kerstdag 636498432000000000L, // 2017-12-26, St. Stephen's Day/Tweede kerstdag 636503616000000000L, // 2018-01-01, New Year's Day/Nieuwjaarsdag 636579648000000000L, // 2018-03-30, Good Friday/Goede Vrijdag // Lots and lots of dates... 3155171616000000000L, // 9999-05-06, Ascension Day/Hemelvaartsdag 3155181120000000000L, // 9999-05-17, Whit Monday/Pinksteren 3155372928000000000L, // 9999-12-25, Christmas Day/Eerste kerstdag 3155373792000000000L, // 9999-12-26, St. Stephen's Day/Tweede kerstdag }; #endregion }
That implementation still passes all tests, and works at in practice as well.
Conclusion #
It took me some time to find a satisfactory solution. I had more than once false start, until I ultimately arrived at the solution I've described here. I consider it simple because it's self-contained, deterministic, easy to understand, and fairly easy to maintain. I even left a comment in the code (not shown here) that described how to recreate the configuration data using the F# script shown here.
The first solution that comes into your mind may not be the simplest solution, but if you take some time to consider alternatives, you may save yourself and your colleagues some future grief.
Comments
What do you think about creating an "admin page" that would allow users to configure the bank holidays themselves (which would then be persisted in the application database)? This moves the burden of correctness to the administrators of the application, who I'm sure are highly motivated to get this right - as well as maintain it. It also removes the need for a deployment in the face of changing holidays.
For the sake of convenience, you could still "seed" the database with the values generated by your F# script
jadarnel27, thank you for writing. Your suggestion could be an option as well, but is hardly the simplest solution. In order to implement that, I'd need to add an administration web site for my application, program the user interface, connect the administration site and my original application (a REST API) to a persistent data source, write code for input validation, etcetera.
Apart from all that work, the bank holidays would have to be stored in an out-of-process data store, such as a database or NoSQL data store, because the REST API that needs this feature is running in a server farm. This adds latency to each lookup, as well as a potential error source. What should happen if the connection to the data store is broken? Additionally, such a data store should be backed up, so we'd also need to establish an operational procedure to ensure that that happens.
It was never a requirement that the owners of the application should be able to administer this themselves. It's certainly an option, but it's so much more complex than the solution outlined above that I think one should start by making a return-on-investment analysis.
Another option: Calculate the holidays! I think you might find som useful code in my HolidaysAPI. It is even in F#, and the Web project is based upon a course or blog post by you.
Alf, thank you for writing. Apart from the alternative library, how is that different from the option I covered under the heading Option: Nager.Date?
Great post, thanks. The minor point here is that it is probably not so effective to do Contains over long[]. I'd consider using something that can check the value existance faster.
EQR, thank you for writing. The performance profile of the implementation wasn't my main concern with this article, so it's likely that it can be improved.
I did do some lackadaisical performance testing, but didn't detect any measurable difference between the implementation shown here, and one using a HashSet. On the other hand, there are other options I didn't try at all. One of these could be to perform a binary search, since the array is already ordered.
Hi Mark, thanks for this post. One small note on your selection of bank holidays. Liberation day is an official bank holiday, but only for civil servants, for us 'normal' people this only happens every 5 years. This is reflected in Nager.Date wrong here
Thomas, thank you for writing. That just goes to show, I think, that holiday calculation is as complicated as any other 'business logic'. It should be treated as such, and not as an algorithmic calculation.