ploeh blog 2017-05-15T11:04:07+00:00 Mark Seemann danish software design http://blog.ploeh.dk Simple holidays http://blog.ploeh.dk/2017/04/24/simple-holidays 2017-04-24T13:42:00+00:00 Mark Seemann <div id="post"> <p> <em>A story about arriving at the simplest solution that could possibly work.</em> </p> <p> The Zen of Python states: <a href="https://www.python.org/dev/peps/pep-0020">Simple is better than complex</a>. If I've met a programmer who disagrees with that, I'm not aware of it. It's hardly a controversial assertion, but what does 'simplicity' mean? Can you even identify a simple solution? </p> <p> I often see software developers defaulting to complex solutions, because a simpler solution isn't immediately obvious. In retrospect, a simple solution often <em>is</em> obvious, but only once you've found it. Until then, it's elusive. </p> <p> I'd like to share a story in which I arrived at a simple solution after several false starts. I hope it can be an inspiration. </p> <p> <strong>Dutch holidays</strong> </p> <p> Recently, I had to write some code that takes into account Dutch holidays. (In order to address any confusion that could arise from this: No, I'm not Dutch, I'm Danish, but currently, I'm doing some work on a system targeting a market in the Netherlands.) Specifically, given a date, I had to find the latest possible Dutch bank day on or before that date. </p> <p> For normal week days (Monday to Friday), it's easy, because such a date is already a bank date. In other words, in that case, you can simply return the input date. Also, in normal weeks, given a Saturday or Sunday, you should return the preceding Friday. The problem is, however, that some Fridays are holidays, and therefore not bank days. </p> <p> Like many other countries, the Netherlands have complicated rules for determining official holidays. Here are some tricky parts: <ul> <li> Some holidays always fall on the same date. One example is <em>Bevrijdingsdag (Liberation Day)</em>, which always falls on May 5. This holiday celebrates a historic event (the end of World War II in Europe), so if you wanted to calculate bank holidays in the past, you'd have to figure out in which year this became a public holiday. Surely, at least, it must have been 1945 or later.</li> <li> Some holidays fall on specific week days. The most prominent example is Easter, where <em>Goede Vrijdag (Good Friday)</em> always (as the name implies) falls on a Friday. Which Friday exactly can be calculated using a complicated algorithm. </li> <li> One holiday (<em>Koningsdag (King's Day)</em>) celebrates the king's birthday. The date is determined by the currently reigning monarch's birthday, and it's called <em>Queen's Day</em> when the reigning monarch is a queen. Obviously, the exact date changes depending on who's king or queen, and you can't predict when it's going to change. And what will happen if the current monarch abdicates or dies <em>before</em> his or her birthday, but <em>after</em> the new monarch's birthday? Does that mean that there will be no such holiday that year? Or what about the converse? Could there be <em>two</em> such holidays if a monarch abdicates <em>after</em> his or her birthday, and the new monarch's birthday falls later the same year? </li> </ul> Such problems aren't particular to the Netherlands. In Denmark, we can find similar examples, as I think you can do in many other countries. Ultimately, what constitutes an official holiday is a political decision. </p> <p> Figuring out if a date is a bank day, then, is what you might call an 'interesting' problem. How would you solve it? Before you read on, take a moment to consider how you'd attempt to solve the problem. If you will, you can consider the test cases immediately below to get a better sense of the problem. </p> <p> <strong>Test cases</strong> </p> <p> Here's a small set of test cases that I wrote in order to describe the problem: <table> <thead> <tr> <th>Test case</th> <th>Input date</th> <th>Expected output</th> </tr> </thead> <tbody> <tr><td>Monday</td><td>2017-03-06</td><td>2017-03-06</td></tr> <tr><td>Tuesday</td><td>2017-03-07</td><td>2017-03-07</td></tr> <tr><td>Wednesday</td><td>2017-03-08</td><td>2017-03-08</td></tr> <tr><td>Thursday</td><td>2017-03-09</td><td>2017-03-09</td></tr> <tr><td>Friday</td><td>2017-03-10</td><td>2017-03-10</td></tr> <tr><td>Saturday</td><td>2017-03-11</td><td>2017-03-10</td></tr> <tr><td>Sunday</td><td>2017-03-12</td><td>2017-03-10</td></tr> <tr><td>Good Friday</td><td>2017-04-14</td><td>2017-04-13</td></tr> <tr><td>Saturday after Good Friday</td><td>2017-04-15</td><td>2017-04-13</td></tr> <tr><td>Sunday after Good Friday</td><td>2017-04-16</td><td>2017-04-13</td></tr> <tr><td>Easter Monday</td><td>2017-04-17</td><td>2017-04-13</td></tr> <tr><td>Ascension Day - Thursday</td><td>2017-05-25</td><td>2017-05-24</td></tr> <tr><td>Whit Monday</td><td>2110-05-26</td><td>2110-05-23</td></tr> <tr><td>Liberation Day</td><td>9713-05-05</td><td>9713-05-04</td></tr> </tbody> </table> You'll notice that while I wrote most of my test cases in the near future (they're actually already in the past, now), I also added some far future dates for good measure. This assumes that the Netherlands will still celebrate Christian religious holidays in a hundred years from now, or their liberation day in 9713. That the Netherlands still exist then as the country we know today is a more than dubious assumption. </p> <p> <strong>Option: query a third-party service</strong> </p> <p> How would you solve the problem? The first solution that occurred to me was to use a third-party service. My guess is that most developers would consider this option. After all, it's essentially third-part data. The official holidays are determined by a third party, in this case the Dutch state. Surely, some Dutch official organisation would publish the list of official holidays somewhere. Perhaps, if you're lucky, there's even an on-line service you can query in order to download the list of holidays in some machine-readable format. </p> <p> There are, however, problems with this alternative: if you query such a service each time you need to find an appropriate bank date, how are you going to handle network errors? What if the third-part service is (temporarily) unavailable? </p> <p> Since I'm trying to figure out bank dates, you may already have guessed that I'm handling money, so it's not desirable to simple throw an exception and say that a caller would have to try again later. This could lead to loss of revenue. </p> <p> Querying a third-party service <em>every time</em> you need to figure out a Dutch bank holiday is out of the question for that reason. It's also likely to be inefficient. </p> <p> <strong>Option: cache third-party data</strong> </p> <p> Public holidays rarely change, so your next attempt could be a variation of the previous. Use third-party data, but instead of querying a third-party service every time you need the information, cache it. </p> <p> The problem with caching is that you're not guaranteed that the data you seek is in the cache. At application start, caches are usually empty. You'd have to rely on making one good query to the third-party data source in order to put the data in the cache. Only if that succeeds can you use the cache. This, again, leaves you vulnerable to the normal failure modes of distributed computing. If you can't reach the third-party data source, you have nothing to put in the cache. </p> <p> This can be a problem at application start, or when the cache data expires. </p> <p> Using a cache <em>reduces</em> the risk that the data is unavailable, but it doesn't eliminate it. It also adds complexity in the form of a cache that has to be configured and managed. Granted, you can use a reusable cache library or service to minimise that cost, so it may not be a big deal. Still, when making a decision about application architecture, I think it helps to explicitly identify advantages and disadvantages. </p> <p> Using a cache felt better to me, but I still wasn't happy. Too many things could go wrong. </p> <p> <strong>Option: persistent cache</strong> </p> <p> An incremental improvement on the previous option would be to write the cache data to persistent storage. This takes care of the issue with the cache being empty at application start-up. You can even deal with cache expiry by keep using stale data if you can't reach the 'official' source of the data. </p> <p> It leaves me a bit concerned, though, because if you allow the system to continue working with stale data, perhaps the application could enter a state where the data <em>never</em> updates. This could happen if the official data source moves, or changes format. In such a case, your application would keep trying to refresh the cache, and it would permanently fail. It would permanently run with stale data. Would you ever discover that problem? </p> <p> My concern is that the application could silently fail. You could counter that by logging a warning somewhere, but that would introduce a permanent burden on the team responsible for operating the application. This isn't impossible, but it does constitute an extra complexity. This alternative still didn't feel good to me. </p> <p> <strong>Option: cron</strong> </p> <p> Because I wasn't happy with any of the above alternatives, I started looking for different approaches to the problem. For a short while, I considered using a .NET implementation of <a href="https://en.wikipedia.org/wiki/Cron">cron</a>, with a crontab file. As far as I can tell, though there's no easy way to define Easter using cron, so I quickly abandoned that line of inquiry. </p> <p> <strong>Option: Nager.Date</strong> </p> <p> I wasn't entirely done with idea of calculating holidays on the fly. While calculating Easter is complicated, it can be done; there <em>is</em> a well-defined algorithm for it. Whenever I run into a general problem like this, I assume that someone has already done the required work, and this is also the case here. I quickly found an open source library called <a href="https://github.com/tinohager/Nager.Date">Nager.Date</a>; I'm sure that there are other alternatives, but Nager.Date looks like it's more than good enough. </p> <p> Such a library would be able to calculate all holidays for a given year, based on the algorithms embedded in it. That looked really promising. </p> <p> And yet... again, I was concerned. Official holidays are, as we've already established, politically decided. Using an algorithmic approach is <em>fundamentally</em> wrong, because that's not really how the holidays are determined. Holidays are defined by decree; it just so happens that some of the decrees take the form of an algorithm (such as Easter). </p> <p> What would happen if the Dutch state decides to add a new holiday? Or to take one away? Of when a new monarch is crowned? In order to handle such changes, we'd now have to hope that Nager.Date would be updated. We could try to make that more likely to happen by sending a pull request, but we'd still be vulnerable to a third party. What if the maintainer of Nager.Date is on vacation? </p> <p> Even if you can get a change into a library like Nager.Date, how is the algorithmic approach going to deal with historic dates? If the monarch changes, you can update the library, but does it correctly handle dates in the past, where the King's Day was different? </p> <p> Using an algorithm to determine a holiday seemed promising, but ultimately, I decided that I didn't like this option either. </p> <p> <strong>Option: configuration file</strong> </p> <p> My main concern about using an algorithm is that it'd make it difficult to handle arbitrary changes and exceptional cases. If we'd use a configuration file, on the other hand, we could always edit the configuration file in order to add or remove holidays for a given year. </p> <p> In essence, I was envisioning a configuration file that simply contained a list of holidays for each year. </p> <p> That sounds fairly simple and maintainable, but from where should the data come? </p> <p> You could probably download a list of official holidays for the next few years, like 2017, 2018, 2019, and so on, but the list would be finite, and probably not cover more than a few years into the future. </p> <p> What if, for example, I'd only be able to find an official list that goes to 2020? What will happen, then, when our application enters 2021? To the rest of the code base, it'd look like there were no holidays in 2021. </p> <p> At this time we can expect that new official lists have been published, so a programmer could obtain such a list and update the configuration file when it's time. This, unfortunately, is easy to forget. Four years in the future, perhaps none of the original programmers are left. It's more than likely that no one will remember to do this. </p> <p> <strong>Option: algorithm-generated configuration file</strong> </p> <p> The problem that the configuration data could 'run out' can be addressed by initialising the configuration file with data generated algorithmically. You could, for example, ask Nager.Date to generate all the holidays for the next many years. In fact, the year 9999 is the maximum year handled by .NET's <code>System.DateTime</code>, so you could ask it to generate all the holidays until 9999. </p> <p> That sounds like a lot, but it's only about half a megabyte of data... </p> <p> This solves the problem of 'running out' of holiday data, but still enables you to edit the holiday data when it changes in the future. For example, if the King's Day changes in 2031, you can change all the King's Day values from 2031 onward, while retaining the correct values for the previous years. </p> <p> This seems promising... </p> <p> <strong>Option: hard-coded holidays</strong> </p> <p> I almost decided to use the previous, configuration-based solution, and I was getting ready to invent a configuration file format, and a reader for it, and so on. Then I recalled <a href="http://mikehadlow.blogspot.com">Mike Hadlow</a>'s article about the <a href="http://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html">configuration complexity clock</a>. </p> <p> I'm fairly certain that the only people who would be editing a hypothetical holiday configuration file would be programmers. In that case, why put the configuration in a proprietary format? Why deal with the hassle of reading and parsing such a file? Why not put the data in code? </p> <p> That's what I decided to do. </p> <p> It's not a perfect solution. It's still necessary to go and change that code file when the holiday rules change. For example, when the King's Day changes, you'd have to edit the file. </p> <p> Still, it's the <em>simplest</em> solution I could come up with. It has no moving parts, and uses a 'best effort' approach in order to guarantee that holidays will always be present. If you can come up with a better alternative, please leave a comment. </p> <p> <strong>Data generation</strong> </p> <p> Nager.Date seemed useful for generating the initial set of holidays, so I wrote a small F# script that generated the necessary C# code snippets: </p> <p> <pre><span style="color:blue;">#r</span>&nbsp;<span style="color:#a31515;">@&quot;packages/Nager.Date.1.3.0/lib/net45/Nager.Date.dll&quot;</span> <span style="color:blue;">open</span>&nbsp;System.IO <span style="color:blue;">open</span>&nbsp;Nager.Date <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">formatHoliday</span>&nbsp;(h&nbsp;:&nbsp;Model.<span style="color:teal;">PublicHoliday</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;y,&nbsp;m,&nbsp;d&nbsp;=&nbsp;h.Date.Year,&nbsp;h.Date.Month,&nbsp;h.Date.Day &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">sprintf</span>&nbsp;<span style="color:#a31515;">&quot;new&nbsp;DateTime(</span><span style="color:teal;">%i</span><span style="color:#a31515;">,&nbsp;</span><span style="color:teal;">%2i</span><span style="color:#a31515;">,&nbsp;</span><span style="color:teal;">%2i</span><span style="color:#a31515;">),&nbsp;//&nbsp;</span><span style="color:teal;">%s</span><span style="color:#a31515;">/</span><span style="color:teal;">%s</span><span style="color:#a31515;">&quot;</span>&nbsp;y&nbsp;m&nbsp;d&nbsp;h.Name&nbsp;h.LocalName <span style="color:blue;">let</span>&nbsp;holidays&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;[2017..9999] &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Seq</span>.<span style="color:navy;">collect</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;y&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:teal;">DateSystem</span>.<span style="color:navy;">GetPublicHoliday</span>&nbsp;(<span style="color:teal;">CountryCode</span>.NL,&nbsp;y)) &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Seq</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">formatHoliday</span> <span style="color:teal;">File</span>.<span style="color:navy;">WriteAllLines</span>&nbsp;(<span style="color:blue;">__SOURCE_DIRECTORY__</span>&nbsp;+&nbsp;<span style="color:#a31515;">&quot;/dutch-holidays.txt&quot;</span>,&nbsp;holidays)</pre> </p> <p> This script simply asks Nager.Date to calculate all Dutch holidays for the years 2017 to 9999, format them as C# code snippets, and write the lines to a text file. The size of that file is 4 MB, because the auto-generated code comments also take up some space. </p> <p> <strong>First implementation attempt</strong> </p> <p> The next thing I did was to copy the text from <code>dutch-holidays.txt</code> to a C# code file, which I had already prepared with a class and a few methods that would query my generated data. The result looked like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">DateTimeExtensions</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">DateTimeOffset</span>&nbsp;AdjustToLatestPrecedingDutchBankDay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">DateTimeOffset</span>&nbsp;value) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;candidate&nbsp;=&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">while</span>&nbsp;(!(IsDutchBankDay(candidate.DateTime))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;=&nbsp;candidate.AddDays(-1); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;candidate; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsDutchBankDay(<span style="color:#2b91af;">DateTime</span>&nbsp;date) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(date.DayOfWeek&nbsp;==&nbsp;<span style="color:#2b91af;">DayOfWeek</span>.Saturday) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(date.DayOfWeek&nbsp;==&nbsp;<span style="color:#2b91af;">DayOfWeek</span>.Sunday) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dutchHolidays.Contains(date.Date)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">#region</span>&nbsp;Dutch&nbsp;holidays &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">DateTime</span>[]&nbsp;dutchHolidays&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;1,&nbsp;&nbsp;1),&nbsp;<span style="color:green;">//&nbsp;New&nbsp;Year&#39;s&nbsp;Day/Nieuwjaarsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;4,&nbsp;14),&nbsp;<span style="color:green;">//&nbsp;Good&nbsp;Friday/Goede&nbsp;Vrijdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;4,&nbsp;17),&nbsp;<span style="color:green;">//&nbsp;Easter&nbsp;Monday/&nbsp;Pasen</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;4,&nbsp;27),&nbsp;<span style="color:green;">//&nbsp;King&#39;s&nbsp;Day/Koningsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;5,&nbsp;&nbsp;5),&nbsp;<span style="color:green;">//&nbsp;Liberation&nbsp;Day/Bevrijdingsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;5,&nbsp;25),&nbsp;<span style="color:green;">//&nbsp;Ascension&nbsp;Day/Hemelvaartsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;&nbsp;6,&nbsp;&nbsp;5),&nbsp;<span style="color:green;">//&nbsp;Whit&nbsp;Monday/Pinksteren</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;12,&nbsp;25),&nbsp;<span style="color:green;">//&nbsp;Christmas&nbsp;Day/Eerste&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2017,&nbsp;12,&nbsp;26),&nbsp;<span style="color:green;">//&nbsp;St.&nbsp;Stephen&#39;s&nbsp;Day/Tweede&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2018,&nbsp;&nbsp;1,&nbsp;&nbsp;1),&nbsp;<span style="color:green;">//&nbsp;New&nbsp;Year&#39;s&nbsp;Day/Nieuwjaarsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2018,&nbsp;&nbsp;3,&nbsp;30),&nbsp;<span style="color:green;">//&nbsp;Good&nbsp;Friday/Goede&nbsp;Vrijdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Lots&nbsp;and&nbsp;lots&nbsp;of&nbsp;dates...</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(9999,&nbsp;&nbsp;5,&nbsp;&nbsp;6),&nbsp;<span style="color:green;">//&nbsp;Ascension&nbsp;Day/Hemelvaartsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(9999,&nbsp;&nbsp;5,&nbsp;17),&nbsp;<span style="color:green;">//&nbsp;Whit&nbsp;Monday/Pinksteren</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(9999,&nbsp;12,&nbsp;25),&nbsp;<span style="color:green;">//&nbsp;Christmas&nbsp;Day/Eerste&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(9999,&nbsp;12,&nbsp;26),&nbsp;<span style="color:green;">//&nbsp;St.&nbsp;Stephen&#39;s&nbsp;Day/Tweede&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">#endregion</span> }</pre> </p> <p> My old computer isn't happy about having to compile 71,918 lines of C# in a single file, but it's doable, and as far as I can tell, Visual Studio caches the result of compilation, so as long as I don't change the file, there's little adverse effect. </p> <p> <strong>Unit tests</strong> </p> <p> In order to verify that the implementation works, I wrote this parametrised test: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">DateTimeExtensionsTests</span> { &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">Theory</span>] &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-06&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-06&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Monday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-07&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-07&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Tuesday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-08&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-08&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Wednesday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-09&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-09&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Thursday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-10&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-10&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Friday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-11&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-10&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Saturday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-03-12&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-03-10&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Sunday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-04-14&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-04-13&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Good&nbsp;Friday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-04-15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-04-13&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Saturday&nbsp;after&nbsp;Good&nbsp;Friday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-04-16&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-04-13&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Sunday&nbsp;after&nbsp;Good&nbsp;Friday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-04-17&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-04-13&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Easter&nbsp;Monday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2017-05-25&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2017-05-24&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Ascension&nbsp;Day&nbsp;-&nbsp;Thursday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2110-05-26&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;2110-05-23&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Whit&nbsp;Monday</span> &nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;9713-05-05&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;9713-05-04&quot;</span>)]&nbsp;<span style="color:green;">//&nbsp;Liberation&nbsp;Day</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AdjustToLatestPrecedingDutchBankDayReturnsCorrectResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;sutS, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;expectedS) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:#2b91af;">DateTimeOffset</span>.Parse(sutS); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.AdjustToLatestPrecedingDutchBankDay(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(<span style="color:#2b91af;">DateTimeOffset</span>.Parse(expectedS),&nbsp;actual); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> All test cases pass. This works in <code>[Theory]</code>, but unfortunately, it turns out, it doesn't work in practice. </p> <p> When used in an ASP.NET Web API application, <code>AdjustToLatestPrecedingDutchBankDay</code> throws a <code>StackOverflowException</code>. It took me a while to figure out why, but it turns out that the stack size is smaller in IIS than when you run a 'normal' .NET process, such as an automated test. </p> <p> <code>System.DateTime</code> is a value type, and as far as I can tell, it uses some stack space during initialisation. When the <code>DateTimeExtensions</code> class is first used, the static <code>dutchHolidays</code> array is initialised, and that uses enough stack space to exhaust the stack when running in IIS. </p> <p> <strong>Final implementation</strong> </p> <p> The stack space problem seems to be related to <code>DateTime</code> initialisation. If I store a similar number of 64-bit integers in an array, it seems that there's no problem. </p> <p> First, I had to modify the <code>formatHoliday</code> function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">formatHoliday</span>&nbsp;(h&nbsp;:&nbsp;Model.PublicHoliday)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;t,&nbsp;y,&nbsp;m,&nbsp;d&nbsp;=&nbsp;h.Date.Ticks,&nbsp;h.Date.Year,&nbsp;h.Date.Month,&nbsp;h.Date.Day &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">sprintf</span>&nbsp;<span style="color:#a31515;">&quot;</span><span style="color:teal;">%19i</span><span style="color:#a31515;">L,&nbsp;//&nbsp;</span><span style="color:teal;">%i</span><span style="color:#a31515;">-</span><span style="color:teal;">%02i</span><span style="color:#a31515;">-</span><span style="color:teal;">%02i</span><span style="color:#a31515;">,&nbsp;</span><span style="color:teal;">%s</span><span style="color:#a31515;">/</span><span style="color:teal;">%s</span><span style="color:#a31515;">&quot;</span>&nbsp;t&nbsp;y&nbsp;m&nbsp;d&nbsp;h.Name&nbsp;h.LocalName</pre> </p> <p> This enabled me to generate a new file with C# code fragments, but now containing ticks instead of <code>DateTime</code> values. Copying those C# fragments into my file gave me this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">DateTimeExtensions</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">DateTimeOffset</span>&nbsp;AdjustToLatestPrecedingDutchBankDay( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">DateTimeOffset</span>&nbsp;value) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;candidate&nbsp;=&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">while</span>&nbsp;(!(IsDutchBankDay(candidate.DateTime))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;=&nbsp;candidate.AddDays(-1); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;candidate; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsDutchBankDay(<span style="color:#2b91af;">DateTime</span>&nbsp;date) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(date.DayOfWeek&nbsp;==&nbsp;<span style="color:#2b91af;">DayOfWeek</span>.Saturday) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(date.DayOfWeek&nbsp;==&nbsp;<span style="color:#2b91af;">DayOfWeek</span>.Sunday) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dutchHolidays.Contains(date.Date.Ticks)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">#region</span>&nbsp;Dutch&nbsp;holidays &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">long</span>[]&nbsp;dutchHolidays&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636188256000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-01-01,&nbsp;New&nbsp;Year&#39;s&nbsp;Day/Nieuwjaarsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636277248000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-04-14,&nbsp;Good&nbsp;Friday/Goede&nbsp;Vrijdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636279840000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-04-17,&nbsp;Easter&nbsp;Monday/&nbsp;Pasen</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636288480000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-04-27,&nbsp;King&#39;s&nbsp;Day/Koningsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636295392000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-05-05,&nbsp;Liberation&nbsp;Day/Bevrijdingsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636312672000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-05-25,&nbsp;Ascension&nbsp;Day/Hemelvaartsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636322176000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-06-05,&nbsp;Whit&nbsp;Monday/Pinksteren</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636497568000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-12-25,&nbsp;Christmas&nbsp;Day/Eerste&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636498432000000000L,&nbsp;<span style="color:green;">//&nbsp;2017-12-26,&nbsp;St.&nbsp;Stephen&#39;s&nbsp;Day/Tweede&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636503616000000000L,&nbsp;<span style="color:green;">//&nbsp;2018-01-01,&nbsp;New&nbsp;Year&#39;s&nbsp;Day/Nieuwjaarsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;636579648000000000L,&nbsp;<span style="color:green;">//&nbsp;2018-03-30,&nbsp;Good&nbsp;Friday/Goede&nbsp;Vrijdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Lots&nbsp;and&nbsp;lots&nbsp;of&nbsp;dates...</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3155171616000000000L,&nbsp;<span style="color:green;">//&nbsp;9999-05-06,&nbsp;Ascension&nbsp;Day/Hemelvaartsdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3155181120000000000L,&nbsp;<span style="color:green;">//&nbsp;9999-05-17,&nbsp;Whit&nbsp;Monday/Pinksteren</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3155372928000000000L,&nbsp;<span style="color:green;">//&nbsp;9999-12-25,&nbsp;Christmas&nbsp;Day/Eerste&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3155373792000000000L,&nbsp;<span style="color:green;">//&nbsp;9999-12-26,&nbsp;St.&nbsp;Stephen&#39;s&nbsp;Day/Tweede&nbsp;kerstdag</span> &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">#endregion</span> }</pre> </p> <p> That implementation still passes all tests, <em>and</em> works at in practice as well. </p> <p> <strong>Conclusion</strong> </p> <p> It took me some time to find a satisfactory solution. I had more than once false start, until I ultimately arrived at the solution I've described here. I consider it simple because it's self-contained, deterministic, easy to understand, and fairly easy to maintain. I even left a comment in the code (not shown here) that described how to recreate the configuration data using the F# script shown here. </p> <p> The first solution that comes into your mind may not be the simplest solution, but if you take some time to consider alternatives, you may save yourself and your colleagues some future grief. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment"> <div class="comment-author"><a href="http://stackoverflow.com/users/861565/jadarnel27">jadarnel27</a></div> <div class="comment-content"> <p> What do you think about creating an "admin page" that would allow users to configure the bank holidays themselves (which would then be persisted in the application database)? This moves the burden of correctness to the administrators of the application, who I'm sure are highly motivated to get this right - as well as maintain it. It also removes the need for a deployment in the face of changing holidays. </p> <p> For the sake of convenience, you could still "seed" the database with the values generated by your F# script </p> </div> <div class="comment-date">2017-04-25 20:00 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> jadarnel27, thank you for writing. Your suggestion could be an option as well, but is hardly the simplest solution. In order to implement that, I'd need to add an administration web site for my application, program the user interface, connect the administration site and my original application (a REST API) to a persistent data source, write code for input validation, etcetera. </p> <p> Apart from all that work, the bank holidays would have to be stored in an out-of-process data store, such as a database or NoSQL data store, because the REST API that needs this feature is running in a server farm. This adds latency to each lookup, as well as a potential error source. What should happen if the connection to the data store is broken? Additionally, such a data store should be backed up, so we'd also need to establish an operational procedure to ensure that that happens. </p> <p> It was never a requirement that the owners of the application should be able to administer this themselves. It's certainly an option, but it's so much more complex than the solution outlined above that I think one should start by making a return-on-investment analysis. </p> </div> <div class="comment-date">2017-04-26 7:55 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://www.lefdal.cc/info">Alf Kåre Lefdal</a></div> <div class="comment-content"> <p> Another option: Calculate the holidays! I think you might find som useful code in my <a href="https://github.com/aklefdal/HolidaysApi/tree/master/HolidaysApi">HolidaysAPI</a>. It is even in F#, and the Web project is based upon a course or blog post by you. </p> </div> <div class="comment-date">2017-04-26 19:26 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Alf, thank you for writing. Apart from the alternative library, how is that different from the option I covered under the heading <em>Option: Nager.Date?</em> </p> </div> <div class="comment-date">2017-04-27 5:38 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.enazarov.ru">EQR</a></div> <div class="comment-content"> <p> Great post, thanks. The minor point here is that it is probably not so effective to do Contains over long[]. I'd consider using something that can check the value existance faster. </p> </div> <div class="comment-date">2017-04-30 19:00 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> EQR, thank you for writing. The performance profile of the implementation wasn't my main concern with this article, so it's likely that it can be improved. </p> <p> I did do some lackadaisical performance testing, but didn't detect any measurable difference between the implementation shown here, and one using a HashSet. On the other hand, there are other options I didn't try at all. One of these could be to perform a binary search, since the array is already ordered. </p> </div> <div class="comment-date">2017-05-15 11:03 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. A reusable ApiController Adapter http://blog.ploeh.dk/2017/03/30/a-reusable-apicontroller-adapter 2017-03-30T10:17:00+00:00 Mark Seemann <div id="post"> <p> <em>Refactor ApiController Adapters to a single, reusable base class.</em> </p> <p> Regular readers of this blog will know that I write many RESTful APIs in F#, using ASP.NET Web API. Since I like to write functional F#, but ASP.NET Web API is an object-oriented framework, I prefer to escape the object-oriented framework as soon as possible. (In general, it makes good architectural sense to write most of your code as framework-independent as possible.) </p> <p> (Regular readers will also have <a href="http://blog.ploeh.dk/2017/01/03/decoupling-application-errors-from-domain-models">seen the above paragraph before</a>.) </p> <p> To bridge the gap between the object-oriented framework and my functional code, I implement Controller classes like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> The above example is a Controller that handles <a href="http://blog.ploeh.dk/2016/12/16/from-rest-to-algebraic-data">incoming payment data</a>. It immediately delegates all work to an injected <code>imp</code> function, and pattern matches on the return value in order to return correct responses. </p> <p> This <code>CreditCardController</code> extends ApiController in order to work nicely with ASP.NET Web API, while the injected <code>imp</code> function is written using functional programming. If you want to be charitable, you could say that the Controller is an <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a> between ASP.NET Web API and the functional F# API that actually implements the service. If you want to be cynical, you could also call it an <a href="http://wiki.c2.com/?AnticorruptionLayer">anti-corruption layer</a>. </p> <p> This works well, but tends to become repetitive: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;System <span style="color:blue;">open</span>&nbsp;System.Web.Http <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">BoundaryFailure</span>&nbsp;= |&nbsp;<span style="color:navy;">RouteFailure</span> |&nbsp;<span style="color:navy;">ValidationFailure</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span> |&nbsp;<span style="color:navy;">IntegrationFailure</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span> <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">HomeController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;[&lt;<span style="color:teal;">AllowAnonymous</span>&gt;] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Get</span>&nbsp;()&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;()&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">HomeDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_ <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_ <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardRecurrentStartController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_ <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardRecurrentController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;transactionKey&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;:&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;transactionKey&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_ <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PushRequestDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> At this point in our code base, we had five Controllers, and they all had similar implementations. They all pass their input parameters to their injected <code>imp</code> functions, and they all pattern match in exactly the same way. There are, however, small variations. Notice that one of the Controllers expose an HTTP GET operation, whereas the other four expose a POST operation. Conceivably, there could also be Controllers that allow both GET and POST, and so on, but this isn't the case here. </p> <p> The parameter list for each of the action methods also vary. Some take two arguments, one takes three, and the GET method takes none. </p> <p> Another variation is that <code>HomeController.Get</code> is annotated with an <code>[&lt;AllowAnonymous&gt;]</code> attribute, while the other action methods aren't. </p> <p> Despite these variations, it's possible to make the code less repetitive. </p> <p> You'd think you could easily refactor the code by turning the identical pattern matches into a reusable function. Unfortunately, it's not so simple, because the methods used to return HTTP responses are all <code>protected</code> (to use a C# term for something that F# doesn't even have). You can call <code>this.Ok ()</code> or <code>this.NotFound ()</code> from a derived class, but not from a 'free' <code>let</code>-bound function. </p> <p> After much trial and error, I finally arrived at this reusable base class: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:teal;">Imp</span>&lt;&#39;inp,&nbsp;&#39;out&gt;&nbsp;=&nbsp;&#39;inp&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:teal;">Result</span>&lt;&#39;out,&nbsp;<span style="color:teal;">BoundaryFailure</span>&gt; [&lt;<span style="color:teal;">AbstractClass</span>&gt;] <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Imp&lt;&#39;a,&#39;b&gt;&nbsp;-&gt;&nbsp;&#39;a&nbsp;-&gt;&nbsp;IHttpActionResult</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;resp&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">NotFound</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> It's been more than a decade, I think, since I last used inheritance to enable reuse, but in this case I could find no other way because of the design of ApiController. It gets the job done, though: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">HomeController</span>&nbsp;(<span style="color:navy;">imp</span>&nbsp;:&nbsp;<span style="color:teal;">Imp</span>&lt;_,&nbsp;<span style="color:teal;">HomeDtr</span>&gt;)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;[&lt;<span style="color:teal;">AllowAnonymous</span>&gt;] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Get</span>&nbsp;()&nbsp;=&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;() <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardController</span>&nbsp;(<span style="color:navy;">imp</span>&nbsp;:&nbsp;<span style="color:teal;">Imp</span>&lt;_,&nbsp;<span style="color:teal;">PaymentDtr</span>&gt;)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(portalId,&nbsp;req) <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardRecurrentStartController</span>&nbsp;(<span style="color:navy;">imp</span>&nbsp;:&nbsp;<span style="color:teal;">Imp</span>&lt;_,&nbsp;<span style="color:teal;">PaymentDtr</span>&gt;)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(portalId,&nbsp;req) <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">CreditCardRecurrentController</span>&nbsp;(<span style="color:navy;">imp</span>&nbsp;:&nbsp;<span style="color:teal;">Imp</span>&lt;_,&nbsp;<span style="color:teal;">PaymentDtr</span>&gt;)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;transactionKey&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PaymentDtr</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(portalId,&nbsp;transactionKey,&nbsp;req) <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>&nbsp;:&nbsp;<span style="color:teal;">Imp</span>&lt;_,&nbsp;<span style="color:teal;">unit</span>&gt;)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">FunctionController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PushRequestDtr</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">Execute</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(portalId,&nbsp;req)</pre> </p> <p> Notice that all five Controllers now derive from <code>FunctionController</code> instead of <code>ApiController</code>. The <code>HomeController.Get</code> method is still annotated with the <code>[&lt;AllowAnonymous&gt;]</code> attribute, and it's still a GET operation, whereas all the other methods still implement POST operations. You'll also notice that the various <code>Post</code> methods have retained their varying number of parameters. </p> <p> In order to make this work, I had to make one trivial change: previously, all the <code>imp</code> functions were curried, but that doesn't fit into a single reusable base implementation. If you consider the type of <code>FunctionController.Execute</code>, you can see that the <code>imp</code> function is expected to take a single input value of the type <code>'a</code> (and return a value of the type <code>Result&lt;'b, BoundaryFailure&gt;</code>). Since any <code>imp</code> function can only take a single input value, I had to uncurry them all. You can see that now all the <code>Post</code> methods pass their input parameters as a single tuple to their injected <code>imp</code> function. </p> <p> You may be wondering about the <code>Imp&lt;&#39;inp,&nbsp;&#39;out&gt;</code> type alias. It's not strictly necessary, but helps keep the code clear. As I've attempted to indicate with the code comment above the <code>Execute</code> method, it's generic. When used in a derived Controller, the compiler can infer the type of <code>'a</code> because it already knows the types of the input parameters. For example, in <code>CreditCardController.Post</code>, the input parameters are already annotated as <code>string</code> and <code>PaymentDtr</code>, so the compiler can easily infer the type of <code>(portalId, req)</code>. </p> <p> On the other hand, the compiler can't infer the type of <code>'b</code>, because that value doesn't relate to <code>IHttpActionResult</code>. To help the compiler, I introduced the <code>Imp</code> type, because it enabled me to concisely annotate the type of the output value, while using a wild-card type for the input type. </p> <p> I wouldn't mind getting rid of Controllers altogether, but this is as close as I've been able to get with ASP.NET Web API. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://github.com/Alxandr">Aleksander Heintz</a></div> <div class="comment-content"> <p> I find a few issues with your blog post. Technically, I don't think anything you write in it is wrong, but there are a few of the statements I find problematic in the sense that people will probably read the blog post and conclude that "this is the only solution because of X" when in fact X is trivial to circumvent. </p> <p> The main issue I have with your post is the following: </p> <p> You'd think you could easily refactor the code by turning the identical pattern matches into a reusable function. Unfortunately, it's not so simple, because the methods used to return HTTP responses are all <code>protected</code> (to use a C# term for something that F# doesn't even have). You can call <code>this.Ok ()</code> or <code>this.NotFound ()</code> from a derived class, but not from a 'free' let-bound function. </p> <p> As stated earlier, there is nothing in your statement that is technically wrong, but I'd argue that you've reached the wrong conclusion due to lack of data. Or in this case familiarity with Web API and it's source code. If you go look at the source for <a href="https://aspnetwebstack.codeplex.com/SourceControl/latest#src/System.Web.Http/ApiController.cs"><code>ApiController</code></a>, you will notice that while true, the methods are <code>protected</code>, they are also trivial one-liners (helper methods) calling public APIs. It is completely possible to create a helper ala: </p> <p> <pre>// NOTE: Untested let handle (ctrl: ApiController) = function | Success of result -&gt; OkNegotiatedContentResult (result, ctrl) | Failure RouteFailure -&gt; NotFoundResult (ctrl) | Failure (ValidationFailure msg) -&gt; BadRequestErrorMessageResult (msg, ctrl) | Failure (IntegrationFailure msg) -&gt; ExceptionResult ((InvalidOperationException msg), ctrl)</pre> </p> <p> Another thing you can do is implement IHttpActionResult on your discriminate union, so you can just return it directly from you controller, in which case you'd end up with code like this: </p> <p> <pre>member this.Post (portalId : string, req : PaymentDtr) = imp portalId req</pre> </p> <p> It's definitely a bit more work, but in no way is it undoable. Thirdly, Web API (and especially the new MVC Core which has taken over for it) is incredibly pluggable through it's DI. You don't have to use the base <code>ApiController</code> class if you don't want to. You can override the resolution logic, the handing of return types, routing, you name it. It should for instance be entirely doable to write a handler for "controllers" that look like this: </p> <p> <pre>[&lt;Controller&gt;] module MyController = [&lt;AllowAnonymous&gt;] // assume impl is imported and in scope here - this controller has no DI let post (portalId : string) (req : PaymentDtr) = impl portalId req</pre> </p> <p> This however would probably be a bit of work requiring quite a bit of reflection voodoo. But somebody could definitely do it and put it in a library, and it would be nuget-installable for all. </p> <p> Anyways, I hope this shows you that there are more options to solve the problem. And who knows, you may have considered all of them and concluded them unviable. I've personally dug through the source code of both MVC and Web API a few times which is why I know a bunch of this stuff, and I just figured you might want to know some of it too :). </p> </div> <div class="comment-date">2017-03-30 22:06 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Aleksander, thank you for writing. I clearly have some scars from working with ASP.NET Web API since version 0.6 (or some value in that neighbourhood). In general, if a method is defined on <code>ApiController</code>, I've learned the hard way that such methods tend to be tightly coupled to the Controller. Often, such methods even look into the untyped context dictionary in the request message object, so I've gotten used to use them whenever they're present. </p> <p> These scars have prevented me from pushing the envelope on the 'new' methods that return various <code>IHttpActionResult</code> objects, but as you write, they truly are thin wrappers over constructors. </p> <p> This does, indeed, enable us to write the mapping of result values as a let-bound function. Thank you for pointing that out! </p> <p> Mine ended up looking like the following. We've added a few more success and error cases since I originally wrote this article, so it looks more complex than the initial example. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">toHttpResult</span>&nbsp;(controller&nbsp;:&nbsp;<span style="color:teal;">ApiController</span>)&nbsp;result&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;result&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(<span style="color:navy;">OK</span>&nbsp;resp)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:teal;">OkNegotiatedContentResult</span>&nbsp;(resp,&nbsp;controller)&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(<span style="color:navy;">Created</span>&nbsp;(resp,&nbsp;location))&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">CreatedNegotiatedContentResult</span>&nbsp;(location,&nbsp;resp,&nbsp;controller)&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">RouteFailure</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:teal;">NotFoundResult</span>&nbsp;controller&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">BadRequestErrorMessageResult</span>&nbsp;(msg,&nbsp;controller)&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;resp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controller.Request.<span style="color:navy;">CreateErrorResponse</span>&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">HttpStatusCode</span>.InternalServerError, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;msg) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">ResponseMessageResult</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">StabilityFailure</span>&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;resp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">HttpResponseMessage</span>&nbsp;(<span style="color:teal;">HttpStatusCode</span>.ServiceUnavailable) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;resp.Headers.RetryAfter&nbsp;&lt;- &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">RetryConditionHeaderValue</span>&nbsp;(<span style="color:teal;">TimeSpan</span>.<span style="color:navy;">FromMinutes</span>&nbsp;5.) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">ResponseMessageResult</span>&nbsp;resp&nbsp;:&gt;&nbsp;_</pre> </p> <p> Letting my <code>BoundaryFailure</code> discriminated union implement <code>IHttpActionResult</code> sounds promising, but how would that be possible? </p> <p> The interface has a single method with the type <code>CancellationToken -&gt; Task&lt;HttpResponseMessage&gt;</code>. In order to create e.g. <code>NotFoundResult</code>, you need either an <code>HttpRequestMessage</code> or an <code>ApiController</code>, and none of those are available when <code>IHttpActionResult.ExecuteAsync</code> is invoked. </p> <p> When it comes to not having to derive from <code>ApiController</code>, I'm aware that even on the full .NET framework (we're not on .NET Core), 'all' the framework needs are <code>IHttpController</code> instances. As you imply, plugging into the framework is likely to be non-trivial, and so far, I haven't found that such an effort would be warranted. I might use it if someone else wrote it, though :) </p> </div> <div class="comment-date">2017-03-31 13:13 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Dependency rejection http://blog.ploeh.dk/2017/02/02/dependency-rejection 2017-02-02T08:56:00+00:00 Mark Seemann <div id="post"> <p> <em>In functional programming, the notion of dependencies must be rejected. Instead, applications should be composed from pure and impure functions.</em> </p> <p> This is the third article in a small article series called <a href="http://blog.ploeh.dk/2017/01/27/from-dependency-injection-to-dependency-rejection">from dependency injection to dependency rejection</a>. In the <a href="http://blog.ploeh.dk/2017/01/30/partial-application-is-dependency-injection">previous article in the series</a>, you learned that dependency injection can't be functional, because it makes everything impure. In this article, you'll see what to do instead. </p> <p> <strong>Indirect input and output</strong> </p> <p> One of the first concepts you learned when you learned to program was that units of operation (functions, methods, procedures) take input and produce output. Input is in the form or input parameters, and output is in the form of return values. (Sometimes, though, a method returns nothing, but we know from category theory that nothing is also a value (called <em>unit</em>).) </p> <p> In addition to such input and output, a unit with dependencies also take indirect input, and produce indirect output: </p> <p> <img src="/content/binary/unit-with-dependencies-and-direct-and-indirect-input-and-output.png" alt="A unit with dependencies and direct and indirect input and output."> </p> <p> When a unit queries a dependency for data, the data returned from the dependency is indirect input. In the restaurant reservation example used in this article series, when <code>tryAccept</code> calls <code>readReservations</code>, the returned reservations are indirect input. </p> <p> Likewise, when a unit invokes a dependency, all arguments passed to that dependency constitute indirect output. In the example, when <code>tryAccept</code> calls <code>createReservation</code>, the reservation value it uses as input argument to that function call becomes output. The intent, in this case, is to save the reservation in a database. </p> <p> <strong>From indirect output to direct output</strong> </p> <p> Instead of producing indirect output, you can refactor functions to produce direct output. </p> <p> <img src="/content/binary/unit-with-dependencies-and-direct-input-and-output-but-no-indirect-output.png" alt="A unit with dependencies and direct input and output, but no indirect output."> </p> <p> Such a refactoring is often problematic in mainstream object-oriented languages like C# and Java, because you wish to control the circumstances in which the indirect output must be produced. Indirect output often implies side-effects, but perhaps the side-effect must only happen when certain conditions are fulfilled. In the restaurant reservation example, the desired side-effect is to add a reservation to a database, but this must only happen when the restaurant has sufficient remaining capacity to serve the requested number of people. Since languages like C# and Java are statement-based, it can be difficult to separate the decision from the action. </p> <p> In expression-based languages like F# and Haskell, <a href="http://blog.ploeh.dk/2016/09/26/decoupling-decisions-from-effects">it's trivial to decouple decisions from effects</a>. </p> <p> In the <a href="http://blog.ploeh.dk/2017/01/30/partial-application-is-dependency-injection">previous article</a>, you saw a version of <code>tryAccept</code> with this signature: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;(DateTimeOffset&nbsp;-&gt;&nbsp;Reservation&nbsp;list)&nbsp;-&gt;&nbsp;(Reservation&nbsp;-&gt;&nbsp;int)&nbsp;-&gt;&nbsp;Reservation</span> <span style="color:green;">//&nbsp;-&gt;&nbsp;int&nbsp;option</span></pre> </p> <p> The second function argument, with the type <code>Reservation&nbsp;-&gt;&nbsp;int</code>, produces indirect output. The <code>Reservation</code> value is the output. The function even violates Command Query Separation and returns the database ID of the added reservation, so that's additional indirect input. The overall function returns <code>int option</code>: the database ID if the reservation was added, and <code>None</code> if it wasn't. </p> <p> Refactoring the indirect output to direct output is easy, then: just remove the <code>createReservation</code> function and return the <code>Reservation</code> value instead: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;(DateTimeOffset&nbsp;-&gt;&nbsp;Reservation&nbsp;list)&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;Reservation&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;<span style="color:navy;">readReservations</span>&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">readReservations</span>&nbsp;reservation.Date&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;reservedSeats&nbsp;+&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;{&nbsp;reservation&nbsp;<span style="color:blue;">with</span>&nbsp;IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> Notice that this refactored version of <code>tryAccept</code> returns a <code>Reservation option</code> value. The implication is that the reservation was accepted if the return value is a <code>Some</code> case, and rejected if the value is <code>None</code>. The decision is embedded in the value, but decoupled from the side-effect of writing to the database. </p> <p> This function clearly never writes to the database, so at the boundary of your application, you'll have to connect the decision to the effect. To keep the example consistent with the previous article, you can do this in a <code>tryAcceptComposition</code> function, like this: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;int&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAcceptComposition</span>&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;10&nbsp;(<span style="color:teal;">DB</span>.<span style="color:navy;">readReservations</span>&nbsp;connectionString) &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Option</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:teal;">DB</span>.<span style="color:navy;">createReservation</span>&nbsp;connectionString)</pre> </p> <p> Notice that the type of <code>tryAcceptComposition</code> remains <code>Reservation&nbsp;-&gt;&nbsp;int&nbsp;option</code>. This is a true refactoring. The overall API remains the same, as does the behaviour. The reservation is added to the database only if there's sufficient remaining capacity, and in that case, the ID of the reservation is returned. </p> <p> <strong>From indirect input to direct input</strong> </p> <p> Just as you can refactor from indirect output to direct output can you refactor from indirect input to direct input. </p> <p> <img src="/content/binary/unit-with-dependencies-and-direct-input-and-output.png" alt="A unit with dependencies and direct input and output."> </p> <p> Again, in statement-based languages like C# and Java, this may be problematic, because you may wish to defer a query, or base it on a decision inside the unit. In expression-based languages you can decouple decisions from effects, and deferred execution can always be done by lazy evaluation, if that's required. In the case of the current example, however, the refactoring is easy: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;Reservation&nbsp;list&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;Reservation&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;reservations&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;reservations&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;reservedSeats&nbsp;+&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;{&nbsp;reservation&nbsp;<span style="color:blue;">with</span>&nbsp;IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> Instead of calling a (potentially impure) function, this version of <code>tryAccept</code> takes a list of existing reservations as input. It still sums over all the quantities, and the rest of the code is the same as before. </p> <p> Obviously, the list of existing reservations must come from somewhere, like a database, so <code>tryAcceptComposition</code> will still have to take care of that: </p> <p> <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b&nbsp;-&gt;&nbsp;&#39;c)&nbsp;-&gt;&nbsp;&#39;b&nbsp;-&gt;&nbsp;&#39;a&nbsp;-&gt;&nbsp;&#39;c</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">flip</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;y&nbsp;=&nbsp;<span style="color:navy;">f</span>&nbsp;y&nbsp;x <span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;int&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAcceptComposition</span>&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;reservation.Date &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">DB</span>.<span style="color:navy;">readReservations</span>&nbsp;connectionString &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">flip</span>&nbsp;(<span style="color:navy;">tryAccept</span>&nbsp;10)&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Option</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:teal;">DB</span>.<span style="color:navy;">createReservation</span>&nbsp;connectionString)</pre> </p> <p> The type and behaviour of this composition is still the same as before, but the data flow is different. First, the function queries the database, which is an impure operation. Then, it pipes the resulting list of reservations to <code>tryAccept</code>, which is now a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a>. It returns a <code>Reservation option</code> that's finally mapped to another impure operation, which writes the reservation to the database if the reservation was accepted. </p> <p> You'll notice that I also added a <code>flip</code> function in order to make the composition more concise, but I could also have used a lambda expression when invoking <code>tryAccept</code>. The <code>flip</code> function is a part of Haskell's standard library, but isn't in F#'s core library. It's not crucial to the example, though. </p> <p> <strong>Evaluation</strong> </p> <p> Did you notice that in the previous diagram, above, all arrows between the unit and its dependencies were gone? This means that the unit no longer has any dependencies: </p> <p> <img src="/content/binary/unit-with-direct-input-and-output.png" alt="A unit with direct input and output, but no dependencies."> </p> <p> Dependencies are, by their nature, impure, and since pure functions can't call impure functions, functional programming must reject the notion of dependencies. Pure functions can't <em>depend</em> on impure functions. </p> <p> Instead, pure functions must take direct input and produce direct output, and the impure boundary of an application must compose impure and pure functions together in order to achieve the desired behaviour. </p> <p> In the previous article, you saw how Haskell can be used to evaluate whether or not an implementation is functional. You can port the above F# code to Haskell to verify that this is the case. </p> <p> <pre><span style="color:#600277;">tryAccept</span>&nbsp;::&nbsp;Int&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[<span style="color:blue;">Reservation</span>]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Reservation</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Maybe&nbsp;<span style="color:blue;">Reservation</span> tryAccept&nbsp;capacity&nbsp;reservations&nbsp;reservation&nbsp;<span style="color:#666666;">=</span> &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;<span style="color:#666666;">=</span>&nbsp;sum&nbsp;<span style="color:#666666;">$</span>&nbsp;map&nbsp;quantity&nbsp;reservations &nbsp;&nbsp;<span style="color:blue;">in</span>&nbsp;&nbsp;<span style="color:#af00db;">if</span>&nbsp;reservedSeats&nbsp;<span style="color:#666666;">+</span>&nbsp;quantity&nbsp;reservation&nbsp;<span style="color:#666666;">&lt;=</span>&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#af00db;">then</span>&nbsp;Just&nbsp;<span style="color:#666666;">$</span>&nbsp;reservation&nbsp;{&nbsp;isAccepted&nbsp;<span style="color:#666666;">=</span>&nbsp;True&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#af00db;">else</span>&nbsp;Nothing</pre> </p> <p> This version of <code>tryAccept</code> is pure, and compiles, but as you learned in the previous article, that's not the crucial question. The question is whether the composition compiles? </p> <p> <pre><span style="color:#600277;">tryAcceptComposition</span>&nbsp;::&nbsp;<span style="color:blue;">Reservation</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;IO&nbsp;(Maybe&nbsp;Int) tryAcceptComposition&nbsp;reservation&nbsp;<span style="color:#666666;">=</span>&nbsp;runMaybeT&nbsp;<span style="color:#666666;">$</span> &nbsp;&nbsp;liftIO&nbsp;(<span style="color:#dd0000;">DB</span><span style="color:#666666;">.</span>readReservations&nbsp;connectionString&nbsp;<span style="color:#666666;">$</span>&nbsp;date&nbsp;reservation) &nbsp;&nbsp;<span style="color:#666666;">&gt;&gt;=</span>&nbsp;<span style="color:#dd0000;">MaybeT</span>&nbsp;<span style="color:#666666;">.</span>&nbsp;return&nbsp;<span style="color:#666666;">.</span>&nbsp;flip&nbsp;(tryAccept&nbsp;<span style="color:#09885a;">10</span>)&nbsp;reservation &nbsp;&nbsp;<span style="color:#666666;">&gt;&gt;=</span>&nbsp;liftIO&nbsp;<span style="color:#666666;">.</span>&nbsp;<span style="color:#dd0000;">DB</span><span style="color:#666666;">.</span>createReservation&nbsp;connectionString</pre> </p> <p> This version of <code>tryAcceptComposition</code> compiles, and works as desired. The code exhibits a common pattern for Haskell: First, gather data from impure sources. Second, pass pure data to pure functions. Third, take the pure output from the pure functions, and do something impure with it. </p> <p> It's like a sandwich, with the best parts in the middle, and some necessary stuff surrounding it. </p> <p> <strong>Summary</strong> </p> <p> Dependencies are, by nature, impure. They're either non-deterministic, have side-effects, or both. Pure functions can't call impure functions (because that would make them impure as well), so pure functions can't have dependencies. Functional programming must reject the notion of dependencies. </p> <p> Obviously, software is only useful with impure behaviour, so instead of injecting dependencies, functional programs must be composed in impure contexts. Impure functions can call pure functions, so at the boundary, an application must gather impure data, and use it to call pure functions. This <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">automatically leads to the ports and adapters architecture</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://dusted.codes/">Dustin Moris Gorski</a></div> <div class="comment-content"> <p>Hi, Thank you for this blog post series. I also read your other posts on ports and adapters and the proposed architecture makes sense in terms of how it works, but I struggle to see the benefit in a real world application. Maybe let me explain my question with a quick example.</p> <p>In the 2nd blog post of this series you demonstrated this function:</p> <pre><code>// int -&gt; (DateTimeOffset -&gt; Reservation list) -&gt; (Reservation -&gt; int) -&gt; Reservation // -&gt; int option let tryAccept capacity readReservations createReservation reservation = let reservedSeats = readReservations reservation.Date |&gt; List.sumBy (fun x -&gt; x.Quantity) if reservedSeats + reservation.Quantity &lt;= capacity then createReservation { reservation with IsAccepted = true } |&gt; Some else None</code></pre> <p>If I understand it correctly this function is pure if <code>readReservations</code> and <code>createReservation</code> are both pure otherwise it is impure.</p> <p>I also understand the benefit of having a pure function, because it is a lot easier to understand the code, test the code and reason about it. That makes sense as well :).</p> <p>So in the 3rd blog post you make <code>tryAccept</code> a pure function, by removing the function dependencies and replacing it with simple values:</p> <pre><code>// int -&gt; Reservation list -&gt; Reservation -&gt; Reservation option let tryAccept capacity reservations reservation = let reservedSeats = reservations |&gt; List.sumBy (fun x -&gt; x.Quantity) if reservedSeats + reservation.Quantity &lt;= capacity then { reservation with IsAccepted = true } |&gt; Some else None</code></pre> <p>However this was only possible because you essentially moved the impure code into another new function:</p> <pre><code>// Reservation -&gt; int option let tryAcceptComposition reservation = reservation.Date |&gt; DB.readReservations connectionString |&gt; flip (tryAccept 10) reservation |&gt; Option.map (DB.createReservation connectionString)</code></pre> <p>So after all the application hasn't really reduced the total number of impure functions (still 3 in each case - <code>readReservations</code>, <code>createReservation</code> and <code>tryAccept[Composition]</code>).</p> <p>The only difference I see is that one impure function has been refactored into 2 functions - one pure and one impure. Considering that the original <code>tryAccept</code> function was already fully testable from a unit testing point of view and quite readable what is the benefit of this additional step? I would almost argue that the original <code>tryAccept</code> function was even easier to read/understand than the combination of <code>tryAccept</code> and <code>tryAcceptComposition</code>. I understand that impure functions like this are not truly functional, but in a real world application you must have some impure functions and I would like to better understand where trade-off benefit of that additional step is? Am I missing something else?</p> <p></p> </div> <div class="comment-date">2017-02-03 10:34 UTC</div> </div> <div class="comment"> <div class="comment-author"> <a href="http://blog.ploeh.dk">Mark Seemann</a> </div> <div class="comment-content"> </div> <p> Dustin, thank you for writing. There are several answers to your question, depending on the perspective one is interested in. I'll see if I can cover the most important ones. </p> <p> <strong>Is it functional?</strong> </p> <p> On the most fundamental level, I'm interested in learning functional programming. In order to do this, I seek out strictly functional solutions to problems. Haskell is a great help in that endeavour, because it's not a hybrid language. It only allows you to do functional programming. </p> <p> Does it make sense to back-port Haskell solutions to F#, then? That depends on what one is trying to accomplish, but if the goal is nothing but learning how to do it functionally, then that goal is accomplished. </p> <p> <strong>Toy examples</strong> </p> <p> On another level, the example I've presented here is obviously nothing but a toy example. It's simplified, because if I presented readers with a more realistic example, the complexity of the real problem could easily drown out the message of the example. Additionally, most readers would probably give up reading. </p> <p> I'm asking my readers to pretend that the problem is more complex than the one I present here; pretend that this problem is a stand-in for a harder problem. </p> <p> In this particular context, there could be all sorts of complications: <ul> <li> Reservations could be for time slots instead of whole dates. In order to keep the example simple, I treat each reservation as simply blocking out an entire date. I once dined at a restaurant where they started serving at 19:00, and if you weren't there on time, you'd miss the first courses. Most restaurants, though, allow you to make reservations for a particular time, and many have more than one serving on a single evening. </li> <li> Most restaurants have tables, not seats. Again, the same restaurant I mentioned above seated 12 people at a bar-like arrangement facing the kitchen, but most restaurants have tables of varying sizes. If they get a reservation for three people, they may have to reserve a table for four. </li> <li> Perhaps the restaurant would like to implement a feature where, if it receives a reservation that doesn't fill out a table (like a reservation for three people, and only four-people tables are left), it'd defer the decision to see if a 'better' reservation arrives later. </li> <li> Some people make reservations, but never show up. For that reason, a restaurant may want to allow a degree of overbooking, just like airlines. How much overbooking to allow is a business decision. </li> <li> A further wrinkle on the overbooking business rule is that you may have a different overbooking policy for Fridays than for, say, Wednesdays. </li> <li> Perhaps the restaurant would like to implement a waiting-list feature as well. </li> </ul> As you can see, we could easily imagine that the business logic could be more convoluted. Keeping all of that decision logic pure would be beneficial. </p> <p> <strong>Separation of concerns</strong> </p> <p> In my experience, there's an entire category of software defects that occur because of state mutation in business logic. You could have an area of your code that calls other code, which calls other code, and so on, for several levels of nesting. Somewhere, deep in the bowels of such a system, a conditional statement flips a boolean flag that consequently impact how the rest of the program runs. I've seen plenty of examples of such software, and it's inhumane; it doesn't fit within human cognitive limits. </p> <p> Code that allows arbitrary side-effects is difficult to reason about. </p> <p> Knowing that an subgraph of your call tree is pure reduces defects like that. This is nothing but another way to restate the <a href="https://en.wikipedia.org/wiki/Command%E2%80%93query_separation">command-query separation</a> principle. In F#, we still can't be sure unless we exert some discipline, but in Haskell, all it takes is a look at the type of a function or value. If it doesn't include <code>IO</code>, you know that it's pure. </p> <p> Separating pure code from impure code is separation of concern. Business logic is one concern, and I/O is another concern, and the better you can separate these, the fewer sources of defects you'll have. True, I haven't reduced the amount of code by much, but I've <em>separated</em> concerns by separating the code that contains (side) effects from the pure code. </p> <p> <strong>Testability</strong> </p> <p> It's true that the partial application version of <code>tryAccept</code> is <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">testable, because it has isolation</a>, but the tests are more complicated than they have to be: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``tryAccept&nbsp;behaves&nbsp;correctly&nbsp;when&nbsp;it&nbsp;can&nbsp;accept``</span> &nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:navy;">NonNegativeInt</span>&nbsp;excessCapacity) &nbsp;&nbsp;&nbsp;&nbsp;(expected&nbsp;:&nbsp;<span style="color:teal;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;&lt;!&gt;&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;&lt;*&gt;&nbsp;<span style="color:teal;">Gen</span>.<span style="color:navy;">listOf</span>&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;&nbsp;<span style="color:teal;">Arb</span>.<span style="color:navy;">fromGen</span>&nbsp;|&gt;&nbsp;<span style="color:teal;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;(reservation,&nbsp;reservations)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;excessCapacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+&nbsp;(reservations&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+&nbsp;reservation.Quantity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">readReservations</span>&nbsp;=&nbsp;((=!)&nbsp;reservation.Date)&nbsp;&gt;&gt;!&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;((=!)&nbsp;{&nbsp;reservation&nbsp;<span style="color:blue;">with</span>&nbsp;IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;})&nbsp;&gt;&gt;!&nbsp;expected &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;<span style="color:navy;">readReservations</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Some</span>&nbsp;expected&nbsp;=!&nbsp;actual [&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``tryAccept&nbsp;behaves&nbsp;correctly&nbsp;when&nbsp;it&nbsp;can&#39;t&nbsp;accept``</span> &nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:navy;">PositiveInt</span>&nbsp;lackingCapacity)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;&lt;!&gt;&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;&lt;*&gt;&nbsp;<span style="color:teal;">Gen</span>.<span style="color:navy;">listOf</span>&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;&nbsp;<span style="color:teal;">Arb</span>.<span style="color:navy;">fromGen</span>&nbsp;|&gt;&nbsp;<span style="color:teal;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;(reservation,&nbsp;reservations)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(reservations&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity))&nbsp;-&nbsp;lackingCapacity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">readReservations</span>&nbsp;_&nbsp;=&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;_&nbsp;=&nbsp;<span style="color:navy;">failwith</span>&nbsp;<span style="color:#a31515;">&quot;Mock&nbsp;shouldn&#39;t&nbsp;be&nbsp;called.&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;<span style="color:navy;">readReservations</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">None</span>&nbsp;=!&nbsp;actual</pre> </p> <p> (You can find these tests in <a href="https://github.com/ploeh/dependency-rejection-samples/commit/d2387cceb81eabc349a63ab7df1249236e9b1d13">commit d2387cceb81eabc349a63ab7df1249236e9b1d13 in the accompanying sample code repository</a>.) Contrast those dependency-injection style tests to these tests against the pure version of <code>tryAccept</code>: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``tryAccept&nbsp;behaves&nbsp;correctly&nbsp;when&nbsp;it&nbsp;can&nbsp;accept``</span> &nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:navy;">NonNegativeInt</span>&nbsp;excessCapacity)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;&lt;!&gt;&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;&lt;*&gt;&nbsp;<span style="color:teal;">Gen</span>.<span style="color:navy;">listOf</span>&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;&nbsp;<span style="color:teal;">Arb</span>.<span style="color:navy;">fromGen</span>&nbsp;|&gt;&nbsp;<span style="color:teal;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;(reservation,&nbsp;reservations)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;excessCapacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+&nbsp;(reservations&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+&nbsp;reservation.Quantity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;reservations&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Some</span>&nbsp;{&nbsp;reservation&nbsp;<span style="color:blue;">with</span>&nbsp;IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}&nbsp;=!&nbsp;actual [&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``tryAccept&nbsp;behaves&nbsp;correctly&nbsp;when&nbsp;it&nbsp;can&#39;t&nbsp;accept``</span> &nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:navy;">PositiveInt</span>&nbsp;lackingCapacity)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;&lt;!&gt;&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;&lt;*&gt;&nbsp;<span style="color:teal;">Gen</span>.<span style="color:navy;">listOf</span>&nbsp;<span style="color:teal;">Gen</span>.reservation &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;&nbsp;<span style="color:teal;">Arb</span>.<span style="color:navy;">fromGen</span>&nbsp;|&gt;&nbsp;<span style="color:teal;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;(reservation,&nbsp;reservations)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(reservations&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity))&nbsp;-&nbsp;lackingCapacity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;reservations&nbsp;reservation &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">None</span>&nbsp;=!&nbsp;actual</pre> </p> <p> They're simpler, and since they don't use <a href="http://xunitpatterns.com/Mock%20Object.html">mocks</a>, they're more robust. They were easier to write, and I subscribe to the spirit of <a href="http://amzn.to/SM8Yv0">GOOS</a>: <em>if test are difficult to write, the system under test should be simplified</em>. </p> <div class="comment-date">2017-02-05 20:09 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://stackoverflow.com/users/1523776/benjamin-hodgson?tab=profile">Benjamin Hodgson</a></div> <div class="comment-content"> <p>Hi Mark,</p> <p>Thanks for your talk at NDC last month, and for writing this series! I feel that the functional community (myself included) has a habit of using examples that aren't obviously relevant to the sort of line-of-business programming most of us do in our day jobs, so articles like this are sorely needed.</p> <p>We talked a little about this in person after your talk at the conference: I wanted to highlight a potential criticism of this style of programming. Namely, there's still some important business logic being carried out by your <code>tryAcceptComposition</code> function, like checking the capacity <i>on the requested reservation date</i>. How do you unit test that <code>readReservations</code> is called with the correct date? Likewise, how do you unit test that rejected reservations don't get saved? Real world business logic isn't always purely functional in nature. Sometimes the side effects that your code performs are part of the requirements.</p> <p>The Haskell philosophy isn't about rejecting side effects outright - it's about measuring and controlling them. I wouldn't write <code>tryAcceptComposition</code> using <code>IO</code>. Instead I'd program to the interface, not the implementation, using an mtl-style class to abstract over monads which support saving and loading reservations.</p> <pre><code>class Monad m =&gt; MonadReservation m where readReservations :: ConnectionString -&gt; Date -&gt; m [Reservation] createReservation :: ConnectionString -&gt; Reservation -&gt; m ReservationId tryAcceptComposition :: MonadReservation m =&gt; Reservation -&gt; m (Maybe ReservationId) tryAcceptComposition r = runMaybeT $ do reservations &lt;- lift $ readReservations connectionString (date r) accepted &lt;- MaybeT $ return $ tryAccept 10 reservations r lift $ createReservation connectionString accepted</code></pre> <p>Code that lives in a <code>MonadReservation</code> context can read and create reservations in the database but nothing else; it doesn't have all the power of <code>IO</code>. During unit testing I can use an instance of <code>MonadReservation</code> that returns canned values, and in production I can use a monad that actually talks to the database.</p> <p>Since type classes are syntactic sugar for passing an argument, this is really just a nicer way of writing your original DI-style code. I don't advocate the "free monad" style that's presently trendy in Scala-land because I find it unnecessarily complex. 90% of the purported advantages of free monads are already supported by simpler language features.</p> <p>I suppose the main downside of this design is that you can't express it in F#, at least not cleanly. It relies on type classes and higher-kinded types.</p> <p>Hope you find this interesting, I'd love to hear what you think!</p> <p>Benjamin</p> </div> <div class="comment-date">2017-02-06 16:28 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Benjamin, thank you for writing. The alternative you propose looks useful in Haskell, but, as you've already suggested, it doesn't translate well into F#. </p> <p> I write F# code professionally, whereas so far, I've only used Haskell to critique my F# code. (If someone who reads this comment would offer to pay me to write some Haskell code, please get in touch.) In other words, I still have much to learn about Haskell. I think I understand as much, however, that I'd be able to use your suggested design to unit test <code>tryAcceptComposition</code> using the <code>Identity</code> monad for Stubs, or perhaps <code>MonadWriter</code> or <code>MonadState</code> for Mocks. I'll have to try that one day... </p> <p> In F#, I write integration tests. Such tests are important regardless, and often they more closely relate to actual requirements, so I find this a worthwhile effort anyway. </p> </div> <div class="comment-date">2017-02-11 22:42 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://twitter.com/la_yumba">Enrico Buonanno</a></div> <div class="comment-content"> <p> Hi Mark, </p> <p> thanks for the post series, which I find interesting and needed. There is one part of your post that I find deserves further exploration. You write: <blockquote> in statement-based languages like C# and Java, this may be problematic, because you may wish to defer a query, or base it on a decision inside the unit. In expression-based languages you can decouple decisions from effects, and deferred execution can always be done by lazy evaluation, if that's required. </blockquote> Firstly, I would say that you can write expression-based programs in any language that has expressions, which naturally includes C# and Java. But that's not particularly relevant to this discussion. </p> <p> More to the point, you're glossing over this as though it were a minor detail, when in fact I don't think it is. Let's explore the case in which "you may wish to defer a query, or base it on a decision inside the unit". The way you do this "by lazy evaluation" would be - I assume - by passing a function as an argument to your unit. But this is then effectively dependency injection, because you're passing in a function which has side effects, which will be called (or not) from the unit. </p> <p> So, it seems to me that your technique of extracting side effects out of the unit provides a good general guideline, but not a completely general way to replace dependency injection. </p> </div> <div class="comment-date">2017-02-16 11:47 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Enrico, thank you for writing. There's a lot to unpack in that quote, which was one of the reasons I didn't expand it. It would have made the article too long, and wandered off compared to its main point. I don't mind going into these details here, though. </p> <p> <strong>Direction of data</strong> </p> <p> In order to get the obvious out of the way first, the issue you point out is with my refactoring of indirect input to direct input. Refactoring from indirect to output to direct output is, as far as I can tell, not on your agenda. Designing with direct input in mind seems uncontroversial to me, so that makes sense. </p> <p> <strong>No hard rules</strong> </p> <p> On this blog, I often write articles as I figure out how to deal with problems. Sometimes, I report on my discoveries at a time where I've yet to accumulate years of experience. What I've learned so far is that dependency injection isn't functional. What I'm <em>still</em> exploring is what to do instead. </p> <p> It's my experience that the type of refactoring I demonstrate here can surprisingly often be performed. I don't want to claim that it's always possible to do it like this. In fact, I'm still looking for good examples where this will not be possible. Whenever I think of a simple enough example that I could share it here, I always realise that if only I simplify the problem, I can put it into the shape seen here. </p> <p> My thinking is, however, constrained by my professional experience. I've been doing web (service) development for so many years now that it constraints my imagination. When you execution scope is exclusively a single HTTP request at a time, you tend to keep things simple. I'd welcome a simplified, but still concrete example where the impure/pure/impure sandwich described here isn't going to be possible. </p> <p> This may seem like a digression, but my point is that I don't claim to be the holder of a single, undeniable truth. Still, I find that this article describes a broadly applicable design and implementation technique. </p> <p> <strong>Language specifics</strong> </p> <p> The next topic we need to consider is our choice of language. When I wrote that deferred execution can always be done by lazy evaluation, that's exactly how Haskell works. Haskell is lazily evaluated, so any value passed as direct input can be unevaluated until required. That goes for <code>IO</code> as well, but then, as we've learned, you can't pass impure data to a pure function. </p> <p> All execution is, in that sense, deferred, unless explicitly forced. Thus, any potential need for deferred execution has no design implications. </p> <p> F#, on the other hand, is an eagerly evaluated language, so there, deferred execution may have design implications. </p> <p> <strong>Performance</strong> </p> <p> Perhaps it's my lack of imagination again, but I can't think of a well-designed system where deferred execution is required for purposes of correctness. As far as I can tell, deferred execution is a performance concern. You wish to defer execution of a query because that operation takes significant time. </p> <p> That's a real concern, but I often find that people worry too much about performance. Again, this is probably my lack of wider experience, as I realise that performance can be important in smart phone apps, games, and the like. Clearly, performance is also important in the world of REST APIs, but I've met a lot of people who worry about performance without ever measuring it. </p> <p> When you start measuring performance, you'll often be surprised to discover where your code spends actual time. So my design approach is always to prioritise making the system work first, and then, if there are performance problems, figure out how to tweak it so that it becomes satisfactory. In my experience, such tweaking is only necessary now and then. I'm not claiming that my code is the fastest it could be, but it's often fast enough, and as easy to maintain as I can make it. </p> <p> <strong>The need for data</strong> </p> <p> Another concern is the need for data. If you consider the above <code>tryAccept</code> function, it <em>always</em> uses <code>reservations</code>. Thus, there's no gain in deferring the database query, because you'll always need the data. </p> <p> Deferred execution is only required in those cases where you have conditional branching, and only in certain cases do you need to read a particular piece of data. </p> <p> Even conditional branching isn't enough of a criterion, though, because you could have branching where, in 99.9 % of the cases, you'd be performing the query anyway. Would you, then, need deferred execution for the remaining 0.1 % of the cases? </p> <p> <strong>Lazy sequences</strong> </p> <p> Still, let's assume that we've implemented a system using pure functions that take pure data, but to our dismay we discover that there's one query that takes time to execute, and that we truly only need it some of the time. In .NET, there are two distinct situations: <ul> <li>We need a scalar value</li> <li>We need a collection of values</li> </ul> If we need a collection of values, we only need to make a minuscule change to the design of our function. Instead of taking an F# list, or an array, as direct input, we can make the function take a sequence (<code>IEnumerable&lt;T&gt;</code> in C#) as input. These can be implemented as lazily evaluated sequences, which gives us the deferred execution we need. </p> <p> <strong>Lazy scalar values</strong> </p> <p> This leaves the corner case where we need a lazily evaluated scalar value. In such cases, I may have to make a concession to performance in my function design, but I wouldn't change the argument to a function, but rather to a <a href="https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/lazy-computations">lazy</a> value. </p> <p> Lazy values are deferred, but memoised, which is the reason I'd prefer them over function arguments. </p> </div> <div class="comment-date">2017-02-18 19:54 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Partial application is dependency injection http://blog.ploeh.dk/2017/01/30/partial-application-is-dependency-injection 2017-01-30T12:40:00+00:00 Mark Seemann <div id="post"> <p> <em>The equivalent of dependency injection in F# is partial function application, but it isn't functional.</em> </p> <p> This is the second article in a small article series called <a href="http://blog.ploeh.dk/2017/01/27/from-dependency-injection-to-dependency-rejection">from dependency injection to dependency rejection</a>. </p> <p> People often ask me how to do dependency injection in F#. That's only natural, since I wrote <a href="http://amzn.to/12p90MG">Dependency Injection in .NET</a> some years ago, and also since I've increasingly focused my energy on F# and other functional programming languages. </p> <p> Over the years, I've seen other F# experts respond to that question, and often, the answer is that partial function application is the F# way to do dependency injection. For some years, I believed that as well. It turns out to be true in one sense, but incorrect in another. Partial application <em>is</em> equivalent to dependency injection. It's just not a functional solution to dealing with dependencies. </p> <p> (To be as clear as I can be: I'm not claiming that partial application isn't functional. What I claim is that partial application used for dependency injection isn't functional.) </p> <p> <strong>Attempted dependency injection using functions</strong> </p> <p> Returning to the example from the <a href="http://blog.ploeh.dk/2017/01/27/dependency-injection-is-passing-an-argument">previous article</a>, you could try to rewrite <code>MaîtreD.TryAccept</code> as a function: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;(DateTimeOffset&nbsp;-&gt;&nbsp;Reservation&nbsp;list)&nbsp;-&gt;&nbsp;(Reservation&nbsp;-&gt;&nbsp;int)&nbsp;-&gt;&nbsp;Reservation</span> <span style="color:green;">//&nbsp;-&gt;&nbsp;int&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;capacity&nbsp;<span style="color:navy;">readReservations</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">readReservations</span>&nbsp;reservation.Date&nbsp;|&gt;&nbsp;<span style="color:teal;">List</span>.<span style="color:navy;">sumBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Quantity) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;reservedSeats&nbsp;+&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;{&nbsp;reservation&nbsp;<span style="color:blue;">with</span>&nbsp;IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> You could imagine that this <code>tryAccept</code> function is part of a module called <code>MaîtreD</code>, just to keep the examples as equivalent as possible. </p> <p> The function takes four arguments. The first is the capacity of the restaurant in question; a primitive integer. The next two arguments, <code>readReservations</code> and <code>createReservation</code> fill the role of the injected <code>IReservationsRepository</code> in the previous article. In the object-oriented example, the <code>TryAccept</code> method used two methods on the repository: <code>ReadReservations</code> and <code>Create</code>. Instead of using an interface, in the F# function, I make the function take two independent functions. They have (almost) the same types as their C# counterparts. </p> <p> The first three arguments correspond to the injected dependencies in the previous <code>MaîtreD</code> class. The fourth argument is a <code>Reservation</code> value, which corresponds to the input to the previous <code>TryAccept</code> method. </p> <p> Instead of returning a nullable integer, this F# version returns an <code>int option</code>. </p> <p> The implementation is also equivalent to the C# example: Read the relevant reservations from the database using the <code>readReservations</code> function argument, and sum over their quantities. Based on the number of already reserved seats, decide whether or not to accept the reservation. If you can accept the reservation, set <code>IsAccepted</code> to <code>true</code>, call the <code>createReservation</code> function argument, and pipe the returned ID (integer) to <code>Some</code>. If you can't accept the reservation, then return <code>None</code>. </p> <p> Notice that the first three arguments are 'dependencies', whereas the last argument is the 'actual input', if you will. This means that you can use partial function application to compose this function. </p> <p> <strong>Application</strong> </p> <p> If you recall the definition of the previous <code>IMaîtreD</code> interface, the <code>TryAccept</code> method was defined like this (C# code snippet): </p> <p> <pre><span style="color:blue;">int</span>?&nbsp;TryAccept(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation);</pre> </p> <p> You could attempt to define a similar function with the type <code>Reservation -&gt; int option</code>. Normally, you'd want to do this closer to the boundary of the application, but the following example demonstrates how to 'inject' real database operations into the function. </p> <p> Imagine that you have a <code>DB</code> module with these functions: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:teal;">DB</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;DateTimeOffset&nbsp;-&gt;&nbsp;Reservation&nbsp;list</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">readReservations</span>&nbsp;<span style="color:#9b9b9b;">connectionString</span>&nbsp;<span style="color:#9b9b9b;">date</span>&nbsp;=&nbsp;<span style="color:green;">//&nbsp;..</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;int</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">createReservation</span>&nbsp;<span style="color:#9b9b9b;">connectionString</span>&nbsp;<span style="color:#9b9b9b;">reservation</span>&nbsp;=&nbsp;<span style="color:green;">//&nbsp;..</span></pre> </p> <p> The <code>readReservations</code> function takes a connection string and a date as arguments, and returns a list of reservations for that date. The <code>createReservation</code> function also takes a connection string, as well as a reservation. When invoked, it creates a new record for the reservation and returns the ID of the newly created row. (This sort of API violates CQS, so you should <a href="http://blog.ploeh.dk/2016/05/06/cqs-and-server-generated-entity-ids">consider alternatives</a>.) </p> <p> If you partially apply these functions with a valid connection string, both have the type desired for their roles in <code>tryAccept</code>. This means that you can create a function from these elements: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;int&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryAcceptComposition</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">read</span>&nbsp;&nbsp;&nbsp;=&nbsp;<span style="color:teal;">DB</span>.<span style="color:navy;">readReservations</span>&nbsp;&nbsp;connectionString &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">create</span>&nbsp;=&nbsp;<span style="color:teal;">DB</span>.<span style="color:navy;">createReservation</span>&nbsp;connectionString &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">tryAccept</span>&nbsp;10&nbsp;<span style="color:navy;">read</span>&nbsp;<span style="color:navy;">create</span></pre> </p> <p> Notice how <code>tryAccept</code> itself is partially applied. Only the arguments corresponding to the C# dependencies are passed to it, so the return value is a function that 'waits' for the last argument: the reservation. As I've attempted to indicate by the code comment above the function, it has the desired type of <code>Reservation -&gt; int option</code>. </p> <p> <strong>Equivalence</strong> </p> <p> Partial application used like this is equivalent to dependency injection. To see how, consider the generated Intermediate Language (IL). </p> <p> F# is a .NET language, so it compiles to IL. You can decompile that IL to C# to get a sense of what's going on. If you do that with the above <code>tryAcceptComposition</code>, you get something like this: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">tryAcceptComposition@17&nbsp;:&nbsp;<span style="color:#2b91af;">FSharpFunc</span>&lt;<span style="color:#2b91af;">Reservation</span>,&nbsp;<span style="color:#2b91af;">FSharpOption</span>&lt;<span style="color:blue;">int</span>&gt;&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;capacity; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">FSharpFunc</span>&lt;<span style="color:#2b91af;">Reservation</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;createReservation; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">FSharpFunc</span>&lt;<span style="color:#2b91af;">DateTimeOffset</span>,&nbsp;<span style="color:#2b91af;">FSharpList</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&gt;&nbsp;readReservations; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;tryAcceptComposition@17( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;capacity, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">FSharpFunc</span>&lt;<span style="color:#2b91af;">DateTimeOffset</span>,&nbsp;<span style="color:#2b91af;">FSharpList</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&gt;&nbsp;readReservations, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">FSharpFunc</span>&lt;<span style="color:#2b91af;">Reservation</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;createReservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.capacity&nbsp;=&nbsp;capacity; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.readReservations&nbsp;=&nbsp;readReservations; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.createReservation&nbsp;=&nbsp;createReservation; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:#2b91af;">FSharpOption</span>&lt;<span style="color:blue;">int</span>&gt;&nbsp;Invoke(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">MaîtreD</span>.tryAccept&lt;<span style="color:blue;">int</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.capacity,&nbsp;<span style="color:blue;">this</span>.readReservations,&nbsp;<span style="color:blue;">this</span>.createReservation,&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I've cleaned it up a bit, mostly by removing all attributes from the various elements. Notice how this is a class, with class fields, and a constructor that takes values for the fields and assigns them. <em>It's constructor injection!</em> </p> <p> Partial application is dependency injection. </p> <p> It compiles, works as expected, but is it functional? </p> <p> <strong>Evaluation</strong> </p> <p> People sometimes ask me: <em>How do I know whether my F# code is functional?</em> </p> <p> I sometimes wonder about that myself, but unfortunately, as nice a language as F# is, it doesn't offer much help in that regard. Its emphasis is on functional programming, but it allows mutation, object-oriented programming, and even procedural programming. It's a friendly and forgiving language. (This also makes it a great 'beginner' functional language, because you can learn functional concepts piecemeal.) </p> <p> Haskell, on the other hand, is a strictly functional language. In Haskell, you can only write your code in the functional way. </p> <p> Fortunately, F# and Haskell are similar enough that it's easy to port F# code to Haskell, as long as the F# code already is 'sufficiently functional'. In order to evaluate if my F# code is properly functional, I sometimes port it to Haskell. If I can get it to compile and run in Haskell, I take that as confirmation that my code is functional. </p> <p> I've <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">previously shown an example similar to this one</a>, but I'll repeat the experiment here. Will porting <code>tryAccept</code> and <code>tryAcceptComposition</code> to Haskell work? </p> <p> It's easy to port <code>tryAccept</code>: </p> <p> <pre><span style="color:#600277;">tryAccept</span>&nbsp;::&nbsp;Int&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(<span style="color:blue;">ZonedTime</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[<span style="color:blue;">Reservation</span>])&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(<span style="color:blue;">Reservation</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Int)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Reservation</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#666666;">-&gt;</span>&nbsp;<span style="color:#dd0000;">Maybe</span>&nbsp;<span style="color:#dd0000;">Int</span> tryAccept&nbsp;capacity&nbsp;readReservations&nbsp;createReservation&nbsp;reservation&nbsp;<span style="color:#666666;">=</span> &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;<span style="color:#666666;">=</span>&nbsp;sum&nbsp;<span style="color:#666666;">$</span>&nbsp;map&nbsp;quantity&nbsp;<span style="color:#666666;">$</span>&nbsp;readReservations&nbsp;<span style="color:#666666;">$</span>&nbsp;date&nbsp;reservation &nbsp;&nbsp;<span style="color:blue;">in</span>&nbsp;&nbsp;<span style="color:#af00db;">if</span>&nbsp;reservedSeats&nbsp;<span style="color:#666666;">+</span>&nbsp;quantity&nbsp;reservation&nbsp;<span style="color:#666666;">&lt;=</span>&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#af00db;">then</span>&nbsp;Just&nbsp;<span style="color:#666666;">$</span>&nbsp;createReservation&nbsp;<span style="color:#666666;">$</span>&nbsp;reservation&nbsp;{&nbsp;isAccepted&nbsp;<span style="color:#666666;">=</span>&nbsp;True&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#af00db;">else</span>&nbsp;Nothing</pre> </p> <p> Clearly, there are differences, but I'm sure that you can also see the similarities. The most important feature of this function is that it's <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a>. All Haskell functions are pure by default, unless explicitly declared to be impure, and that's not the case here. This function is pure, and so are both <code>readReservations</code> and <code>createReservation</code>. </p> <p> The Haskell version of <code>tryAccept</code> compiles, but what about <code>tryAcceptComposition</code>? </p> <p> Like the F# code, the experiment is to see if it's possible to 'inject' functions that actually operate against a database. Equivalent to the F# example, imagine that you have this <code>DB</code> module: </p> <p> <pre><span style="color:#600277;">readReservations</span>&nbsp;::&nbsp;<span style="color:blue;">ConnectionString</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">ZonedTime</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;IO&nbsp;[<span style="color:blue;">Reservation</span>] readReservations&nbsp;connectionString&nbsp;date&nbsp;<span style="color:#666666;">=</span>&nbsp;<span style="color:green;">--&nbsp;.. </span> <span style="color:#600277;">createReservation</span>&nbsp;::&nbsp;<span style="color:blue;">ConnectionString</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Reservation</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;IO&nbsp;Int createReservation&nbsp;connectionString&nbsp;reservation&nbsp;<span style="color:#666666;">=</span>&nbsp;<span style="color:green;">--&nbsp;..</span></pre> </p> <p> Database operations are, by definition, impure, and Haskell admirably models that with the type system. Notice how both functions return <code>IO</code> values. </p> <p> If you partially apply both functions with a valid connection string, the <code>IO</code> context remains. The type of <code>DB.readReservations connectionString</code> is <code>ZonedTime -&gt; IO [Reservation]</code>, and the type of <code>DB.createReservation connectionString</code> is <code>Reservation -&gt; IO Int</code>. You can try to pass them to <code>tryAccept</code>, but the types don't match: </p> <p> <pre><span style="color:#600277;">tryAcceptComposition</span>&nbsp;::&nbsp;<span style="color:blue;">Reservation</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;IO&nbsp;(Maybe&nbsp;Int) tryAcceptComposition&nbsp;reservation&nbsp;<span style="color:#666666;">=</span> &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;read&nbsp;&nbsp;&nbsp;<span style="color:#666666;">=</span>&nbsp;<span style="color:#dd0000;">DB</span><span style="color:#666666;">.</span>readReservations&nbsp;&nbsp;connectionString &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;create&nbsp;<span style="color:#666666;">=</span>&nbsp;<span style="color:#dd0000;">DB</span><span style="color:#666666;">.</span>createReservation&nbsp;connectionString &nbsp;&nbsp;<span style="color:blue;">in</span>&nbsp;tryAccept&nbsp;<span style="color:#09885a;">10</span>&nbsp;read&nbsp;create&nbsp;reservation</pre> </p> <p> <em>This doesn't compile.</em> </p> <p> It doesn't compile, because the database operations are impure, and <code>tryAccept</code> wants pure functions. </p> <p> In short, partial application used for dependency injection isn't functional. </p> <p> <strong>Summary</strong> </p> <p> Partial application in F# can be used to achieve a result equivalent to dependency injection. It compiles and works as expected, but it's not functional. The reason it's not functional is that (most) dependencies are, by their very nature, impure. They're either non-deterministic, have side-effects, or both, and that's often the underlying reason that they are factored into dependencies in the first place. </p> <p> Pure functions, however, can't call impure functions. If they could, they would become impure themselves. This rule is enforced by Haskell, but not by F#. </p> <p> When you inject impure operations into an F# function, that function becomes impure as well. Dependency injection makes everything impure, which explains why it isn't functional. </p> <p> Functional programming solves the problem of decoupling (side) effects from program logic another way. That's the topic of the next article. </p> <p> <strong>Next:</strong> <a href="http://blog.ploeh.dk/2017/02/02/dependency-rejection">Dependency rejection</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Dependency injection is passing an argument http://blog.ploeh.dk/2017/01/27/dependency-injection-is-passing-an-argument 2017-01-27T09:27:00+00:00 Mark Seemann <div id="post"> <p> <em>Is dependency injection really just passing an argument? A brief review.</em> </p> <p> This is the first article in a small article series called <a href="http://blog.ploeh.dk/2017/01/27/from-dependency-injection-to-dependency-rejection">from dependency injection to dependency rejection</a>. </p> <p> In a talk at the 2012 Northeast Scala Symposium, <a href="https://t.co/oD1oQYZwbz">Rúnar Bjarnason casually remarked</a> that dependency injection is "really just a pretentious way to say 'taking an argument'". Given that I've written a 500+ pages <a href="http://amzn.to/12p90MG">book about dependency injection</a>, you might expect me to disagree with that. Yet, there's some truth to that statement, although it's not quite as simple as that. </p> <p> In this article, I'll show you some simple examples and explain why, on the one hand, Rúnar Bjarnason is right, but also, on the other hand, why there's a bit more to it. </p> <p> <strong>Restaurant reservation example</strong> </p> <p> Like the other articles in this series, the example scenario is on-line restaurant reservation. Imagine that you've been asked to develop an HTTP-based API that accepts JSON documents containing restaurant reservations. Furthermore, assume that you're using ASP.NET Web API with C# for the job, and that you're aspiring to use <a href="http://amzn.to/WBCwx7">domain-driven design</a>. </p> <p> In order to handle the incoming POST request, you could write an action method like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpActionResult</span>&nbsp;Post(<span style="color:#2b91af;">ReservationRequestDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;validationMsg&nbsp;=&nbsp;validator.Validate(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(validationMsg&nbsp;!=&nbsp;<span style="color:#a31515;">&quot;&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.BadRequest(validationMsg); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;r&nbsp;=&nbsp;mapper.Map(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;maîtreD.TryAccept(r); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(id&nbsp;==&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.StatusCode(<span style="color:#2b91af;">HttpStatusCode</span>.Forbidden); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.Ok(); }</pre> </p> <p> This method follows a simple and familiar path: validate input, map to a domain model, delegate to said model, examine posterior state, and return a result. </p> <p> You may have noticed, though, that this method doesn't do all the work itself. It delegates some of the work to collaborators: <code>validator</code>, <code>mapper</code>, and <code>maîtreD</code>. Where do these collaborators come from? </p> <p> They are <em>dependencies</em>. Could you make the <code>Post</code> method take them as arguments? </p> <p> Unfortunately, you can't. The <code>Post</code> method constitutes part of the boundary of the HTTP API. ASP NET Web API routes and dispatches incoming HTTP requests by convention, and <em>action methods</em> must follow that convention. You can't just make the function take any argument you'd like, so you have to find another place to pass those dependencies to the object. </p> <p> The second-best option (after the <code>Post</code> method itself) is via the constructor: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;ReservationsController( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IValidator</span>&nbsp;validator, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IMapper</span>&nbsp;mapper, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IMaîtreD</span>&nbsp;maîtreD) {&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.validator&nbsp;=&nbsp;validator; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.mapper&nbsp;=&nbsp;mapper; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.maîtreD&nbsp;=&nbsp;maîtreD; }</pre> </p> <p> This is the application of a design pattern called <em>constructor injection</em>. It captures the dependencies in class fields, making them available for members (like <code>Post</code>) of the class. </p> <p> This turns out to be a regular pattern. </p> <p> <strong>Turtles all the way down</strong> </p> <p> You could argue that the <code>Post</code> method is a special case, since it's part of the boundary of the system, and therefore must adhere to specific rules. On the other hand, these rule don't apply deeper in the implementation, so could you implement other objects by simply passing in dependencies as arguments? </p> <p> Consider, as an example, the implementation of <code>IMaîtreD.TryAccept</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>?&nbsp;TryAccept(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservedSeats&nbsp;=&nbsp;reservationsRepository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations(reservation.Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Sum(r&nbsp;=&gt;&nbsp;r.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservedSeats&nbsp;+&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;capacity) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservation.IsAccepted&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;reservationsRepository.Create(reservation); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">null</span>; }</pre> </p> <p> This method has another collaborator: <code>reservationsRepository</code>. It's another dependency. Where does it come from? </p> <p> Could you make the <code>TryAccept</code> method take <code>reservationsRepository</code> as an argument? </p> <p> Unfortunately, that's not possible either, because the method is defined by the <code>IMaîtreD</code> interface: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IMaîtreD</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>?&nbsp;TryAccept(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation); }</pre> </p> <p> You may recall that the above <code>Post</code> method is programmed against the <code>IMaîtreD</code> interface, and not the concrete class. It'd be a leaky abstraction to add <code>IReservationsRepository</code> as an argument to <code>IMaîtreD.TryAccept</code>, because not all implementations of the interface may need that dependency. Or perhaps another implementation has another dependency. Should we add that to the parameter list of <code>IMaîtreD.TryAccept</code>as well? </p> <p> Surely, that's not a tenable design principle. On the other hand, by using constructor injection, you can decouple implementation details from your abstractions: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;MaîtreD(<span style="color:blue;">int</span>&nbsp;capacity,&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;reservationsRepository) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.capacity&nbsp;=&nbsp;capacity; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.reservationsRepository&nbsp;=&nbsp;reservationsRepository; }</pre> </p> <p> This constructor not only takes an <code>IReservationsRepository</code> object, but also an integer that represents the capacity of the restaurant in question. This demonstrates that <a href="http://blog.ploeh.dk/2012/07/02/PrimitiveDependencies">dependencies can also be primitive values</a>. </p> <p> <strong>Summary</strong> </p> <p> Dependency injection is, in a sense, only a specific way for objects to take arguments. Often, however, objects have roles defined by the the interfaces they implement. Such objects may need collaborators that are not available via the APIs defined by these interfaces, so you'll have to supply dependencies via members that belong to the concrete class in question. Passing dependencies via a class' constructor is the best way to do that. </p> <p> <strong>Next:</strong> <a href="http://blog.ploeh.dk/2017/01/30/partial-application-is-dependency-injection">Partial application is dependency injection</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. From dependency injection to dependency rejection http://blog.ploeh.dk/2017/01/27/from-dependency-injection-to-dependency-rejection 2017-01-27T07:55:00+00:00 Mark Seemann <div id="post"> <p> <em>The problem typically solved by dependency injection in object-oriented programming is solved in a completely different way in functional programming.</em> </p> <p> Several years ago, I wrote a book called <a href="http://amzn.to/12p90MG">Dependency Injection in .NET</a>, which was published in 2011. The book contains examples in C#, but since then I've increasingly become interested in functional programming to the extend that I now consider F# my primary language. </p> <p> With that combination, it's no wonder that people often ask me how to do dependency injection in functional programming. </p> <p> I've seen more than one answer, from other people, explaining how partial function application is equivalent to dependency injection. In a small series of articles, I'll explain both why this is true, but also why it's <em>not functional</em>. I'll conclude by showing a functional alternative to decoupling logic and (side) effects. </p> <p> <img src="/content/binary/dependency-injection-in-scala-comic.jpg" alt="Bob: How do I do dependency injection in Scala? Other man: You don't, because Scala is a functional language. Bob: Fine, it's functional. How do I inject dependencies? Other man: You use a free monad which allows you to build a monad from any Functor. Bob: Did you just tell me to go fuck myself? Other man: I believe I did, Bob."> </p> <p> (<a href="https://twitter.com/hmemcpy/status/771359835514368000">Comic</a> courtesy of <a href="http://thinkdifferent.ly">John Muellerleile</a> and <a href="http://hmemcpy.com">Igal Tabachnik</a>.) </p> <p> There's another school of functional programmers who believe that dependency injection in functional programming involves a Free monad. </p> <p> You can often make do with less, though. </p> <p> In my experience, it's usually enough to refactor a unit to take only direct input and output, and then compose an impure/pure/impure 'sandwich'. You'll see an example later. </p> <p> This article series contains the following parts: <ol> <li><a href="http://blog.ploeh.dk/2017/01/27/dependency-injection-is-passing-an-argument">Dependency injection is passing an argument</a></li> <li><a href="http://blog.ploeh.dk/2017/01/30/partial-application-is-dependency-injection">Partial application is dependency injection</a></li> <li><a href="http://blog.ploeh.dk/2017/02/02/dependency-rejection">Dependency rejection</a></li> </ol> All articles revolve around a common example, which is one of my favourite scenarios: on-line restaurant reservation. You can see an actual example client in my <a href="http://www.shareasale.com/r.cfm?u=1017843&b=611266&m=53701&afftrack=&urllink=pluralsight%2Ecom%2Fcourses%2Ffunctional%2Darchitecture%2Dfsharp">Functional Architecture with F#</a> Pluralsight course. The (somewhat dated) client source code is <a href="https://github.com/ploeh/booking-web-ui">available on GitHub</a>. The server-side F# and Haskell example code for this article series is <a href="https://github.com/ploeh/dependency-rejection-samples">available on GitHub</a>. </p> <p> The scenario is to implement an HTTP-based API that can accept incoming JSON documents that represent restaurant reservations. </p> <p> I should point out that nowhere in this article series do I reject dependency injection as a set of object-oriented patterns. In object-oriented programming, dependency injection is a well-known and comprehensively described way to achieve decoupling and testability. In the next article, you'll see a brief review of dependency injection in C#. </p> <p> <strong>Next:</strong> <a href="http://blog.ploeh.dk/2017/01/27/dependency-injection-is-passing-an-argument">Dependency injection is passing an argument</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Decoupling application errors from domain models http://blog.ploeh.dk/2017/01/03/decoupling-application-errors-from-domain-models 2017-01-03T12:26:00+00:00 Mark Seemann <div id="post"> <p> <em>How to prevent application-specific error cases from infecting your domain models.</em> </p> <p> Functional error-handling is often done with the Either monad. If all is good, the <em>right</em> case is returned, but if things go wrong, you'll want to return a value that indicates the error. In an application, you'll often need to be able to distinguish between different kinds of errors. </p> <p> <strong>From application errors to HTTP responses</strong> </p> <p> When an application encounters an error, it should respond appropriately. A GUI-based application should inform the user about the error, a batch job should log it, and a <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">REST</a> API should return the appropriate HTTP status code. </p> <p> Regular readers of this blog will know that I write many RESTful APIs in F#, using ASP.NET Web API. Since I like to write functional F#, but ASP.NET Web API is an object-oriented framework, I prefer to escape the object-oriented framework as soon as possible. (In general, it makes good architectural sense to write most of your code as framework-independent as possible.) </p> <p> In my <a href="http://bit.ly/tddwithfsharp">Test-Driven Development with F#</a> Pluralsight course (a <a href="http://www.infoq.com/presentations/mock-fsharp-tdd">free, condensed version is also available</a>), I demonstrate how to handle various error cases in a Controller class: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">ReservationsController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(dtr&nbsp;:&nbsp;<span style="color:teal;">ReservationDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;dtr&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationError</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span>&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">StatusCode</span>&nbsp;<span style="color:teal;">HttpStatusCode</span>.Forbidden&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;_</pre> </p> <p> The injected <code>imp</code> function is a complete, composed, vertical feature implementation that performs both input validation, business logic, and data access. If input validation fails, it'll return <code>Failure (ValidationError msg)</code>, and that value is translated to a <code>400 Bad Request</code> response. Likewise, if the business logic returns <code>Failure CapacityExceeded</code>, the response becomes <code>403 Forbidden</code>, and a success is returned as <code>200 OK</code>. </p> <p> Both <code>ValidationError</code> and <code>CapacityExceeded</code> are cases of an <code>Error</code> type. This is only a simple example, so these are the only cases defined by that type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">Error</span>&nbsp;= |&nbsp;<span style="color:navy;">ValidationError</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span> |&nbsp;<span style="color:navy;">CapacityExceeded</span></pre> </p> <p> This seems reasonable, but there's a problem. </p> <p> <strong>Error infection</strong> </p> <p> In F#, a function can't use a type unless that type is already defined. This is a problem because the <code>Error</code> type defined above mixes different concerns. If you seek to <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">make illegal states unrepresentable</a>, it follows that validation is not a concern in your domain model. Validation is still important at the boundary of an application, so you can't just ignore it. The <code>ValidationError</code> case relates to the application boundary, while <code>CapacityExceeded</code> relates to the domain model. </p> <p> Still, when implementing your domain model, you may want to return a <code>CapacityExceeded</code> value from time to time: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;Result&lt;Reservation,Error&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">checkCapacity</span>&nbsp;capacity&nbsp;reservedSeats&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;capacity&nbsp;&lt;&nbsp;reservation.Quantity&nbsp;+&nbsp;reservedSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;reservation</pre> </p> <p> Notice how the return type of this function is <code>Result&lt;Reservation,Error&gt;</code>. In order to be able to implement your domain model, you've now pulled in the <code>Error</code> type, which also defines the <code>ValidationError</code> case. Your domain model is now polluted by an application boundary concern. </p> <p> I think many developers would consider this trivial, but in my experience, failure to manage dependencies is the dominant reason for <a href="https://en.wikipedia.org/wiki/Software_rot">code rot</a>. It makes the code less general, and less reusable, because it's now coupled to something that may not fit into a different context. </p> <p> Particularly, the situation in the example looks like this: </p> <p> <img src="/content/binary/error-infection-dependency-diagram.png" alt="Dependency diagram"> </p> <p> Boundary and data access modules depend on the domain model, <a href="http://blog.ploeh.dk/2013/12/03/layers-onions-ports-adapters-its-all-the-same">as they should</a>, but everything depends on the <code>Error</code> type. This is wrong. Modules or libraries should be able to define their own error types. </p> <p> The <code>Error</code> type belongs in the <a href="http://blog.ploeh.dk/2011/07/28/CompositionRoot">Composition Root</a>, but it's impossible to put it there because F# prevents circular dependencies (a <a href="http://blog.ploeh.dk/2015/04/13/less-is-more-language-features">treasured language feature</a>). </p> <p> Fortunately, the fix is straightforward. </p> <p> <strong>Mapped Either values</strong> </p> <p> A domain model should be self-contained. As Robert C. Martin puts it in <a href="http://amzn.to/19W4JHk">APPP</a>: <blockquote> Abstractions should not depend upon details. Details should depend upon abstractions. </blockquote> Your domain model is an abstraction of the real world (that's why it's called a <em>model</em>), and is the reason you're developing a piece of software in the first place. So start with the domain model: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">BookingError</span>&nbsp;=&nbsp;<span style="color:navy;">CapacityExceeded</span> <span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;Result&lt;Reservation,BookingError&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">checkCapacity</span>&nbsp;capacity&nbsp;reservedSeats&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;capacity&nbsp;&lt;&nbsp;reservation.Quantity&nbsp;+&nbsp;reservedSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;reservation</pre> </p> <p> In this example, there's only a single type of domain error (<code>CapacityExceeded</code>), but that's mostly because this is an example. Real production code could define a domain error union with several cases. The crux of the matter is that <code>BookingError</code> isn't infected with irrelevant implementation details like validation error types. </p> <p> You're still going to need an exhaustive discriminated union to model all possible error cases for your particular application, but that type belongs in the Composition Root. Accordingly, you also need a way to return validation errors in your validation module. Often, a <code>string</code> is all you need: </p> <p> <pre><span style="color:green;">//&nbsp;ReservationDtr&nbsp;-&gt;&nbsp;Result&lt;Reservation,string&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">validateReservation</span>&nbsp;(dtr&nbsp;:&nbsp;<span style="color:teal;">ReservationDtr</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;dtr.Date&nbsp;|&gt;&nbsp;<span style="color:teal;">DateTimeOffset</span>.<span style="color:navy;">TryParse</span>&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;(<span style="color:blue;">true</span>,&nbsp;date)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Reservation</span>.Date&nbsp;=&nbsp;date &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;dtr.Name &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;dtr.Email &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;dtr.Quantity&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span></pre> </p> <p> The <code>validateReservation</code> function returns a <code>Reservation</code> value when validation succeeds, and a simple <code>string</code> with an error message if it fails. </p> <p> You could, conceivably, return <code>string</code> values for errors from many different places in your code, so you're going to map them into an appropriate error case that makes sense in your application. </p> <p> In this particular example, the Controller shown above should still look like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">Error</span>&nbsp;= |&nbsp;<span style="color:navy;">ValidationError</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span> |&nbsp;<span style="color:navy;">DomainError</span> <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">ReservationsController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(dtr&nbsp;:&nbsp;<span style="color:teal;">ReservationDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;dtr&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationError</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">DomainError</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">StatusCode</span>&nbsp;<span style="color:teal;">HttpStatusCode</span>.Forbidden&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;_</pre> </p> <p> Notice how similar this is to the initial example. The important difference, however, is that <code>Error</code> is defined in the same module that also implements <code>ReservationsController</code>. This is part of the composition of the specific application. </p> <p> In order to make that work, you're going to need to map from one failure type to another. This is trivial to do with an extra function belonging to your Result (or Either) module: </p> <p> <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;Result&lt;&#39;c,&#39;a&gt;&nbsp;-&gt;&nbsp;Result&lt;&#39;c,&#39;b&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapFailure</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;succ&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;succ &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">f</span>&nbsp;fail)</pre> </p> <p> This function takes any <code>Result</code> value and maps the failure case instead of the success case. It enables you to transform e.g. a <code>BookingError</code> into a <code>DomainError</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;candidate&nbsp;=&nbsp;<span style="color:blue;">either</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:navy;">validateReservation</span>&nbsp;candidate&nbsp;|&gt;&nbsp;<span style="color:navy;">mapFailure</span>&nbsp;<span style="color:navy;">ValidationError</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;&nbsp;i&nbsp;=&nbsp;<span style="color:teal;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:navy;">checkCapacity</span>&nbsp;10&nbsp;i&nbsp;r&nbsp;|&gt;&nbsp;<span style="color:navy;">mapFailure</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">DomainError</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:teal;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString&nbsp;r&nbsp;}</pre> </p> <p> This composition is a variation of the composition I've <a href="http://blog.ploeh.dk/2016/03/21/composition-with-an-either-computation-expression">previously published</a>. The only difference is that the error cases are now mapped into the application-specific <code>Error</code> type. </p> <p> <strong>Conclusion</strong> </p> <p> Errors can occur in diverse places in your code base: when validating input, when making business decisions, when writing to, or reading from, databases, and so on. </p> <p> When you use the Either monad for error handling, in a strongly typed language like F#, you'll need to define a discriminated union that models all the error cases you care about in the specific application. You can map module-specific error types into such a comprehensive error type using a function like <code>mapFailure</code>. In Haskell, it would be the <code>first</code> function of the <code>Bifunctor</code> typeclass, so this is a well-known function. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://github.com/wortho">David Worthington</a></div> <div class="comment-content"> <p> Mark, </p> <p> Why is it a problem to use <code>HttpStatusCode</code> in the domain model. They appear to be a standard way of categorizing errors. </p> </div> <div class="comment-date">2017-02-07 13:37 UTC</div> </div> <div class="comment"> <div class="comment-author"> <a href="http://blog.ploeh.dk">Mark Seemann</a> </div> <div class="comment-content"> </div> <p> David, thank you for writing. The answer depends on your goals and definition of <em>domain model</em>. </p> <p> I usually think of domain models in terms of separation of concerns. The purpose of a domain model is to model the business logic, and as Martin Fowler writes in <a href="http://bit.ly/patternsofeaa">PoEAA</a> about the Domain Model pattern, "you'll want the minimum of coupling from the <em>Domain Model</em> to other layers in the system. You'll notice that a guiding force of many layering patterns is to keep as few dependencies as possible between the domain model and the other parts of the system." </p> <p> In other words, you're separating the concern of implementing the business rules from the concerns of being able to save data in a database, render it on a screen, send emails, and so on. While also important, these are separate concerns, and I want to be able to vary those independently. </p> <p> People often hear statements like that as though I want to reserve myself the right to replace my SQL Server database with Neo4J (more on that later, though!). That's actually not my main goal, but I find that if concerns are mixed, <em>all</em> change becomes harder. It becomes more difficult to change how data is saved in a database, <em>and</em> it becomes harder to change business rules. </p> <p> The Dependency Inversion Principle tries to address such problems by advising that abstractions shouldn't depend on implementation details, but instead, implementation details should depend on abstractions. </p> <p> This is where the goals come in. I find Robert C. Martin's definition of software architecture helpful. Paraphrased from memory, he defines a software architect's role as enabling change; not predicting change, but making sure that when change has to happen, it's as economical as possible. </p> <p> As an architect, one of the heuristics I use is that I try to imagine how easily I can replace one component with another. It's not that I really believe that I may have to replace the SQL Server database with Neo4J, but thinking about how hard it would be gives me some insights about how to structure a software solution. </p> <p> I also imagine what it'd be like to port an application to another environment. Can I port my web site's business rules to a batch job? Can I port my desktop client to a smart phone app? Again, it's not that I necessarily predict that I'll have to do this, but it tells me something about the degrees of freedom offered by the architecture. </p> <p> If not explicitly addressed, the opposite of freedom tends to happen. In <a href="http://amzn.to/19W4JHk">APPP</a>, Robert C. Martin describes a number of design smells, one of them <em>Immobility</em>: "A design is immobile when it contains parts that could be useful in other systems, but the effort and risk involved with separating those parts from the original system are too great. This is an unfortunate, but very common occurrence." </p> <p> Almost as side-effect, an immobile system is difficult to test. A unit test is a different environment than the intended environment. Well-architected systems are easy to unit test. </p> <p> HTTP is a communications protocol. Its purpose is to enable exchange of information over networks. While it does that well, it's specifically concerned with that purpose. This includes HTTP status code. </p> <p> If you use the heuristic of imagining that you'd have to move the heart of your application to a batch job, status codes like <code>301 Moved Permanently</code>, <code>404 Not Found</code>, or <code>405 Method Not Allowed</code> make little sense. </p> <p> Using HTTP status codes in a domain model couples the model to a particular environment, at least conceptually. It has little to do with the <em>ubiquitous language</em> that Eric Evans discusses in <a href="http://amzn.to/WBCwx7">DDD</a>. </p> <div class="comment-date">2017-02-07 17:08 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. From REST to algebraic data http://blog.ploeh.dk/2016/12/16/from-rest-to-algebraic-data 2016-12-16T07:23:00+00:00 Mark Seemann <div id="post"> <p> <em>Mapping RESTful HTTP requests to values of algebraic data types is easy.</em> </p> <p> In previous articles, you've seen <a href="http://blog.ploeh.dk/2016/11/28/easy-domain-modelling-with-types">how to easily model a simple domain model with algebraic data types</a>, and <a href="http://blog.ploeh.dk/2016/12/07/domain-modelling-with-rest">how to use RESTful API design to surface such a model at the boundary of an application</a>. In this article, you'll see how trivial it is to map incoming HTTP requests back to values of <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>. </p> <p> The advantage of <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">REST</a> is that you can make illegal states unrepresentable. Clients follow links, and while clients are supposed to treat links as opaque values, URLs still contain information your API can use. </p> <p> <strong>Routing and dispatching</strong> </p> <p> Continuing where the <a href="http://blog.ploeh.dk/2016/12/07/domain-modelling-with-rest">previous article</a> left off, clients can issue POST requests against a URL like <code>https://example.com/credit-card</code>. On the server, a well-known piece of code handles such requests. (In the example code base I've used so far, I've been using ASP.NET Web API, so the code that handles such a request is a Controller.) Since you know that URLs like that are always routed to that particular piece of code, you can create a new <code>PaymentType</code> value that specifically represents an individual payment with a credit card: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;paymentType&nbsp;=&nbsp;<span style="color:navy;">Individual</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;credit-card&quot;</span>;&nbsp;Action&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Pay&quot;</span>&nbsp;}</pre> </p> <p> If, on the other hand, the client is using a provided link to POST a representation against the URL <code>https://example.com/recurrent/start/credit-card</code>, your server-side dispatcher will route the request to a different handler (Controller), in which case you can create a <code>PaymentType</code> value like this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;paymentType&nbsp;=&nbsp;<span style="color:navy;">Parent</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;credit-card&quot;</span>;&nbsp;Action&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Pay&quot;</span>&nbsp;}</pre> </p> <p> Finally, if the client has already created a parent payment and is now using the resulting link to create child payments, it may be POSTing to a URL like <code>https://example.com/recurrent/42</code>. Your server-side dispatcher will route that request to a third handler. Most web frameworks, including ASP.NET Web API, will be able to pull values out of URLs. In this case, you can configure it so that it pulls the value <code>42</code> out of the URL and binds it to a value called <code>transactionKey</code>. With this, again it's trivial to create a <code>PaymentType</code> value: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;paymentType&nbsp;=&nbsp;<span style="color:navy;">Child</span>&nbsp;(transactionKey,&nbsp;{&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;credit-card&quot;</span>;&nbsp;Action&nbsp;=&nbsp;<span style="color:#a31515;">&quot;PayRecurrent&quot;</span>&nbsp;})</pre> </p> <p> Notice that, despite containing different data, and being created three different places in the code base, they all have the same type: <code>PaymentType</code>. This means that you can pass these values to a common <code>pay</code> function, which handles the actual communication with the third-party payment service. </p> <p> <strong>Code reuse</strong> </p> <p> Independent of the route the data arrived at, a central, reusable function named <code>pay</code> handles all such payments. This is still an impure boundary function that takes various other input apart from <code>PaymentType</code>. Without going into too much detail, it has a type like <code>Config -&gt; PaymentType -&gt; Result&lt;PaymentDtr,BoundaryFailure&gt;</code>. Don't worry if some of the details look obscure; the important point is that <code>pay</code> is a function that takes a <code>PaymentType</code> value as input. You can visualise the transition from HTTP requests to a function call like this: </p> <p> <img src="/content/binary/from-http-to-payment-type-values.png" alt="Three different URLs mapped to three different PaymentType values, which are again passed to the single pay function"> </p> <p> The <code>pay</code> function is composed from various smaller functions, some <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a> and some impure. Ultimately, it transforms all the input data to the format required by the third-party payment service, and forwards the transaction information. Inside that function you'll find the pattern match that you saw in <a href="http://blog.ploeh.dk/2016/12/07/domain-modelling-with-rest">my previous article</a>. </p> <p> <strong>Summary</strong> </p> <p> By making good use of routing and dispatching, you can easily map incoming HTTP requests to values of algebraic data types. This enables you to close the loop on exposing your domain model at the boundary of your system. Not only can clients request data from your API in terms of your model, but when clients send data to your API, you can translate that data back to your model. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Domain modelling with REST http://blog.ploeh.dk/2016/12/07/domain-modelling-with-rest 2016-12-07T09:15:00+00:00 Mark Seemann <div id="post"> <p> <em>Make illegal states unrepresentable by using hyperlinks as the engine of application state.</em> </p> <p> Every piece of software, whether it's a web service, smart phone app, batch job, or speech recognition system, interfaces with the world in some way. Sometimes, that interface is a user interface, sometimes it's a machine-readable interface; sometimes it involves rendering pixels on a screen, and sometimes it involves writing to files, selecting records from a database, sending emails, and so on. </p> <p> Programmers often struggle with how to model these interactions. This is particularly difficult because at the boundaries, systems no longer adhere to popular programming paradigms. Previously, I've explained why, <a href="http://blog.ploeh.dk/2011/05/31/AttheBoundaries,ApplicationsareNotObject-Oriented">at the boundaries, applications aren't object-oriented</a>. By the same type of argument, neither are they functional (as in 'functional programming'). </p> <p> If that's the case, why should you even bother with 'domain modelling'? Particularly, does it even matter that, with <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>, you can <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">make illegal states unrepresentable</a>? If you need to compromise once you hit the boundary of your application, is it worth the effort? </p> <p> It is, if you structure your application correctly. Proper (<a href="http://martinfowler.com/articles/richardsonMaturityModel.html">level 3</a>) <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">REST</a> architecture gives you one way to structure applications in such a way that you can surface the constraints of your domain model to the interface layer. When done correctly, you can also make illegal states unrepresentable at the boundary. </p> <p> <strong>A payment example</strong> </p> <p> In my <a href="http://blog.ploeh.dk/2016/11/28/easy-domain-modelling-with-types">previous article</a>, I demonstrated how to use (static) types to model an on-line payment domain. To summarise, my task was to model three types of payments: <ul> <li>Individual payments, which happen only once.</li> <li>Parent payments, which start a long-term payment relationship.</li> <li>Child payments, which are automated payments originally authorised by an initial parent payment.</li> </ul> The constraint I had to model is that a child payment requires a <em>transaction key</em> that identifies the original parent payment. When making a payment, however, it's only valid to supply a transaction key for a child payment. It'd be invalid to supply a transaction key to a parent or an individual payment. On the other hand, you have to distinguish individual payments from parent payments, because only parent payments can be used to start a long-term payment relationship. All this leads to this table of possible combinations: <table> <thead> <tr> <th></th> <th><code>"StartRecurrent" : "false"</code></th> <th><code>"StartRecurrent" : "true"</code></th> </tr> </thead> <tbody> <tr> <td><code>"OriginalTransactionKey" : null</code></td> <td><em>Individual</em></td> <td><em>Parent</em></td> </tr> <tr> <td><code>"OriginalTransactionKey" : "1234ABCD"</code></td> <td><em>Child</em></td> <td><strong>(Illegal)</strong></td> </tr> </tbody> </table> The table shows that it's illegal to simultaneously provide a transaction key and set <code>StartRecurrent</code> to <code>true</code>. The other three combinations, on the other hand, are valid. </p> <p> As I demonstrated in my <a href="http://blog.ploeh.dk/2016/11/28/easy-domain-modelling-with-types">previous article</a>, you can easily model this with algebraic data types. </p> <p> At the boundary, however, there are no static types, so how could you model something like that as a web service? </p> <p> <strong>A RESTful solution</strong> </p> <p> A major advantage of REST is that it gives you a way to realise your domain model at the application boundary. It does require, though, that you design the API according to <a href="http://martinfowler.com/articles/richardsonMaturityModel.html">level 3 of the Richardson maturity model</a>. In other words, it's not REST if you're merely tunnelling JSON (or XML) through HTTP. It's still not REST if you publish URL templates and expect clients to fill data into specific place-holders of those URLs. </p> <p> It's REST if the only way a client can interact with your API is by following hyperlinks. </p> <p> If you follow those design principles, however, it's easy to model the above payment domain as a RESTful API. </p> <p> In the following, I will show examples in XML, but it could as well have been JSON. After all, <a href="http://blog.ploeh.dk/2015/06/22/rest-implies-content-negotiation">a true REST API <em>must</em> support content negotiation</a>. One of the reasons that I prefer XML is that I can use <a href="https://en.wikipedia.org/wiki/XPath">XPath</a> to point out various nodes. </p> <p> A client must begin at a pre-published 'home' resource, just like the home page of a web site. This resource presents <a href="https://en.wikipedia.org/wiki/Affordance">affordances</a> in the shape of hyperlinks. As recommended by the <a href="http://amzn.to/YFnkRg">RESTful Web Services Cookbook</a>, I always use <a href="https://en.wikipedia.org/wiki/Atom_(standard)">ATOM</a> links: </p> <p> <pre><span style="color:blue;">&lt;</span><span style="color:#a31515;">home</span><span style="color:blue;">&nbsp;</span><span style="color:red;">xmlns</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">http://example.com/payment</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">xmlns:atom</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">http://www.w3.org/2005/Atom</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">payment-methods</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">payment-method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">atom:link</span><span style="color:blue;">&nbsp;</span><span style="color:red;">rel</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">example:pay-individual</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">href</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">https://example.com/gift-card</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">id</span><span style="color:blue;">&gt;</span>gift-card<span style="color:blue;">&lt;/</span><span style="color:#a31515;">id</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">payment-method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">payment-method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">atom:link</span><span style="color:blue;">&nbsp;</span><span style="color:red;">rel</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">example:pay-individual</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">href</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">https://example.com/credit-card</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">atom:link</span><span style="color:blue;">&nbsp;</span><span style="color:red;">rel</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">example:pay-parent</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">href</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">https://example.com/recurrent/start/credit-card</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">id</span><span style="color:blue;">&gt;</span>credit-card<span style="color:blue;">&lt;/</span><span style="color:#a31515;">id</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">payment-method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">payment-methods</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&lt;/</span><span style="color:#a31515;">home</span><span style="color:blue;">&gt;</span> </pre> </p> <p> A client receiving the above response is effectively presented with a choice. It can choose to pay with a gift card or credit card, and nothing else, however much it'd like to pay with, say, PayPal. For each of these payment methods, zero or more links are available. </p> <p> Specifically, there are links to create both an individual or a parent payment with a credit card, but it's only possible to make an individual payment with a gift card. You can't set up a long-term, automated payment relationship with a gift card. (This may or may not make sense, depending on how you look at it, but it's fundamentally a business decision.) </p> <p> Links are defined by <em>relationship types</em>, which are unique identifiers present in the <code>rel</code> attributes. You can think of them as equivalent to the human-readable text in an HTML <code>a</code> tag that suggests to a human user what to expect from clicking the link; only, <code>rel</code> attribute values are machine-readable and part of the contract between client and service. </p> <p> Notice how the above XML representation only gives a client the option of making an individual or a parent payment with a credit card. A client can't make a child payment at this point. This follows the domain model, because you can't make a child payment without first having made a parent payment. </p> <p> <strong>RESTful individual payments</strong> </p> <p> If a client wishes to make an individual payment, it follows the link identified by </p> <p> <pre>/home/payment-methods/payment-method[id = 'credit-card']/links/atom:link[@rel = 'example:pay-individual']/@href</pre> </p> <p> In the above XPath query, I've ignored the default document namespace in order to make the expression more readable. The query returns <code>https://example.com/credit-card</code>, and the client can now make an HTTP POST request against that URL with a JSON or XML document containing details about the payment (not shown here, because it's not important for this particular discussion). </p> <p> <strong>RESTful parent payments</strong> </p> <p> If a client wishes to make a parent payment, the initial procedure is the same. First, it follows the link identified by </p> <p> <pre>/home/payment-methods/payment-method[id = 'credit-card']/links/atom:link[@rel = 'example:pay-parent']/@href</pre> </p> <p> The result of that XPath query is <code>https://example.com/recurrent/start/credit-card</code>, so the client can make an HTTP POST request against that URL with the payment details. Unlike the response for an individual payment, the response for a parent payment contains <em>another</em> link: </p> <p> <pre><span style="color:blue;">&lt;</span><span style="color:#a31515;">payment</span><span style="color:blue;">&nbsp;</span><span style="color:red;">xmlns</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">http://example.com/payment</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">xmlns:atom</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">http://www.w3.org/2005/Atom</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">atom:link</span><span style="color:blue;">&nbsp;</span><span style="color:red;">rel</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">example:pay-child</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">href</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">https://example.com/recurrent/42</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">atom:link</span><span style="color:blue;">&nbsp;</span><span style="color:red;">rel</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">example:payment-details</span>&quot; <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color:red;">href</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">https://example.com/42</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">links</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">amount</span><span style="color:blue;">&gt;</span>13.37<span style="color:blue;">&lt;/</span><span style="color:#a31515;">amount</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">currency</span><span style="color:blue;">&gt;</span>EUR<span style="color:blue;">&lt;/</span><span style="color:#a31515;">currency</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">invoice</span><span style="color:blue;">&gt;</span>1234567890<span style="color:blue;">&lt;/</span><span style="color:#a31515;">invoice</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&lt;/</span><span style="color:#a31515;">payment</span><span style="color:blue;">&gt;</span></pre> </p> <p> This response echoes the details of the payment: this is a payment of 13.37 EUR for invoice 1234567890. It also includes some links that a client can use to further interact with the payment: <ul> <li>The <code>example:payment-details</code> link can be used to query the API for details about the payment, for example its status.</li> <li>The <code>example:pay-child</code> link can be used to make a child payment.</li> </ul> The <code>example:pay-child</code> link is only returned if the previous payment was a parent payment. When a client makes an individual payment, this link isn't present in the response, but when the client makes a parent payment, it is. </p> <p> Another design principle of REST is that <a href="https://www.w3.org/Provider/Style/URI.html.en">cool URIs don't change</a>; once the API has shown a URL like <code>https://example.com/recurrent/42</code> to a client, it should honour that URL indefinitely. The upshot of that is that a client can <em>save</em> that URL for later use. If a client wants to, say, renew a subscription, it can make a new HTTP POST request to that URL <em>a month later</em>, and that's going to be a child payment. Clients don't have to <a href="http://blog.ploeh.dk/2013/05/01/rest-lesson-learned-avoid-hackable-urls">hack the URL</a> in order to figure out what the transaction key is; they can simply store the complete URL as is and use it later. </p> <p> <strong>A network of options</strong> </p> <p> Using a design like the one sketched above, you can make illegal states unrepresentative. There's no way for a client to make a payment with <code>StartRecurrent = true</code> and a non-null transaction key; there's no link to that combination. Such an API uses <a href="https://en.wikipedia.org/wiki/HATEOAS">hypermedia as the engine of application state</a>. </p> <p> It shouldn't be surprising that proper RESTful design works that way. After all, REST is essentially a distillate of the properties that make the World Wide Web work. On a human-readable web page, the user follows links to other pages, and a well-designed web site will only enable a link if the destination exists. </p> <p> You can even draw a graph of the the API I've sketched above: </p> <p> <img src="/content/binary/graph-of-payment-paths.png" alt="Graph of payment options, including a start node, and end node, and a node for each of the three payment types"> </p> <p> In this diagram, you can see that when you make an individual payment, that's all you can do. You can also see that the only way to make a child payment is by first making a parent payment. There's also a path from parent payments directly to the end node, because a client doesn't <em>have</em> to make a child payment just because it made a parent payment. </p> <p> If you think that this looks like a finite state machine, then that's no coincidence. That's exactly what it is. You have states (the nodes) and paths between them. If a state is illegal, then don't add that node; only add nodes for legal states, then add links between the nodes that model legal transitions. </p> <p> Incidentally, languages like F# excel at implementing finite state machines, so it's no wonder I like to implement RESTful APIs in F#. </p> <p> <strong>Summary</strong> </p> <p> Truly RESTful design enables you to make illegal states unrepresentable by using hypermedia as the engine of application state. This gives you a powerful design tool to ensure that clients can only perform correct operations. </p> <p> As I also wrote in <a href="http://blog.ploeh.dk/2016/11/28/easy-domain-modelling-with-types">my previous article</a>, this, too, is no silver bullet. You can turn an API into a pit of success, but there are still many fault scenarios that you can't prevent. </p> <p> If you were intrigued by this article, but are having trouble applying these design techniques to your own field, I'm <a href="http://blog.ploeh.dk/hire-me">available for hire</a> for short or long-term engagements. </p> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Easy domain modelling with types http://blog.ploeh.dk/2016/11/28/easy-domain-modelling-with-types 2016-11-28T07:21:00+00:00 Mark Seemann <div id="post"> <p> <em>Algebraic data types make domain modelling easy.</em> </p> <p> People often ask me if I think that F# is a good general-purpose language, and when I emphatically answer <em>yes!</em>, the natural next question is: <em>why?</em> </p> <p> For years, I've been able to answer this question in the abstract, but I've been looking for a good concrete example with which I could illustrate the answer. I believe that I've now found such an example. </p> <p> The abstract answer, by the way, is that F# has <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>, which makes domain modelling much easier than in languages that don't have such types. Don't worry if the word 'algebraic' sounds scary, though. It's not at all difficult to understand, and I'll show you a simple example. </p> <p> <strong>Payment types</strong> </p> <p> At the moment, I'm working on an integration project: I'm developing a RESTful API that serves as Facade in front of a third-party payment provider. The third-party provider exposes their own API and web-based GUI that enable our end users to pay for services using credit cards, PayPal, and so on. The API that I'm developing presents a simplified, RESTful API to other clients in our organisation. </p> <p> The example you're going to see here is real code that I'm writing in order to implement the desired functionality. </p> <p> The system must be able to handle several different types of payment: <ul> <li>Sometimes, a user pays for a single thing, and that's the end of that transaction.</li> <li>Other times, however, a user engages into a long-term payment relationship. This could be, for example, a subscription, or an 'auto-fill' style of relationship. This is handled in two distinct phases:</li> <ul> <li>An initial payment (can sometimes be for a zero amount) that authorises the merchant to make further transactions.</li> <li>Subsequent payments, based off that initial payment. These payments can be automated, because they require no further user interaction than the initial authorisation.</li> </ul> </ul> The third-party service calls these 'long-term relationship' payments for <em>recurring</em> payments, but in order to distinguish between the first and the subsequent payments in such a relationship, I decided to call them <em>parent</em> and <em>child</em> payments; accordingly, I call the one-off payments <em>individual</em> payments. </p> <p> You can indicate the type of payment when interacting with the payment service's JSON-based API, like this: </p> <p> <pre>{ &nbsp;&nbsp;... &nbsp;&nbsp;"StartRecurrent": "false" &nbsp;&nbsp;... }</pre> </p> <p> Obviously, as the (illegal) ellipses suggests, there's much more data associated with a payment, but that's not important in this example. Since <code>StartRecurrent</code> is <code>false</code>, this is either an individual payment, or a child payment. If you want to start a long-term relationship, you must create a parent payment and set <code>StartRecurrent</code> to <code>true</code>. </p> <p> Child payments, however, are a bit different, because you have to tell the payment service about the parent payment: </p> <p><pre>{ &nbsp;&nbsp;... &nbsp;&nbsp;"OriginalTransactionKey": "1234ABCD", &nbsp;&nbsp;"StartRecurrent": "false" &nbsp;&nbsp;... }</pre> </p> <p> As you can see, when making a child payment, you supply the transaction ID for the parent payment. (This ID is given to you by the payment service when you initiate the parent payment.) </p> <p> In this case, you're clearly <em>not</em> starting a <em>new</em> recurrent transaction. </p> <p> There are two dimensions of variation in this example: <code>StartRecurrent</code> and <code>OriginalTransactionKey</code>. Let's put them in a table: <table> <thead> <tr> <th></th> <th><code>"StartRecurrent" : "false"</code></th> <th><code>"StartRecurrent" : "true"</code></th> </tr> </thead> <tbody> <tr> <td><code>"OriginalTransactionKey" : null</code></td> <td><em>Individual</em></td> <td><em>Parent</em></td> </tr> <tr> <td><code>"OriginalTransactionKey" : "1234ABCD"</code></td> <td><em>Child</em></td> <td><strong>(Illegal)</strong></td> </tr> </tbody> </table> As the table suggests, the combination of an <code>OriginalTransactionKey</code> and setting <code>StartRecurrent</code> to <code>true</code> is illegal, or, in best case, meaningless. </p> <p> How would you model the rules laid out in the above table? In languages like C#, it's difficult, but in F# it's easy. </p> <p> <strong>C# attempts</strong> </p> <p> Most C# developers would, I think, attempt to model a payment transaction with a class. If they aim for <a href="http://blog.ploeh.dk/2011/05/24/Poka-yokeDesignFromSmelltoFragrance">poka-yoke design</a>, they might come up with a design like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PaymentType</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;PaymentType(<span style="color:blue;">bool</span>&nbsp;startRecurrent) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.StartRecurrent&nbsp;=&nbsp;startRecurrent; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;PaymentType(<span style="color:blue;">string</span>&nbsp;originalTransactionKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(originalTransactionKey&nbsp;==&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(originalTransactionKey)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.StartRecurrent&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.OriginalTransactionKey&nbsp;=&nbsp;originalTransactionKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;StartRecurrent&nbsp;{&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;OriginalTransactionKey&nbsp;{&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;<span style="color:blue;">get</span>;&nbsp;} }</pre> </p> <p> This goes a fair way towards <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">making illegal states unrepresentable</a>, but it doesn't communicate to a fellow programmer how it should be used. </p> <p> Code that <em>uses</em> instances of this <code>PaymentType</code> class could attempt to read the <code>OriginalTransactionKey</code>, which, depending on the type of payment, could return null. That sort of design leads to <a href="http://blog.ploeh.dk/2013/07/08/defensive-coding">defensive coding</a>. </p> <p> Other people might attempt to solve the problem by designing a class hierarchy: </p> <p> <img src="/content/binary/hypothetical-payment-class-hierarchy.png" alt="A hypothetical payment class hierarchy, showing a Payment base class, and three derived classes: IndividualPayment, ParentPayment, and ChildPayment."> </p> <p> (A variation on this design is to define an <code>IPayment</code> interface, and three concrete classes that implement that interface.) </p> <p> This design trades better protection of invariants for violations of the <a href="https://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov Substitution Principle</a>. Clients will have to (attempt to) downcast to subtypes in order to access all relevant data (particularly <code>OriginalTransactionKey</code>). </p> <p> For completeness sake, I can think of at least one other option with significantly different trade-offs: applying the <a href="https://en.wikipedia.org/wiki/Visitor_pattern">Visitor design pattern</a>. This is, however, quite a complex solution, and most people will find the disadvantages greater than the benefits. </p> <p> Is it such a big deal, then? After all, it's only a single data value (<code>OriginalTransactionKey</code>) that may or may not be there. Surely, most programmers will be able to deal with that. </p> <p> This may be true in this isolated case, but keep in mind that this is only a motivating <em>example</em>. In many other situations, the domain you're trying to model is much more intricate, with many more exceptions to general rules. The more dimensions you add, the more difficult it becomes to reason about the code. </p> <p> <strong>F# model</strong> </p> <p> F#, on the other hand, makes dealing with such problems so simple that it's almost anticlimactic. The reason is that F#'s type system enables you to model <em>alternatives</em> of data, in addition to the combinations of data that C# (or Java) enables. Such alternatives are called <a href="https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/discriminated-unions">discriminated unions</a>. </p> <p> In the code base I'm currently developing, I model the various payment types like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PaymentService</span>&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;<span style="color:teal;">string</span>;&nbsp;Action&nbsp;:&nbsp;<span style="color:teal;">string</span>&nbsp;} <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PaymentType</span>&nbsp;= |&nbsp;<span style="color:navy;">Individual</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">PaymentService</span> |&nbsp;<span style="color:navy;">Parent</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">PaymentService</span> |&nbsp;<span style="color:navy;">Child</span>&nbsp;<span style="color:blue;">of</span>&nbsp;originalTransactionKey&nbsp;:&nbsp;<span style="color:teal;">string</span>&nbsp;*&nbsp;paymentService&nbsp;:&nbsp;<span style="color:teal;">PaymentService</span></pre> </p> <p> Here, <code>PaymentService</code> is a record type with some data about the payment (e.g. which credit card to use). </p> <p> Even if you're not used to reading F# code, you can see three alternatives outlined on each of the three lines of code that start with a vertical bar (<code>|</code>). The <code>PaymentType</code> type has <em>exactly</em> three 'subtypes' (they're called <em>cases</em>, though). The illegal state of a non-null <code>OriginalTransactionKey</code> combined with <code>StartRecurrent</code> value of <code>true</code> is not possible. It can't be compiled. </p> <p> Not only that, but all clients given a <code>PaymentType</code> value <em>must</em> deal with all three cases (or the compiler will issue a warning). Here's one example where our code is creating the JSON document to send to the payment service: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;name,&nbsp;action,&nbsp;startRecurrent,&nbsp;transaction&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;req.PaymentType&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Individual</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;name;&nbsp;Action&nbsp;=&nbsp;action&nbsp;}&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;name,&nbsp;action,&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Parent</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;name;&nbsp;Action&nbsp;=&nbsp;action&nbsp;}&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;name,&nbsp;action,&nbsp;<span style="color:blue;">true</span>,&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Child</span>&nbsp;(transactionKey,&nbsp;{&nbsp;Name&nbsp;=&nbsp;name;&nbsp;Action&nbsp;=&nbsp;action&nbsp;})&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;name,&nbsp;action,&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:navy;">Some</span>&nbsp;transactionKey</pre> </p> <p> This code example also extracts <code>name</code> and <code>action</code> from the <code>PaymentType</code> value, but the relevant values to be aware of are <code>startRecurrent</code> and <code>transaction</code>. <ul> <li>For an individual payment, <code>startRecurrent</code> becomes <code>false</code> and <code>transaction</code> becomes <code>None</code> (meaning that the value is missing).</li> <li>For a parent payment, <code>startRecurrent</code> becomes <code>true</code> and <code>transaction</code> becomes <code>None</code>.</li> <li>For a child payment, <code>startRecurrent</code> becomes <code>false</code> and <code>transaction</code> becomes <code>Some transactionKey</code>.</li> </ul> Notice that the (parent) <code>transactionKey</code> is only available when the payment is a child payment. </p> <p> The values <code>startRecurrent</code> and <code>transaction </code> (as well as <code>name</code> and <code>action</code>) are then used to create a JSON document. I'm not showing that part of the code here, since there's actually a lot going on in the real code base, and it's not related to how to model the domain. Imagine that these values are passed to a constructor. </p> <p> This is a real-world example that, I hope, demonstrates why I prefer F# over C# for domain modelling. The type system enables me to model alternatives as well as combinations of data, and thereby <em>making illegal states unrepresentable</em> - all in only a few lines of code. </p> <p> <strong>Summary</strong> </p> <p> Classes, in languages like C# and Java, enable you to model <em>combinations</em> of data. The more fields or properties you add to a class, the more combinations are possible. This is often useful, but sometimes you need to be able to model <em>alternatives</em>, rather than combinations. </p> <p> Some languages, like F#, Haskell, OCaml, Elm, Kotlin, and many others, have type systems that give you the power to model both combinations and alternatives. Such types systems are said to have <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>, but while the word sounds 'mathy', such types make it much easier to model complex domains. </p> <p> There are more reasons to love F# than only its algebraic data types, but this is the foremost reason I find it a better language for mainstream development work than C#. </p> <p> If you want to see a more complex example of modelling with types, a good next step would be <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">the first article</a> in my <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">Types + Properties = Software</a> article series. </p> <p> Finally, I should be careful that I don't oversell the idea of making illegal states unrepresentable. Algebraic data types give you an extra dimension in which you can model domains, but there are still rules that they can't enforce. As an example, you can't state that integers must only fall in a certain range (e.g. only positive integers allowed). There are other type systems, such as <a href="https://en.wikipedia.org/wiki/Dependent_type">dependent types</a>, that give you even more power to embed domain rules into types, but as far as I know, there are no type systems that can fully model all rules as types. You'll still have to write some code as well. </p> <p> The article is an instalment in the <a href="https://sergeytihon.wordpress.com/2016/10/23/f-advent-calendar-in-english-2016">2016 F# Advent calendar</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://lambdaklub.wordpress.com">Botond Balázs</a></div> <div class="comment-content"> <p>Mark,</p> <p>I must be missing something important but it seems to me that the only advantage of using F# in this case is that the match is enforced to be exhaustive by the compiler. And of course the syntax is also nicer than a bunch of if's. In all other respects, the solution is basically equivalent to the C# class hierarchy approach.</p> <p>Am I mistaken?</p> </div> <div class="comment-date">2016-12-03 08:38 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Botond, thank you for writing. The major advantage is that enumeration of all possible cases is available at compile-time. One derived advantage of that is that the compiler can check whether a piece of code handles all cases. That's already, in my experience, a big deal. The sooner you can get feedback on your work, the better, and <a href="http://blog.ploeh.dk/2011/04/29/Feedbackmechanismsandtradeoffs">it doesn't get faster than compile-time feedback</a>. </p> <p> Another advantage of having all cases encoded in the type system is that it gives you better tool support. Imagine that you're looking at the return value of a function, and that this is the first time you're encountering that return type. If the return value is an abstract base class (or interface), you'll need to resort to either the documentation or reflection in order to figure out which subtypes exist. There can be arbitrarily many subtypes, and they can be scattered over arbitrarily many libraries (assemblies). Figuring out what to do with an abstract base class introduces a context switch that could have been avoided. </p> <p> This is exactly another advantage offered by discriminated unions: when a function returns a discriminated union, you can immediately get tool support to figure out what to do with it, even if you've never encountered the type before. </p> <p> The problem with examples such as the above is that I'm trying to explain how a language feature can help you with modelling <em>complex</em> domains, but if I try to present a really complex problem, no-one will have the patience to read the article. Instead, I have to come up with an example that's so simple that the reader doesn't give up, and hopefully still complex enough that the reader can imagine how it's a stand-in for a more complex problem. </p> <p> When you look at the problem presented above, it's not <em>that</em> complex, so you can still keep a C# implementation in your head. As you add more variability to the problem, however, you can easily find yourself in a situation where the combinatorial explosion of possible values make it difficult to ensure that you've dealt with all edge cases. This is one of the main reasons that C# and Java code often throws run-time exceptions (particularly null-reference exceptions). </p> <p> It did, in fact, turn out that the above example domain became more complex as I learned more about the entire range of problems I had to solve. When I described the problem above, I thought that all payments would have pre-selected payment methods. In other words, when a user is presented with a check-out page, he or she selects the payment method (PayPal, direct debit, and so on), and only then, when we know payment method, do we initiate the payment flow. It turns out, though, that in some cases, we should start the payment flow first, and then let the user pick the payment method from a list of options. It should be noted, however, that user-selection only makes sense for interactive payments, so a <em>child</em> payment can never be user-selectable (since it's automated). </p> <p> It was trivial to extend the domain model with that new requirement: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PaymentService</span>&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;<span style="color:teal;">string</span>;&nbsp;Action&nbsp;:&nbsp;<span style="color:teal;">string</span>&nbsp;} <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PaymentMethod</span>&nbsp;= |&nbsp;<span style="color:navy;">PreSelected</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">PaymentService</span> |&nbsp;<span style="color:navy;">UserSelectable</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span>&nbsp;<span style="color:teal;">list</span> <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">TransactionKey</span>&nbsp;=&nbsp;<span style="color:navy;">TransactionKey</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span>&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">override</span>&nbsp;this.<span style="color:navy;">ToString</span>&nbsp;()&nbsp;=&nbsp;<span style="color:blue;">match</span>&nbsp;this&nbsp;<span style="color:blue;">with</span>&nbsp;<span style="color:navy;">TransactionKey</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;s <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PaymentType</span>&nbsp;= |&nbsp;<span style="color:navy;">Individual</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">PaymentMethod</span> |&nbsp;<span style="color:navy;">Parent</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">PaymentMethod</span> |&nbsp;<span style="color:navy;">Child</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">TransactionKey</span>&nbsp;*&nbsp;<span style="color:teal;">PaymentService</span></pre> </p> <p> This effectively uses the static type system to state that both the <code>Individual</code> and <code>Parent</code> cases can be defined in one of two ways: <code>PreSelected</code> or <code>UserSelectable</code>, each of which, again, contains heterogeneous data (<code>PaymentService</code> versus <code>string list</code>). Child payments, on the other hand, can't be user-selectable, but must be defined by a <code>PaymentService</code> value, as well as a transaction key (at this point, I'd also created a single-case union for the transaction key, but that's a different topic; it's still a string). </p> <p> Handling all the different combinations was equally easy, and the compiler guarantees that I've handled all possible combinations: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;services,&nbsp;selectables,&nbsp;startRecurrent,&nbsp;transaction&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;req.PaymentType&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Individual</span>&nbsp;(<span style="color:navy;">PreSelected</span>&nbsp;ps)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">service</span>&nbsp;ps,&nbsp;<span style="color:navy;">None</span>,&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Individual</span>&nbsp;(<span style="color:navy;">UserSelectable</span>&nbsp;us)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[||],&nbsp;us&nbsp;|&gt;&nbsp;<span style="color:teal;">String</span>.<span style="color:navy;">concat</span>&nbsp;<span style="color:#a31515;">&quot;,&nbsp;&quot;</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span>,&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Parent</span>&nbsp;(<span style="color:navy;">PreSelected</span>&nbsp;ps)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">service</span>&nbsp;ps,&nbsp;<span style="color:navy;">None</span>,&nbsp;<span style="color:blue;">true</span>,&nbsp;&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Parent</span>&nbsp;(<span style="color:navy;">UserSelectable</span>&nbsp;us)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[||],&nbsp;us&nbsp;|&gt;&nbsp;<span style="color:teal;">String</span>.<span style="color:navy;">concat</span>&nbsp;<span style="color:#a31515;">&quot;,&nbsp;&quot;</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span>,&nbsp;<span style="color:blue;">true</span>,&nbsp;&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Child</span>&nbsp;(<span style="color:navy;">TransactionKey</span>&nbsp;transactionKey,&nbsp;ps)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">service</span>&nbsp;ps,&nbsp;<span style="color:navy;">None</span>,&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:navy;">Some</span>&nbsp;transactionKey</pre> </p> <p> How would you handle this with a class hierarchy, and what would the consuming code look like? </p> </div> <div class="comment-date">2016-12-06 10:50 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. When variable names are in the way http://blog.ploeh.dk/2016/10/25/when-variable-names-are-in-the-way 2016-10-25T06:20:00+00:00 Mark Seemann <div id="post"> <p> <em>While Clean Code recommends using good variable names to communicate the intent of code, sometimes, variable names can be in the way.</em> </p> <p> Good guides to more readable code, like <a href="http://amzn.to/XCJi9X">Clean Code</a>, explain how explicitly introducing variables with descriptive names can make the code easier to understand. There's much literature on the subject, so I'm not going to reiterate it here. It's not the topic of this article. </p> <p> In the majority of cases, introducing a well-named variable will make the code more readable. There are, however, no rules without exceptions. After all, <a href="http://martinfowler.com/bliki/TwoHardThings.html">one of the hardest tasks in programming is naming things</a>. In this article, I'll show you an example of such an exception. While the example is slightly elaborate, it's a <em>real-world</em> example I recently ran into. </p> <p> <strong>Escaping object-orientation</strong> </p> <p> Regular readers of this blog will know that I write many <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">RESTful APIs</a> in F#, but using ASP.NET Web API. Since I like to write functional F#, but ASP.NET Web API is an object-oriented framework, I prefer to escape the object-oriented framework as soon as possible. (In general, it makes good architectural sense to write most of your code as framework-independent as possible.) </p> <p> ASP.NET Web API expects you handle HTTP requests using <em>Controllers</em>, so I use Constructor Injection to inject a function that will do all the actual work, and delegate each request to a function call. It often looks like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(<span style="color:#9b9b9b;">portalId</span>&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">PushRequestDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> This particular Controller only handles HTTP POST requests, and it does it by delegating to the injected <code>imp</code> function and translating the return value of that function call to various HTTP responses. This enables me to compose <code>imp</code> from F# functions, and thereby escape the object-oriented design of ASP.NET Web API. In other words, each Controller is an <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a> over a functional implementation. </p> <p> For good measure, though, this Controller implementation ought to be unit tested. </p> <p> <strong>A naive unit test attempt</strong> </p> <p> Each HTTP request is handled at the boundary of the system, and <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">the boundary of the system is always impure</a> - even in Haskell. This is particularly clear in the case of the above <code>PushController</code>, because it handles <code>Success ()</code>. In success cases, the result is <code>()</code> (<code>unit</code>), which strongly implies a side effect. Thus, a unit test ought to care not only about what <code>imp</code> returns, but also the input to the function. </p> <p> While you could write a unit test like the following, it'd be naive. </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;correct&nbsp;result&nbsp;on&nbsp;validation&nbsp;failure``</span>&nbsp;req&nbsp;(<span style="color:navy;">NonNull</span>&nbsp;msg)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;_&nbsp;=&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">fail</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;sut.<span style="color:navy;">Post</span>&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">convertsTo</span><span style="background:yellow;">&lt;</span><span style="background:yellow;">Results</span><span style="background:yellow;">.</span><span style="color:teal;background:yellow;">BadRequestErrorMessageResult</span><span style="background:yellow;">&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">.</span><span style="background:yellow;">Message</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;((</span><span style="background:yellow;">=</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">msg</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> This unit test uses <a href="https://fscheck.github.io/FsCheck">FsCheck</a>'s integration for <a href="https://xunit.github.io">xUnit.net</a>, and <a href="http://www.swensensoftware.com/unquote">Unquote</a> for assertions. Additionally, it uses a <code>convertsTo</code> function that I've <a href="http://blog.ploeh.dk/2014/03/21/composed-assertions-with-unquote">previously described</a>. </p> <p> The <code>imp</code> function for <code>PushController</code> must have the type <code>PushRequestDtr -&gt; Result&lt;unit, BoundaryFailure&gt;</code>. In the unit test, it uses a wild-card (<code>_</code>) for the input value, so its type is <code>'a -&gt; Result&lt;'b, BoundaryFailure&gt;</code>. That's a wider type, but it matches the required type, so the test compiles (and passes). </p> <p> FsCheck populates the <code>req</code> argument to the test function itself. This value is passed to <code>sut.Post</code>. If you look at the definition of <code>sut.Post</code>, you may wonder what happened to the individual (and currently unused) <code>portalId</code> argument. The explanation is that the <code>Post</code> method can be viewed as a method with two parameters, but it can also be viewed as an impure function that takes a <em>single</em> argument of the type <code>string * PushRequestDtr</code> - a tuple. In other words, the test function's <code>req</code> argument is a tuple. The test is not only concise, but also robust against refactorings. If you change the signature of the <code>Post</code> method, odds are that the test will still compile. (This is one of the many benefits of type inference.) </p> <p> The problem with the test is that it doesn't verify the data flow into <code>imp</code>, so this version of <code>PushController</code> <em>also</em> passes the test: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(<span style="color:#9b9b9b;">portalId</span>&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;<span style="color:#9b9b9b;">req</span>&nbsp;:&nbsp;<span style="color:teal;">PushRequestDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;minimalReq&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Transaction&nbsp;=&nbsp;{&nbsp;Invoice&nbsp;=&nbsp;<span style="color:#a31515;">&quot;&quot;</span>;&nbsp;Status&nbsp;=&nbsp;{&nbsp;Code&nbsp;=&nbsp;{&nbsp;Code&nbsp;=&nbsp;0&nbsp;}&nbsp;}&nbsp;}&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;minimalReq&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> As the name implies, the <code>minimalReq</code> value is the 'smallest' value I can create of the <code>PushRequestDtr</code> type. As you can see, this implementation ignores the <code>req</code> method argument and instead passes <code>minimalReq</code> to <code>imp</code>. This is clearly wrong, but it passes the unit test test. </p> <p> <strong>Data flow testing</strong> </p> <p> Not only should you care about the output of <code>imp</code>, but you should also care about the input. This is because <code>imp</code> is inherently impure, so it'd be conceivable that the input values matter in some way. </p> <p> As <a href="http://bit.ly/xunitpatterns">xUnit Test Patterns</a> explains, automated tests should contain no branching, so I don't think it's a good idea to define a test-specific <code>imp</code> function using conditionals. Instead, we can use guard assertions to verify that the input is as expected: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;correct&nbsp;result&nbsp;on&nbsp;validation&nbsp;failure``</span>&nbsp;req&nbsp;(<span style="color:navy;">NonNull</span>&nbsp;msg)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;candidate&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;=!&nbsp;<span style="color:navy;">snd</span>&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">fail</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;sut.<span style="color:navy;">Post</span>&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">convertsTo</span><span style="background:yellow;">&lt;</span><span style="background:yellow;">Results</span><span style="background:yellow;">.</span><span style="color:teal;background:yellow;">BadRequestErrorMessageResult</span><span style="background:yellow;">&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">.</span><span style="background:yellow;">Message</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;((</span><span style="background:yellow;">=</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">msg</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> The <code>imp</code> function is now implemented using Unquote's custom <code>=!</code> operator, which means that <code>candidate</code> must equal <code>req</code>. If not, Unquote will throw an exception, and thereby fail the test. </p> <p> If <code>candidate</code> is equal to <code>snd req</code>, the <code>=!</code> operator does nothing, enabling the <code>imp</code> function to return the value <code>Result.fail (ValidationFailure msg)</code>. </p> <p> This version of the test verifies the entire data flow through <code>imp</code>: both input and output. </p> <p> There is, however, a small disadvantage to writing the <code>imp</code> code this way. It isn't a big issue, but it annoys me. </p> <p> Here's the heart of the matter: I had to come up with a name for the local <code>PushRequestDtr</code> value that the <code>=!</code> operator evaluates against <code>snd req</code>. I chose to call it <code>candidate</code>, which may seem reasonable, but that naming strategy doesn't scale. </p> <p> In order to keep the introductory example simple, I chose a Controller method that doesn't (yet) use its <code>portalId</code> argument, but the code base contains other Controllers, for example this one: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">IdealController</span>&nbsp;(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:teal;">ApiController</span>&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>&nbsp;(portalId&nbsp;:&nbsp;<span style="color:teal;">string</span>,&nbsp;req&nbsp;:&nbsp;<span style="color:teal;">IDealRequestDtr</span>)&nbsp;:&nbsp;<span style="color:teal;">IHttpActionResult</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;portalId&nbsp;req&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;(resp&nbsp;:&nbsp;<span style="color:teal;">IDealResponseDtr</span>)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;resp&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;_ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">IntegrationFailure</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.<span style="color:navy;">InternalServerError</span>&nbsp;(<span style="color:teal;">InvalidOperationException</span>&nbsp;msg)&nbsp;:&gt;&nbsp;_</pre> </p> <p> This Controller's <code>Post</code> method passes <em>both</em> <code>portalId</code> and <code>req</code> to <code>imp</code>. In order to perform data flow verification of that implementation, the test has to look like this: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;correct&nbsp;result&nbsp;on&nbsp;success``</span>&nbsp;portalId&nbsp;req&nbsp;resp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;pid&nbsp;candidate&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;pid&nbsp;=!&nbsp;portalId &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;=!&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">succeed</span>&nbsp;resp &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">IdealController</span>&nbsp;(<span style="color:navy;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;sut.<span style="color:navy;">Post</span>&nbsp;(portalId,&nbsp;req) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">convertsTo</span><span style="background:yellow;">&lt;</span><span style="background:yellow;">Results</span><span style="background:yellow;">.</span><span style="color:teal;background:yellow;">OkNegotiatedContentResult</span><span style="background:yellow;">&lt;</span><span style="color:teal;background:yellow;">IDealResponseDtr</span><span style="background:yellow;">&gt;&gt;</span><span style="background:yellow;"></span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">.</span><span style="background:yellow;">Content</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;((</span><span style="background:yellow;">=</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">resp</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> This is where I began to run out of good argument names. You need names for the <code>portalId</code> and <code>req</code> arguments of <code>imp</code>, but you can't use those names because they're already in use. You can't even shadow the names of the outer values, because the test-specific <code>imp</code> function has to close over those outer values in order to compare them to their expected values. </p> <p> While I decided to call the local <em>portal ID</em> argument <code>pid</code>, it's hardly helpful. Explicit arguments have become a burden rather than a help to the reader. If only we could get rid of those explicit arguments. </p> <p> <strong>Point free</strong> </p> <p> Functional programming offers a well-known alternative to explicit arguments, commonly known as <a href="https://en.wikipedia.org/wiki/Tacit_programming">point-free programming</a>. Some people find point-free style unreadable, but sometimes it can make the code more readable. Could this be the case here? </p> <p> If you look at the test-specific <code>imp</code> functions in both of the above examples with explicit arguments, you may notice that they follow a common pattern. First they invoke one or more guard assertions, and then they return a value. You can model this with a custom operator: </p> <p> <pre><span style="color:green;">//&nbsp;&#39;Guard&#39;&nbsp;composition.&nbsp;Returns&nbsp;the&nbsp;return&nbsp;value&nbsp;if&nbsp;``assert``&nbsp;doesn&#39;t&nbsp;throw.</span> <span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;unit)&nbsp;-&gt;&nbsp;&#39;b&nbsp;-&gt;&nbsp;&#39;a&nbsp;-&gt;&nbsp;&#39;b</span> <span style="color:blue;">let</span>&nbsp;(&gt;&gt;!)&nbsp;<span style="color:navy;">``assert``</span>&nbsp;returnValue&nbsp;x&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">``assert``</span>&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;returnValue</pre> </p> <p> The first argument, <code>``assert``</code>, is a function with the type <code>'a -&gt; unit</code>. This is the assertion function: it takes any value as input, and returns <code>unit</code>. The implication is that it'll throw an exception if the assertion fails. </p> <p> After invoking the assertion, the function returns the <code>returnValue</code> argument. </p> <p> The reason I designed it that way is that it's <em>composable</em>, which you'll see in a minute. The reason I named it <code>&gt;&gt;!</code> was that I wanted some kind of arrow, and I thought that the exclamation mark relates nicely to Unquote's use of exclamation marks. </p> <p> This enables you to compose the first <code>imp</code> example (for <code>PushController</code>) in point-free style: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;correct&nbsp;result&nbsp;on&nbsp;validation&nbsp;failure``</span>&nbsp;req&nbsp;(<span style="color:navy;">NonNull</span>&nbsp;msg)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;=&nbsp;((=!)&nbsp;(<span style="color:navy;">snd</span>&nbsp;req))&nbsp;&gt;&gt;!&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">fail</span>&nbsp;(<span style="color:navy;">ValidationFailure</span>&nbsp;msg) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">PushController</span>&nbsp;(<span style="color:navy;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;sut.<span style="color:navy;">Post</span>&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">convertsTo</span><span style="background:yellow;">&lt;</span><span style="background:yellow;">Results</span><span style="background:yellow;">.</span><span style="color:teal;background:yellow;">BadRequestErrorMessageResult</span><span style="background:yellow;">&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">.</span><span style="background:yellow;">Message</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;((</span><span style="background:yellow;">=</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">msg</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> At first glance, most people would be likely to consider this to be <em>less</em> readable than before, and clearly, that's a valid standpoint. On the other hand, once you get used to identify the <code>&gt;&gt;!</code> operator, this becomes a concise shorthand. A data-flow-verifying <code>imp</code> mock function is composed of an assertion on the left-hand side of <code>&gt;&gt;!</code>, and a return value on the right-hand side. </p> <p> Most importantly, those hard-to-name arguments are gone. </p> <p> Still, let's dissect the expression <code>((=!) (snd req)) &gt;&gt;! Result.fail (ValidationFailure msg)</code>. </p> <p> The expression on the left-hand side of the <code>&gt;&gt;!</code> operator is the assertion. It uses Unquote's <em>must equal</em> <code>=!</code> operator as a function. (In F#, infix operators are functions, and you can use them as functions by surrounding them by brackets.) While you can write an assertion as <code>candidate =! snd req</code> using infix notation, you can also write the same expression as a function call: <code>(=!) (snd req) candidate</code>. Since this is a function, it can be partially applied: <code>(=!) (snd req)</code>; the type of that expression is <code>PushRequestDtr -&gt; unit</code>, which matches the required type <code>'a -&gt; unit</code> that <code>&gt;&gt;!</code> expects from its <code>``assert``</code> argument. That explains the left-hand side of the <code>&gt;&gt;!</code> operator. </p> <p> The right-hand side is easier, because that's the return value of the composed function. In this case the value is <code>Result.fail (ValidationFailure msg)</code>. </p> <p> You already know that the type of <code>&gt;&gt;!</code> is <code>('a -&gt; unit) -&gt; 'b -&gt; 'a -&gt; 'b</code>. Replacing the generic type arguments with the actual types in use, <code>'a</code> is <code>PushRequestDtr</code> and <code>'b</code> is <code>Result&lt;'a ,BoundaryFailure&gt;</code>, so the type of <code>imp</code> is <code>PushRequestDtr -&gt; Result&lt;'a ,BoundaryFailure&gt;</code>. When you set <code>'a</code> to <code>unit</code>, this fits the required type of <code>PushRequestDtr -&gt; Result&lt;unit, BoundaryFailure&gt;</code>. </p> <p> This works because in its current incarnation, the <code>imp</code> function for <code>PushController</code> only takes a single value as input. Will this also work for <code>IdealController</code>, which passes both <code>portalId</code> and <code>req</code> to its <code>imp</code> function? </p> <p> <strong>Currying</strong> </p> <p> The <code>imp</code> function for <code>IdealController</code> has the type <code>string -&gt; IDealRequestDtr -&gt; Result&lt;IDealResponseDtr, BoundaryFailure&gt;</code>. Notice that it takes two arguments instead of one. Is it possible to compose an <code>imp</code> function with the <code>&gt;&gt;!</code> operator? </p> <p> Consider the above example that exercises the success case for <code>IdealController</code>. What if, instead of writing </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;pid&nbsp;candidate&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;pid&nbsp;=!&nbsp;portalId &nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;=!&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">succeed</span>&nbsp;resp</pre> </p> <p> you write the following? </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;=&nbsp;((=!)&nbsp;req)&nbsp;&gt;&gt;!&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">succeed</span>&nbsp;resp </pre> </p> <p> Unfortunately, that does work, because the type of that function is <code>string * IDealRequestDtr -&gt; Result&lt;IDealResponseDtr, 'a&gt;</code>, and not <code>string -&gt; IDealRequestDtr -&gt; Result&lt;IDealResponseDtr, BoundaryFailure&gt;</code>, as it should be. It's almost there, but the input values are tupled, instead of curried. </p> <p> You can easily correct that with a standard <a href="https://gist.github.com/ploeh/6d8050e121a5175fabb1d08ef5266cd7">curry function</a>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;=&nbsp;((=!)&nbsp;req)&nbsp;&gt;&gt;!&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">succeed</span>&nbsp;resp&nbsp;|&gt;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span> </pre> </p> <p> The <code>Tuple2.curry</code> function takes as input a function that has tupled arguments, and turns it into a curried function. Exactly what we need here! </p> <p> The entire test is now: </p> <p> <pre>[&lt;<span style="color:teal;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;correct&nbsp;result&nbsp;on&nbsp;success``</span>&nbsp;req&nbsp;resp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;=&nbsp;((=!)&nbsp;req)&nbsp;&gt;&gt;!&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">succeed</span>&nbsp;resp&nbsp;|&gt;&nbsp;<span style="color:teal;">Tuple2</span>.<span style="color:navy;">curry</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:teal;">IdealController</span>&nbsp;(<span style="color:navy;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;sut.<span style="color:navy;">Post</span>&nbsp;req &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">convertsTo</span><span style="background:yellow;">&lt;</span><span style="background:yellow;">Results</span><span style="background:yellow;">.</span><span style="color:teal;background:yellow;">OkNegotiatedContentResult</span><span style="background:yellow;">&lt;</span><span style="color:teal;background:yellow;">IDealResponseDtr</span><span style="background:yellow;">&gt;&gt;</span><span style="background:yellow;"></span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">r</span><span style="background:yellow;">.</span><span style="background:yellow;">Content</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:teal;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;((</span><span style="background:yellow;">=</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">resp</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> Whether or not you find this more readable than the previous example is, as always, subjective, but I like it because it's a succinct, composable way to address data flow verification. Once you get over the initial shock of partially applying Unquote's <code>=!</code> operator, as well as the cryptic-looking <code>&gt;&gt;!</code> operator, you may begin to realise that the same idiom is repeated throughout. In fact, it's more than an idiom. It's an implementation of a design pattern. </p> <p> <strong>Mocks</strong> </p> <p> When talking about unit testing, I prefer the vocabulary of <a href="http://bit.ly/xunitpatterns">xUnit Test Patterns</a>, because of its unrivalled consistent terminology. Using Gerard Meszaros' nomenclature, a <a href="http://martinfowler.com/bliki/TestDouble.html">Test Double</a> with built-in verification of interaction is called a <em>Mock</em>. </p> <p> Most people (including me) dislike Mocks because they tend to lead to brittle unit tests. They <em>tend</em> to, but <a href="http://blog.ploeh.dk/2013/10/23/mocks-for-commands-stubs-for-queries">sometimes you need them</a>. Mocks are useful when you care about side-effects. </p> <p> Functional programming emphasises <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>, which, by definition, are free of side-effects. <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">In pure functional programming, you don't need Mocks</a>. </p> <p> Since F# is a multi-paradigmatic language, you sometimes have to write code in a more object-oriented style. In the example you've seen here, I've shown you how to unit test that Controllers correctly work as Adapters over (impure) functions. Here, Mocks are useful, even if they have no place in the rest of the code base. </p> <p> Being able to express a Mock with a couple of minimal functions is, in my opinion, preferable to adding a big dependency to a 'mocking library'. </p> <p> <strong>Concluding remarks</strong> </p> <p> Sometimes, explicit values and arguments are in the way. By their presence, they force you to name them. Often, naming is good, because it compels you to make tacit knowledge explicit. In rare cases, though, the important detail isn't a value, or an argument, but instead an <em>activity</em>. An example of this is when verifying data flow. While the values are obviously present, the focus ought to be on the comparison. Thus, by making the local function arguments implicit, you can direct the reader's attention to the interaction - in this case, Unquote's <code>=!</code> <em>must equal</em> comparison. </p> <p> In the introduction to this article, I told you that the code you've seen here is a real-life example. This is true. </p> <p> I submitted my refactoring to point-free style as an internal pull request on the project I'm currently working. When I did that, I was genuinely in doubt about the readability improvement this would give, so I asked my reviewers for their opinions. I was genuinely ready to accept if they wanted to reject the pull request. </p> <p> My reviewers disagreed internally, ultimately had a vote, and decided to reject the pull request. I don't blame them. We had a civil discussion about the pros and cons, and while they understood the advantages, they felt that the disadvantages weighed heavier. </p> <p> In their context, I understand why they decided to decline the change, but that doesn't mean that I don't find this an interesting experiment. I expect to use something like this in the future in some contexts, while in other contexts, I'll stick with the more verbose (and harder to name) test-specific functions with explicit arguments. </p> <p> Still, I like to solve problems using well-known compositions, which is the reason I prefer a composable, idiomatic approach over ad-hoc code. </p> <p> If you'd like to learn more about unit testing and property-based testing in F# (and C#), you can watch some of <a href="http://bit.ly/ploehralsight">my Pluralsight courses</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Decoupling decisions from effects http://blog.ploeh.dk/2016/09/26/decoupling-decisions-from-effects 2016-09-26T21:51:00+00:00 Mark Seemann <div id="post"> <p> <em>Functional programming emphasises pure functions, but sometimes decisions must be made based on impure data. The solution is to decouple decisions and effects.</em> </p> <p> Functional programmers love <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. Not only do they tend to be easy to reason about, they are also <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. It'd be wonderful if we could build entire systems only from pure functions, but every functional programmer knows that the world is impure. Instead, we strive towards implementing as much of our code base as pure functions, so that an application is <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">impure only at its boundaries</a>. </p> <p> The more you can do this, the more testable the system becomes. One rule of thumb about unit testing that I often use is that if a particular candidate for unit testing has a <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a> of 1, it may be acceptable to skip unit testing it. Instead, we can consider such a unit a <a href="http://xunitpatterns.com/Humble%20Object.html">humble unit</a>. If you can separate decisions from <em>effects</em> (which is what functional programmers often call impurities), you can often make the impure functions humble. </p> <p> In other words: put all logic in pure functions that can be unit tested, and implement impure effects as humble functions that you don't need to unit test. </p> <p> You want to see an example. So do I! </p> <p> <strong>Example: conditional reading from file</strong> </p> <p> In a <a href="https://gist.github.com/jcansdale/de58c2b2c389851f72b00ef525be820c">recent discussion</a>, Jamie Cansdale asks how I'd design and unit test something like the following C# method if I could instead redesign it in F#. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;GetUpperText(<span style="color:blue;">string</span>&nbsp;path) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:#2b91af;">File</span>.Exists(path))&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;DEFAULT&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;text&nbsp;=&nbsp;<span style="color:#2b91af;">File</span>.ReadAllText(path); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;text.ToUpperInvariant(); }</pre> </p> <p> Notice how this method contains two impure operations: <code>File.Exists</code> and <code>File.ReadAllText</code>. Decision logic seems interleaved with IO. How can decisions be separated from effects? </p> <p> (For good measure I ought to point out that obviously, the above example is so simple that by itself, it almost doesn't warrant testing. Think of it as a stand-in for a more complex problem.) </p> <p> With a statement-based language like C#, it can be difficult to see how to separate decision logic from effects without introducing interfaces, but with expression-based languages like F#, it becomes close to trivial. In this article, I'll show you three alternatives. </p> <p> All three alternatives, however, make use of the same function for turning text into upper case: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;string</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getUpper</span>&nbsp;(text&nbsp;:&nbsp;<span style="color:teal;">string</span>)&nbsp;=&nbsp;text.<span style="color:navy;">ToUpperInvariant</span>&nbsp;()</pre> </p> <p> Obviously, this function is so trivial that it's hardly worth testing, but remember to think about it as a stand-in for a more complex problem. It's a pure function, so it's easy to unit test: </p> <p> <pre>[&lt;<span style="color:teal;">Theory</span>&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;foo&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;FOO&quot;</span>)&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;bar&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;BAR&quot;</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``getUpper&nbsp;returns&nbsp;correct&nbsp;value``</span>&nbsp;input&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">getUpper</span>&nbsp;input &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> This test uses <a href="https://xunit.github.io">xUnit.net 2.1.0</a> and <a href="http://www.swensensoftware.com/unquote">Unquote 3.1.2</a>. The <code>=!</code> operator is a custom Unquote operator; you can read it as <em>must equal</em>; that is: <em>expected must equal actual</em>. It'll throw an exception if this isn't the case. </p> <p> <strong>Custom unions</strong> </p> <p> Languages like F# come with <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>, which means that in addition to complex structures, they also enable you to express types as <em>alternatives</em>. This means that you can represent a decision as one or more alternative <em>pure</em> values. </p> <p> Although the examples you'll see later in this article are simpler, I think it'll be helpful to start with an ad hoc solution to the problem. Here, the decision is to either read from a file, or return a default value. You can express that using a custom discriminated union: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:teal;">Action</span>&nbsp;=&nbsp;<span style="color:navy;">ReadFromFile</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span>&nbsp;|&nbsp;<span style="color:navy;">UseDefault</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:teal;">string</span> </pre> </p> <p> This type models two mutually exclusive cases: either you decide to read from the file identified by a file path (<code>string</code>), or your return a default value (also modelled as a <code>string</code>). </p> <p> Using this <code>Action</code> type, you can write a <em>pure</em> function that makes the decision: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;bool&nbsp;-&gt;&nbsp;Action</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">decide</span>&nbsp;path&nbsp;fileExists&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;fileExists &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">ReadFromFile</span>&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">UseDefault</span>&nbsp;<span style="color:#a31515;">&quot;DEFAULT&quot;</span></pre> </p> <p> This function takes two arguments: <code>path</code> (a <code>string</code>) and <code>fileExists</code> (a <code>bool</code>). If <code>fileExists</code> is <code>true</code>, it returns the <code>ReadFromFile</code> case; otherwise, it returns the <code>UseDefault</code> case. </p> <p> Notice how this function neither checks whether the file exists, nor does it attempt to read the contents of the file. It only makes a decision based on input, and returns information about this decision as output. This function is <em>pure</em>, so (as I've claimed numerous times) is easy to unit test: </p> <p> <pre>[&lt;<span style="color:teal;">Theory</span>&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;ploeh.txt&quot;</span>)&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;fnaah.txt&quot;</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``decide&nbsp;returns&nbsp;correct&nbsp;result&nbsp;when&nbsp;file&nbsp;exists``</span>&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">decide</span>&nbsp;path&nbsp;<span style="color:blue;">true</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">ReadFromFile</span>&nbsp;path&nbsp;=!&nbsp;actual [&lt;<span style="color:teal;">Theory</span>&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;ploeh.txt&quot;</span>)&gt;] [&lt;<span style="color:teal;">InlineData</span>(<span style="color:#a31515;">&quot;fnaah.txt&quot;</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``decide&nbsp;returns&nbsp;correct&nbsp;result&nbsp;when&nbsp;file&nbsp;doesn&#39;t&nbsp;exist``</span>&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">decide</span>&nbsp;path&nbsp;<span style="color:blue;">false</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">UseDefault</span>&nbsp;<span style="color:#a31515;">&quot;DEFAULT&quot;</span>&nbsp;=!&nbsp;actual</pre> </p> <p> One unit test function exercises the code path where the file exists, whereas the other test exercises the code path where it doesn't. Straightforward. </p> <p> There's still some remaining work, because you need to somehow compose your pure functions with <code>File.Exists</code> and <code>File.ReadAllText</code>. You also need a way to extract the string value from the two cases of <code>Action</code>. One way to do that is to introduce another pure function: </p> <p> <pre><span style="color:green;">//&nbsp;(string&nbsp;-&gt;&nbsp;string)&nbsp;-&gt;&nbsp;Action&nbsp;-&gt;&nbsp;string</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getValue</span>&nbsp;<span style="color:navy;">f</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">ReadFromFile</span>&nbsp;path&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">f</span>&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">UseDefault</span>&nbsp;value&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;value</pre> </p> <p> This is a function that returns the <code>UseDefault</code> data for that case, but invokes a function <code>f</code> in the <code>ReadFromFile</code> case. Again, since this function is pure it's easy to unit test it, but I'll leave that as an exercise. </p> <p> You now have all the building blocks required to compose a function similar to the above <code>GetUpperText</code> C# method: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;string</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getUpperText</span>&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">File</span>.<span style="color:navy;">Exists</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">decide</span>&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">getValue</span>&nbsp;(<span style="color:teal;">File</span>.<span style="color:navy;">ReadAllText</span>&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">getUpper</span>)</pre> </p> <p> This implementation pipes <code>path</code> into <code>File.Exists</code>, which returns a Boolean value indicating whether the file exists. This Boolean value is then piped into <code>decide path</code>, which (as you may recall) returns an <code>Action</code>. That value is finally piped into <code>getValue (File.ReadAllText &gt;&gt; getUpper)</code>. Recall that <code>getValue</code> will only invoke the function argument when the <code>Action</code> is <code>ReadFromFile</code>, so <code>File.ReadAllText &gt;&gt; getUpper</code> is only executed in this case. </p> <p> Notice how decisions and effectful functions are interleaved. All the decision functions are covered by unit tests; only <code>File.Exists</code> and <code>File.ReadAllText</code> aren't covered, but I find it reasonable to treat these as humble functions. </p> <p> <strong>Either</strong> </p> <p> Normally, decisions often involve a choice between two alternatives. In the above example, you saw how the alternatives were named <code>ReadFromFile</code> and <code>UseDefault</code>. Since a choice between two alternatives is so common, there's a well-known 'pattern' that gives you general-purpose tools to model decisions. This is known as the <em>Either</em> monad. </p> <p> The F# core library doesn't (<a href="https://github.com/fsharp/FSharpLangDesign/blob/master/RFCs/FS-1004-result-type.md">yet</a>) come with an implementation of the Either monad, but it's easy to add. In this example, I'm using code from <a href="https://fsharpforfunandprofit.com">Scott Wlaschin's</a> <a href="https://fsharpforfunandprofit.com/posts/recipe-part2">railway-oriented programming</a>, although slightly modified, and including only the most essential building blocks for the example: </p> <p> <pre> <span style="color:blue;">type</span>&nbsp;<span style="color:teal;">Result<'TSuccess, 'TFailure></span>&nbsp;= | Success <span style="color:blue;">of</span> 'TSuccess | Failure <span style="color:blue;">of</span> 'TFailure <span style="color:blue;">module</span>&nbsp;<span style="color:teal;">Result</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&nbsp;&#39;c&gt;)&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&nbsp;&#39;c&gt;&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&nbsp;&#39;c&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">bind</span>&nbsp;<span style="color:navy;">f</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;succ&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">f</span>&nbsp;succ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&nbsp;&#39;c&gt;&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&nbsp;&#39;c&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">f</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;succ&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;(<span style="color:navy;">f</span>&nbsp;succ) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;bool)&nbsp;-&gt;&nbsp;&#39;a&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&nbsp;&#39;a&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">split</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;=&nbsp;<span style="color:blue;">if</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;x&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;(&#39;c&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&nbsp;&#39;c&gt;&nbsp;-&gt;&nbsp;&#39;b</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">either</span>&nbsp;<span style="color:navy;">f</span>&nbsp;<span style="color:navy;">g</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;succ&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">f</span>&nbsp;succ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;fail&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">g</span>&nbsp;fail</pre> </p> <p> In fact, the <code>bind</code> and <code>map</code> functions aren't even required for this particular example, but I included them anyway, because otherwise, readers already familiar with the Either monad would wonder why they weren't there. </p> <p> All these functions are generic and pure, so they are easy to unit test. I'm not going to show you the unit tests, however, as I consider the functions belonging to that <code>Result</code> module as reusable functions. This is a module that would ship as part of a well-tested library. In fact, <a href="https://github.com/fsharp/FSharpLangDesign/blob/master/RFCs/FS-1004-result-type.md">it'll soon be added to the F# core library</a>. </p> <p> With the already tested <code>getUpper</code> function, you now have all the building blocks required to implement the desired functionality: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;string</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getUpperText</span>&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">split</span>&nbsp;<span style="color:teal;">File</span>.<span style="color:navy;">Exists</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Result</span>.<span style="color:navy;">either</span>&nbsp;(<span style="color:teal;">File</span>.<span style="color:navy;">ReadAllText</span>&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">getUpper</span>)&nbsp;(<span style="color:blue;">fun</span>&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#a31515;">&quot;DEFAULT&quot;</span>)</pre> </p> <p> This composition pipes <code>path</code> into <code>Result.split</code>, which uses <code>File.Exists</code> as a predicate to decide whether the path should be packaged into a <code>Success</code> or <code>Failure</code> case. The resulting <code>Result&lt;string, string&gt;</code> is then piped into <code>Result.either</code>, which invokes <code>File.ReadAllText &gt;&gt; getUpper</code> in the <code>Success</code> case, and the anonymous function in the <code>Failure</code> case. </p> <p> Notice how, once again, the impure functions <code>File.Exists</code> and <code>File.ReadAllText</code> are used as humble functions, but interleaved with testable, pure functions that make all the decisions. </p> <p> <strong>Maybe</strong> </p> <p> Sometimes, a decision isn't so much between two alternatives as it's a decision between something that may exist, but also may not. You can model this with the <em>Maybe</em> monad, which in F# comes in the form of the built-in <code>option</code> type. </p> <p> In fact, so much is already built in (and tested by the F# development team) that you almost don't need to add anything yourself. The only function you could consider adding is this: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:teal;">Option</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;&#39;a&nbsp;-&gt;&nbsp;&#39;a&nbsp;option&nbsp;-&gt;&nbsp;&#39;a</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">defaultIfNone</span>&nbsp;def&nbsp;x&nbsp;=&nbsp;<span style="color:navy;">defaultArg</span>&nbsp;x&nbsp;def</pre> </p> <p> As you can see, this function simply swaps the arguments for the built-in <code>defaultArg</code> function. This is done to make it more pipe-friendly. This function <a href="https://github.com/fsharp/FSharpLangDesign/blob/master/RFCs/FS-1007-additional-Option-module-functions.md">will most likely be added in a future version of F#</a>. </p> <p> That's all you need: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;string</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getUpperText</span>&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Option</span>.<span style="color:navy;">filter</span>&nbsp;<span style="color:teal;">File</span>.<span style="color:navy;">Exists</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Option</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:teal;">File</span>.<span style="color:navy;">ReadAllText</span>&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">getUpper</span>) &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:teal;">Option</span>.<span style="color:navy;">defaultIfNone</span>&nbsp;<span style="color:#a31515;">&quot;DEFAULT&quot;</span></pre> </p> <p> This composition starts with the <code>path</code>, puts it into a <code>Some</code> case, and pipes that <code>option</code> value into <code>Option.filter File.Exists</code>. This means that the <code>Some</code> case will only stay a <code>Some</code> value if the file exists; otherwise, it will be converted to a <code>None</code> value. Whatever the <code>option</code> value is, it's then piped into <code>Option.map (File.ReadAllText &gt;&gt; getUpper)</code>. The composed function <code>File.ReadAllText &gt;&gt; getUpper</code> is only executed in the <code>Some</code> case, so if the file doesn't exist, the function will not attempt to read it. Finally, the <code>option</code> value is piped into <code>Option.defaultIfNone</code>, which returns the mapped value, or <code>"DEFAULT"</code> if the value was <code>None</code>. </p> <p> Like in the two previous examples, the decision logic is implemented by pure functions, whereas the impure functions <code>File.Exists</code> and <code>File.ReadAllText</code> are handled as humble functions. </p> <p> <strong>Summary</strong> </p> <p> Have you noticed a pattern in all the three examples? Decisions are separated from effects using discriminated unions (both the above <code>Action</code>, <code>Result&lt;'TSuccess, 'TFailure&gt;</code>, and the built-in <code>option</code> are discriminated unions). In my experience, as long as you need to decide between two alternatives, the Either or Maybe monads are often sufficient to decouple decision logic from effects. Often, I don't even need to write any tests, because I compose my functions from the known, well-tested functions that belong to the respective monads. </p> <p> If your decision has to branch between three or more alternatives, you can consider a custom discriminated union. For this particular example, though, I think I prefer the third, Maybe-based composition, but closely followed by the Either-based composition. </p> <p> In this article, you saw three examples of how to decouple decision from effects; and I didn't even show you the <em>Free</em> monad! </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://github.com/blainne">Grzegorz Sławecki</a></div> <div class="comment-content"> <p> Mark, </p> <p> I can't understand how can the <code>getValue</code> function be pure. While I agree that it's easy to test, it's still the higher order function and it's purity depends on the purity of function passed as the argument. Even in Your example it takes <code>File.ReadAllText&nbsp;&gt;&gt;&nbsp;getUpper</code> which actually reaches to a file on the disk and I perceive it as reaching to an external shared state. Is there something I misunderstood? </p> </div> <div class="comment-date">2016-10-14 09:06 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Grzegorz, thank you for writing. You make a good point, and in a sense you're correct. F# doesn't enforce purity, and this is both an advantage and a disadvantage. It's an advantage because it makes it easier for programmers migrating from C# to make a gradual transition to a more functional programming style. It's also an advantage exactly because it relies on the programmer's often-faulty reasoning to ensure that code is properly functional. </p> <p> Functions in F# are only pure if they're implemented to be pure. For any given function type (signature) you can always create an impure function that fits the type. (If nothing else, you can always write "Hello, world!" to the console, before returning a value.) </p> <p> The result of this is that few parts of F# are pure in the sense that you imply. Even <code>List.map</code> could be impure, if passed an impure function. In other words, higher-order functions in F# are only pure if composed of exclusively pure parts. </p> <p> Clearly, this is in stark contrast to Haskell, where purity is enforced at the type level. In Haskell, a throw-away, poorly designed mini-API like the <code>Action</code> type and associated functions shown here wouldn't even compile. The Either and Maybe examples, on the other hand, would. </p> <p> My assumption here is that function composition happens at the edge of the application - that is, in an impure (<code>IO</code>) context. </p> </div> <div class="comment-date">2016-10-15 09:02 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Untyped F# HTTP route defaults for ASP.NET Web API http://blog.ploeh.dk/2016/08/09/untyped-f-http-route-defaults-for-aspnet-web-api 2016-08-09T04:24:00+00:00 Mark Seemann <div id="post"> <p> <em>In ASP.NET Web API, route defaults can be provided by a dictionary in F#.</em> </p> <p> When you define a route in ASP.NET Web API 2, you most likely use the <a href="https://msdn.microsoft.com/en-us/library/system.web.http.httproutecollectionextensions.maphttproute">MapHttpRoute</a> overload where you have to supply default values for the route template: </p> <p> <pre><span style="color:blue;">public static</span> IHttpRoute MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span> HttpRouteCollection routes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span> name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span> routeTemplate, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">object</span> defaults)</pre> </p> <p> The <code>defaults</code> arguments has the type <code>object</code>, but while the compiler will allow you to put any value here, the implicit <em>intent</em> is that in C#, you should pass an anonymous object with the route defaults. A standard route looks like this: </p> <p> <pre>configuration.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;DefaultAPI&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{controller}/{id}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Home&quot;</span>,&nbsp;Id&nbsp;=&nbsp;<span style="color:#2b91af;">RouteParameter</span>.Optional&nbsp;});</pre> </p> <p> Notice how the name of the properties (<code>Controller</code> and <code>Id</code>) (case-insensitively) match the place-holders in the route template (<code>{controller}</code> and <code>{id}</code>). </p> <p> While it's not clear from the type of the argument that this is what you're supposed to do, once you've learned it, it's easy enough to do, and rarely causes problems in C#. </p> <p> <strong>Flexibility</strong> </p> <p> You can debate the soundness of this API design, but as far as I can tell, it attempts to strike a balance between flexibility and syntax easy on the eyes. It does, for example, enable you to define a list of routes like this: </p> <p> <pre>configuration.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityYear&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>&nbsp;}); configuration.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityMonth&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}/{month}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>&nbsp;}); configuration.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityDay&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}/{month}/{day}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>&nbsp;}); configuration.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;DefaultAPI&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{controller}/{id}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Home&quot;</span>,&nbsp;Id&nbsp;=&nbsp;<span style="color:#2b91af;">RouteParameter</span>.Optional&nbsp;});</pre> </p> <p> In this example, there are three alternative routes to an <em>availability</em> resource, keyed on either an entire year, a month, or a single date. Since the route <em>templates</em> (e.g. <code>availability/{year}/{month}</code>) don't specify an <code>id</code> place-holder, there's no reason to provide a default value for it. On the other hand, it would have been possible to define defaults for the custom place-holders <code>year</code>, <code>month</code>, or <code>day</code>, if you had so desired. In this example, however, there are no defaults for these place-holders, so if values aren't provided, none of the <em>availability</em> routes are matched, and the request falls through to the <code>DefaultAPI</code> route. </p> <p> Since you can supply an anonymous object in C#, you can give it any property you'd like, and the code will still compile. There's no type safety involved, but using an anonymous object enables you to use a compact syntax. </p> <p> <strong>Route defaults in F#</strong> </p> <p> The API design of the MapHttpRoute method seems forged with C# in mind. I don't know how it works in Visual Basic .NET, but in F# there are no anonymous objects. How do you supply route defaults, then? </p> <p> As I described in my article on <a href="http://blog.ploeh.dk/2013/08/23/how-to-create-a-pure-f-aspnet-web-api-project">creating an F# Web API project</a>, you can define a record type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">HttpRouteDefaults</span>&nbsp;=&nbsp;{&nbsp;Controller&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>;&nbsp;Id&nbsp;:&nbsp;<span style="color:#4ec9b0;">obj</span>&nbsp;} </pre> </p> <p> You can use it like this: </p> <p> <pre><span style="color:#4ec9b0;">GlobalConfiguration</span>.Configuration.Routes.<span style="color:navy;">MapHttpRoute</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;DefaultAPI&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{controller}/{id}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Controller&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Home&quot;</span>;&nbsp;Id&nbsp;=&nbsp;<span style="color:#4ec9b0;">RouteParameter</span>.Optional&nbsp;})&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span></pre> </p> <p> That works fine for <code>DefaultAPI</code>, but it's hardly flexible. You <em>must</em> supply both a <code>Controller</code> and a <code>Id</code> value. If you need to define routes like the <em>availability</em> routes above, you can't use this HttpRouteDefaults type, because you can't omit the <code>Id</code> value. </p> <p> While defining <em>another</em> record type is only a one-liner, you're confronted with the problem of naming these types. </p> <p> In C#, the use of anonymous objects is, despite appearances, an untyped approach. Could something similar be possible with F#? </p> <p> It turns out that the MapHttpRoute also works if you pass it an <code>IDictionary&lt;string, object&gt;</code>, which is possible in F#: </p> <p> <pre>config.Routes.MapHttpRoute( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;DefaultAPI&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{controller}/{id}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;dict&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#a31515;">&quot;Controller&quot;</span>,&nbsp;box&nbsp;<span style="color:#a31515;">&quot;Home&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#a31515;">&quot;Id&quot;</span>,&nbsp;box&nbsp;RouteParameter.Optional)])&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span></pre> </p> <p> While this looks more verbose than the previous alternative, it's more flexible. It's also <a href="http://c2.com/cgi/wiki?StringlyTyped">stringly typed</a>, which normally isn't an endorsement, but in this case is <em>honest</em>, because it's as strongly typed as the MapHttpRoute method. <a href="https://www.python.org/dev/peps/pep-0020">Explicit is better than implicit</a>. </p> <p> The complete route configuration corresponding to the above example would look like this: </p> <p> <pre>config.Routes.<span style="color:navy;">MapHttpRoute</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityYear&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">dict</span>&nbsp;[(<span style="color:#a31515;">&quot;Controller&quot;</span>,&nbsp;<span style="color:navy;">box</span>&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>)])&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span> config.Routes.<span style="color:navy;">MapHttpRoute</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityMonth&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}/{month}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">dict</span>&nbsp;[(<span style="color:#a31515;">&quot;Controller&quot;</span>,&nbsp;<span style="color:navy;">box</span>&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>)])&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span> config.Routes.<span style="color:navy;">MapHttpRoute</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;AvailabilityDay&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;availability/{year}/{month}/{day}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">dict</span>&nbsp;[(<span style="color:#a31515;">&quot;Controller&quot;</span>,&nbsp;<span style="color:navy;">box</span>&nbsp;<span style="color:#a31515;">&quot;Availability&quot;</span>)])&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span> config.Routes.<span style="color:navy;">MapHttpRoute</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;DefaultAPI&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{controller}/{id}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">dict</span>&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#a31515;">&quot;Controller&quot;</span>,&nbsp;<span style="color:navy;">box</span>&nbsp;<span style="color:#a31515;">&quot;Home&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#a31515;">&quot;Id&quot;</span>,&nbsp;<span style="color:navy;">box</span>&nbsp;<span style="color:teal;">RouteParameter</span>.Optional)])&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span></pre> </p> <p> If you're interested in learning more about developing ASP.NET Web API services in F#, watch my Pluralsight course <a href="http://bitly.com/functional-architecture-with-fsharp">A Functional Architecture with F#</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Conditional composition of functions http://blog.ploeh.dk/2016/07/04/conditional-composition-of-functions 2016-07-04T06:53:00+00:00 Mark Seemann <div id="post"> <p> <em>A major benefit of Functional Programming is the separation of decisions and (side-)effects. Sometimes, however, one decision ought to trigger an impure operation, and then proceed to make more decisions. Using functional composition, you can succinctly conditionally compose functions.</em> </p> <p> In my <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">article on how Functional Architecture falls into the Ports and Adapters pit of success</a>, I describe how Haskell forces you to separate concerns: <ul> <li> Your Domain Model should be <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a>, with business decisions implemented by pure functions. Not only does it make it easier for you to reason about the business logic, it also has the side-benefit that <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">pure functions are intrinsically testable</a>.</li> <li> Side-effects, and other impure operations (such as database queries) can be isolated and implemented as <a href="http://xunitpatterns.com/Humble%20Object.html">humble</a> functions. </li> </ul> A common pattern that you can often use is: <ol> <li>Read some data using an impure query.</li> <li>Pass that data to a pure function.</li> <li>Use the return value from the pure function to perform various side-effects. You could, for example, write data to a database, send an email, or update a user interface.</li> </ol> Sometimes, however, things are more complex. Based on the answer from one pure function, you may need to query an additional data source to gather extra data, call a second pure function, and so on. In this article, you'll see one way to accomplish this. </p> <p> <strong>Caravans for extra space</strong> </p> <p> Based on my <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">previous restaurant-booking example</a>, <a href="https://github.com/MartinRykfors">Martin Rykfors</a> suggests "a new feature request. The restaurant has struck a deal with the local caravan dealership, allowing them to rent a caravan to park outside the restaurant in order to increase the seating capacity for one evening. Of course, sometimes there are no caravans available, so we'll need to query the caravan database to see if there is a big enough caravan available that evening:" </p> <p> <pre>findCaravan :: ServiceAddress -&gt; Int -&gt; ZonedTime -&gt; IO (Maybe Caravan)</pre> </p> <p> The above <code>findCaravan</code> is a slight modification of the function Martin suggests, because I imagine that the caravan dealership exposes their caravan booking system as a web service, so the function needs a service address as well. This change doesn't impact the proposed solution, though. </p> <p> This problem definition fits the above general problem statement: you'd only want to call the <code>findCaravan</code> function if <code>checkCapacity</code> returns <code>Left CapacityExceeded</code>. </p> <p> That's still a business decision, so you ought to implement it as a pure function. If you (for a moment) imagine that you have a <code>Maybe Caravan</code> instead of an <code>IO (Maybe Caravan)</code>, you have all the information required to make that decision: </p> <p> <pre>checkCaravanCapacityOnError :: Error -&gt; Maybe Caravan -&gt; Reservation -&gt; Either Error Reservation checkCaravanCapacityOnError CapacityExceeded (Just caravan) reservation = if caravanCapacity caravan &lt; quantity reservation then Left CapacityExceeded else Right reservation checkCaravanCapacityOnError err _ _ = Left err</pre> </p> <p> Notice that this function not only takes <code>Maybe Caravan</code>, it also takes an <code>Error</code> value. This encodes into the function's type that you should only call it if you have an <code>Error</code> that originates from a previous step. This <code>Error</code> value also enables the function to only check the caravan's capacity if the previous <code>Error</code> was a <code>CapacityExceeded</code>. <code>Error</code> can also be <code>ValidationError</code>, in which case there's no reason to check the caravan's capacity. </p> <p> This takes care of the Domain Model, but you still need to figure out how to get a <code>Maybe Caravan</code> value. Additionally, if <code>checkCaravanCapacityOnError</code> returns <code>Right Reservation</code>, you'd probably want to reserve the caravan for the evening. You can imagine that this is possible with the following function: </p> <p> <pre>reserveCaravan :: ServiceAddress -&gt; ZonedTime -&gt; Caravan -&gt; IO ()</pre> </p> <p> This function reserves the caravan at the supplied time. In order to keep the example simple, you can imagine that the provided <code>ZonedTime</code> indicates an entire day (or evening), and not just an instant. </p> <p> <strong>Composition of caravan-checking</strong> </p> <p> As a first step, you can compose an impure function that <ol> <li>Queries the caravan dealership for a caravan</li> <li>Calls the pure <code>checkCaravanCapacityOnError</code> function</li> <li>Reserves the caravan if the return value was <code>Right Reservation</code></li> </ol> Notice how these steps follow the impure-pure-impure pattern I described above. You can compose such a function like this: </p> <p> <pre>import Control.Monad (forM_) import Control.Monad.Trans (liftIO) import Control.Monad.Trans.Either (EitherT(..), hoistEither) checkCaravan :: Reservation -&gt; Error -&gt; EitherT Error IO Reservation checkCaravan reservation err = do c &lt;- liftIO $ findCaravan svcAddr (quantity reservation) (date reservation) newRes &lt;- hoistEither $ checkCaravanCapacityOnError err c reservation liftIO $ forM_ c $ reserveCaravan svcAddr (date newRes) return newRes</pre> </p> <p> It starts by calling <code>findCaravan</code> by closing over <code>svcAddr</code> (a <code>ServiceAddress</code> value). This is an impure operation, but you can use <code>liftIO</code> to make <code>c</code> a <code>Maybe Caravan</code> value that can be passed to <code>checkCaravanCapacityOnError</code> on the next line. This function returns <code>Either Error Reservation</code>, but since this function is defined in an <code>EitherT Error IO Reservation</code> context, <code>newRes</code> is a <code>Reservation</code> value. Still, it's important to realise that exactly because of this context, execution will short-circuit at that point if the return value from <code>checkCaravanCapacityOnError</code> is a <code>Left</code> value. In other words, all subsequent expression are only evaluated if <code>checkCaravanCapacityOnError</code> returns <code>Right</code>. This means that the <code>reserveCaravan</code> function is only called if a caravan with enough capacity is available. </p> <p> The <code>checkCaravan</code> function will <em>unconditionally</em> execute if called, so as the final composition step, you'll need to figure out how to compose it into the overall <code>postReservation</code> function in such a way that it's only called if <code>checkCapacity</code> returns <code>Left</code>. </p> <p> <strong>Conditional composition</strong> </p> <p> In the previous incarnation of this example, the overall entry point for the HTTP request in question was this <code>postReservation</code> function: </p> <p> <pre>postReservation :: ReservationRendition -&gt; IO (HttpResult ()) postReservation candidate = fmap toHttpResult $ runEitherT $ do r &lt;- hoistEither $ validateReservation candidate i &lt;- liftIO $ getReservedSeatsFromDB connStr $ date r hoistEither $ checkCapacity 10 i r &gt;&gt;= liftIO . saveReservation connStr</pre> </p> <p> Is it possible to compose <code>checkCaravan</code> into this function in such a way that it's only going to be executed if <code>checkCapacity</code> returns <code>Left</code>? Yes, by adding to the <code>hoistEither $ checkCapacity 10 i r</code> pipeline: </p> <p> <pre>import Control.Monad.Trans (liftIO) import Control.Monad.Trans.Either (EitherT(..), hoistEither, right, eitherT) postReservation :: ReservationRendition -&gt; IO (HttpResult ()) postReservation candidate = fmap toHttpResult $ runEitherT $ do r &lt;- hoistEither $ validateReservation candidate i &lt;- liftIO $ getReservedSeatsFromDB connStr $ date r eitherT (checkCaravan r) right $ hoistEither $ checkCapacity 10 i r &gt;&gt;= liftIO . saveReservation connStr</pre> </p> <p> Contrary to F#, you have to read Haskell pipelines from right to left. In the second-to-last line of code, you can see that I've added <code>eitherT (checkCaravan r) right</code> to the left of <code>hoistEither $ checkCapacity 10 i r</code>, which was already there. This means that, instead of binding the result of <code>hoistEither $ checkCapacity 10 i r</code> directly to the <code>saveReservation</code> composition (via the monadic <code>&gt;&gt;=</code> <em>bind</em> operator), that result is first passed to <code>eitherT (checkCaravan r) right</code>. </p> <p> The <code>eitherT</code> function composes two other functions: the leftmost function is invoked if the input is <code>Left</code>, and the right function is invoked if the input is <code>Right</code>. In this particular example, <code>(checkCaravan r)</code> is the closure being invoked in the <code>Left</code> - and only in the <code>Left</code> - case. In the <code>Right</code> case, the value is passed on unmodified, but elevated back into the <code>EitherT</code> context using the <code>right</code> function. </p> <p> (BTW, the above composition has a subtle bug: the capacity is still hard-coded as <code>10</code>, even though reserving extra caravans actually increases the overall capacity of the restaurant for the day. I'll leave it as an exercise for you to make the capacity take into account any reserved caravans. You can download <a href="https://gist.github.com/ploeh/c999e2ae2248bd44d775">all of the source code</a>, if you want to give it a try.) </p> <p> <strong>Interleaving</strong> </p> <p> Haskell has strong support for composition of functions. Not only can you interleave pure and impure code, but you can also do it <em>conditionally</em>. In the above example, the <code>eitherT</code> function holds the key to that. The overall flow of the <code>postReservation</code> function is: <ol> <li>Validate the input</li> <li>Get already reserved seats from the database</li> <li>Check the reservation request against the restaurant's remaining capacity</li> <li>If the capacity is exceeded, attempt to reserve a caravan of sufficient capacity</li> <li>If one of the previous steps decided that the restaurant has enough capacity, then save the reservation in the database</li> <li>Convert the result (whether it's <code>Left</code> or <code>Right</code>) to an HTTP response</li> </ol> Programmers who are used to implementing such solutions using C#, Java, or similar languages, may feel hesitant by delegating a branching decision to a piece of composition code, instead of something they can unit test. </p> <p> Haskell's type system is remarkably helpful here. Haskell programmers often joke that <em>if it compiles, it works</em>, and there's a morsel of truth in that sentiment. Both functions used with <code>eitherT</code> must return a value of the same type, but the left function <em>must</em> be a function that takes the <code>Left</code> type as input, whereas the right function <em>must</em> be a function that takes the <code>Right</code> type as input. In the above example, <code>(checkCaravan r)</code> is a partially applied function with the type <code>Error -&gt; EitherT Error IO Reservation</code>; that is: the input type is <code>Error</code>, so it can only be composed with an <code>Either Error a</code> value. That matches the return type of <code>checkCapacity 10 i r</code>, so the code compiles, but if I accidentally switch the arguments to <code>eitherT</code>, it doesn't compile. </p> <p> I find it captivating to figure out how to 'click' together such interleaving functions using the composition functions that Haskell provides. Often, when the composition compiles, it works as intended. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="http://stackoverflow.com/users/1523776/benjamin-hodgson?tab=profile">Benjamin Hodgson</a></div> <div class="comment-content"> <p> This is something of a tangent, but I wanted to hint to you that Haskell can help reduce the boilerplate it takes to compose monadic computations like this. </p> <p> The <a href="https://hackage.haskell.org/package/mtl-2.2.1/docs/Control-Monad-Except.html#t:MonadError"><code>MonadError</code></a> class abstracts monads which support throwing and catching errors. If you don't specify the concrete monad (transformer stack) a computation lives in, it's easier to compose it into a larger environment. </p> <p> <pre>checkCaravanCapacityOnError :: MonadError Error m =&gt; Error -&gt; Maybe Caravan -&gt; Reservation -&gt; m Reservation checkCaravanCapacityOnError CapacityExceeded (Just caravan) reservation | caravanCapacity caravan &lt; quantity reservation = throwError CapacityExceeded | otherwise = return reservation checkCaravanCapacityOnError err _ _ = throwError err</pre> </p> <p> I'm programming to the interface, not the implementation, by using <code>throwError</code> and <code>return</code> instead of <code>Left</code> and <code>Right</code>. This allows me to dispense with calls to <code>hoistEither</code> when I come to call my function in the context of a bigger monad: </p> <p> <pre>findCaravan :: MonadIO m =&gt; ServiceAddress -&gt; Int -&gt; ZonedTime -&gt; m (Maybe Caravan) reserveCaravan :: MonadIO m =&gt; ServiceAddress -&gt; ZonedTime -&gt; m () checkCaravan :: (MonadIO m, MonadError Error m) =&gt; Reservation -&gt; Error -&gt; m Reservation checkCaravan reservation err = do c &lt;- findCaravan svcAddr (quantity reservation) (date reservation) newRes &lt;- checkCaravanCapacityOnError err c reservation traverse_ (reserveCaravan svcAddr (date newRes)) c return newRes</pre> </p> <p> Note how <code>findCaravan</code> and <code>reserveCaravan</code> only declare a <code>MonadIO</code> constraint, whereas <code>checkCaravan</code> needs to do both IO and error handling. The type class system lets you declare the capabilities you need from your monad without specifying the monad in question. The elaborator figures out the right number of calls to <code>lift</code> when it builds the <code>MonadError</code> dictionary, which is determined by the concrete type you choose for <code>m</code> at the edge of your system. </p> <p> A logical next step here would be to further constrain the effects that a given IO function can perform. In this example, I'd consider writing a separate class for monads which support calling the caravan service: <code>findCaravan :: MonadCaravanService m =&gt; ServiceAddress -&gt; Int -&gt; ZonedTime -&gt; m (Maybe Caravan)</code>. This ensures that <code>findCaravan</code> can <i>only</i> call the caravan service, and not perform any other IO. It also makes it easier to mock functions which call the caravan service by writing a fake instance of <code>MonadCaravanService</code>. </p> <p> F# doesn't support this style of programming because it lacks higher-kinded types. You can't abstract over <code>m</code>; you have to pick a concrete monad up-front. This is bad for code reuse: if you need to add (for example) runtime configuration to your application you have to rewrite the implementation of your monad, and potentially every function which uses it, rather than just tacking on a <code>MonadReader</code> constraint to the functions that need it and adding a call to <code>runReaderT</code> at the entry point to your application. </p> <p> Finally, monad transformers are but one style of effect typing; <a href="http://okmij.org/ftp/Haskell/extensible/more.pdf"><code>extensible-effects</code></a> is an alternative which is gaining popularity. </p> </div> <div class="comment-date">2016-07-04 12:05 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://github.com/MartinRykfors">Martin Rykfors</a></div> <div class="comment-content"> <p> Hi Mark, </p> <p> Thank you very much for the elaborate explanation. I'm also delighted that you stuck with my admittedly contrived example of using caravans to compensate for the restaurant's lack of space. Or is that nothing compared to some of the stranger real-life feature requests some of us have seen? </p> <p> I agree with your point on my previous comment that my suggestion could be considered a leaky abstraction and would introduce unnecessary requirements to the implementation. It just feels strange to let go of the idea that the domain logic is to be unconditionally pure, and not just mostly pure with the occasional impure function passed in as an argument. It's like what you say towards the end of the post - I feel hesitant to mix branching code and composition code together. The resulting solution you propose here is giving me second thoughts though. The <code>postReservation</code> function didn't become much more complex as I'd feared, with the branching logic nicely delegated to the <code>eitherT</code> function. The caravan logic also gets its own composition function that is easy enough to understand on its own. I guess I've got some thinking to do. </p> <p> So, a final question regarding this example: To what extent would you apply this technique when solving the same problem in F#? It seems that we are using an increasing amount of Haskell language features not present in F#, so maybe not everything would translate over cleanly. </p> </div> <div class="comment-date">2016-07-04 19:05 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Martin, I'm still experimenting with how that influences my F# code. I'd like to at least attempt to back-port something like this to F# using computation expressions, but it may turn out that it's not worth the effort. </p> </div> <div class="comment-date">2016-07-04 20:46 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Roman numerals via property-based TDD http://blog.ploeh.dk/2016/06/28/roman-numerals-via-property-based-tdd 2016-06-28T07:28:00+00:00 Mark Seemann <div id="post"> <p> <em>An example of doing the Roman numerals kata with property-based test-driven development.</em> </p> <p> The <em>Roman numerals</em> kata is a simple programming exercise. You should implement conversion to and from <a href="https://en.wikipedia.org/wiki/Roman_numerals">Roman numerals</a>. This always struck me as the ultimate case for example-driven development, but I must also admit that I've managed to get stuck on the exercise doing exactly that: throwing more and more examples at the problem. Prompted by previous successes with property-based testing, I wondered whether the problem would be more tractable if approached with property-based testing. This turned out to be the case. </p> <p> <strong>Single values</strong> </p> <p> When modelling a problem with property-based testing, you should attempt to express it in terms of general rules, but even so, the fundamental rule of Roman numerals is that there are certain singular symbols that have particular numeric values. There are no overall principles guiding these relationships; they simply are. Therefore, you'll still need to express these singular values as particular values. This is best done with a simple parametrised test, here using <a href="http://xunit.github.io">xUnit.net 2.1</a>: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Theory</span>&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;I&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;1)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;V&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;5)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;X&quot;</span>,&nbsp;&nbsp;&nbsp;10)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;L&quot;</span>,&nbsp;&nbsp;&nbsp;50)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;C&quot;</span>,&nbsp;&nbsp;100)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;D&quot;</span>,&nbsp;&nbsp;500)&gt;] [&lt;<span style="color:#4ec9b0;">InlineData</span>(<span style="color:#a31515;">&quot;M&quot;</span>,&nbsp;1000)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``elemental&nbsp;symbols&nbsp;have&nbsp;correct&nbsp;values``</span>&nbsp;(symbol&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>)&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Some</span>&nbsp;expected&nbsp;=!&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>&nbsp;symbol</pre> </p> <p> The <code>=!</code> operator is a custom operator defined by <a href="http://www.swensensoftware.com/unquote">Unquote</a> (3.1.1), an assertion library. You can read it as <em>must equal</em>; that is, you can read this particular assertion as <em>some expected must equal tryParseRoman symbol</em>. </p> <p> As you can see, this simple test expresses the relationship between the singular Roman numeral values and their decimal counterparts. You might still consider this automated test as example-driven, but I more think about it as establishing the ground rules for how Roman numerals work. If you look at <a href="https://en.wikipedia.org/wiki/Roman_numerals#Roman_numeric_system">the Wikipedia article</a>, for instance, it also starts explaining the system by listing the values of these seven symbols. </p> <p> <strong>Round-tripping</strong> </p> <p> A common technique when applying property-based testing to parsing problems is to require that values can round-trip. This should also be the case here: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;romanRange&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[1..3999]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> [&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``tryParse&nbsp;is&nbsp;the&nbsp;inverse&nbsp;of&nbsp;toRoman``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;romanRange&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;i&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="color:navy;background:yellow;">Some</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">i</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">=</span><span style="background:yellow;">&nbsp;(</span><span style="background:yellow;">i</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Numeral</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">toRoman</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">bind</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Numeral</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">tryParseRoman</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> This property uses <a href="http://fscheck.github.io/FsCheck">FsCheck</a> (2.2.4). First, it expresses a range of relevant integers. For various reasons (that we'll return to) we're only going to attempt conversion of the integers between 1 and 3,999. The value <code>romanRange</code> has the type Arbitrary&lt;int&gt;, where Arbitrary&lt;'a&gt; is a type defined by FsCheck. You can think of it as a generator of random values. In this case, <code>romanRange</code> generates random integers between 1 and 3,999. </p> <p> When used with Prop.forAll, the property states that for all values drawn from romanRange, the anonymous function should succeed. The <code>i</code> argument within that anonymous function is populated by <code>romanRange</code>, and the function is executed 100 times (by default). </p> <p> The <code>test</code> function is another Unquote function. It evaluates and reports on the quoted boolean expression; if it evaluates to true, nothing happens, but it throws an exception if it evaluates to false. </p> <p> The particular expression states that if you call toRoman with <code>i</code>, and then call tryParseRoman with the return value of that function call, the result should be equal to <code>i</code>. Both sides should be wrapped in a Some case, though, since both toRoman and tryParseRoman might also return None. For the values in romanRange, however, you'd expect that the round-trip always succeeds. </p> <p> <strong>Additivity</strong> </p> <p> The fundamental idea about Roman numerals is that they are additive: I means 1, II means (1 + 1 =) 2, XXX means (10 + 10 + 10 =) 30, and MLXVI means (1000 + 50 + 10 + 5 + 1 =) 1066. You simply count and add. Yes, there are special rules for subtractive shorthand, but forget about those for a moment. If you have a Roman numeral with symbols in strictly descending order, you can simply add the symbol values together. </p> <p> You can express this with FsCheck. It looks a little daunting, but actually isn't that bad. I'll show it first, and then walk you through it: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``symbols&nbsp;in&nbsp;descending&nbsp;order&nbsp;are&nbsp;additive``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">stringRepeat</span>&nbsp;char&nbsp;count&nbsp;=&nbsp;<span style="color:#4ec9b0;">String</span>&nbsp;(char,&nbsp;count) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;count&nbsp;symbol&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[0..count]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:navy;">stringRepeat</span>&nbsp;symbol)&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;thousands&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;3&nbsp;<span style="color:#a31515;">&#39;M&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fiveHundreds&nbsp;=&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;1&nbsp;<span style="color:#a31515;">&#39;D&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;hundreds&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;3&nbsp;<span style="color:#a31515;">&#39;C&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fifties&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;1&nbsp;<span style="color:#a31515;">&#39;L&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;tens&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;3&nbsp;<span style="color:#a31515;">&#39;X&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fives&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;1&nbsp;<span style="color:#a31515;">&#39;V&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;ones&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">genSymbols</span>&nbsp;3&nbsp;<span style="color:#a31515;">&#39;I&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;symbols&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[thousands;&nbsp;fiveHundreds;&nbsp;hundreds;&nbsp;fifties;&nbsp;tens;&nbsp;fives;&nbsp;ones] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">sequence</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:#4ec9b0;">String</span>.<span style="color:navy;">Concat</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;symbols&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>&nbsp;s &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;s &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:navy;">string</span>&nbsp;&gt;&gt;&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">choose</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">sum</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> The first two lines are two utility functions. The function <code>stringRepeat</code> has the type <code>char -&gt; int -&gt; string</code>. It simply provides a curried form of the String constructor overload that enables you to repeat a particular char value. As an example, <code>stringRepeat 'I' 0</code> is "", <code>stringRepeat 'X' 2</code> is "XX", and so on. </p> <p> The function <code>genSymbols</code> has the type <code>int -&gt; char -&gt; Gen&lt;string&gt;</code>. It returns a generator that produces a repeated string no longer than the specified length. Thus, <code>genSymbols 3 'M'</code> is a generator that draws random values from the set [""; "M"; "MM"; "MMM"], <code>genSymbols 1 'D'</code> is a generator that draws random values from the set [""; "D"], and so on. Notice that the empty string is one of the values that the generator may use. This is by design. </p> <p> Using genSymbols, you can define generators for all the symbols: up to three thousands, up to one five hundreds, up to three hundreds, etc. <code>thousands</code>, <code>fiveHundreds</code>, <code>hundreds</code>, and so on, are all values of the type Gen&lt;string&gt;. </p> <p> You can combine all these string generators to a single generator using Gen.sequence, which takes a seq&lt;Gen&lt;'a&gt;&gt; as input and returns a Gen&lt;'a list&gt;. In this case, the input is [thousands; fiveHundreds; hundreds; fifties; tens; fives; ones], which has the type <code>Gen&lt;string&gt; list</code>, so the output is a <code>Gen&lt;string list&gt;</code> value. Values generated could include ["MM"; ""; ""; ""; "X"; ""; ""], [""; "D"; "CC"; "L"; "X"; "V"; ""], and so on. </p> <p> Instead of lists of strings, you need single string values. These are easy to create using the built-in method String.Concat. You have to do it within a Gen value, though, so it's <code>Gen.map String.Concat</code>. Finally, you can convert the resulting Gen&lt;string&gt; to an Arbitrary using Arb.fromGen. The final <code>symbols</code> value has the type Arbitrary&lt;string&gt;. It'll generate values such as "MMX" and "DCCLXV". </p> <p> This is a bit of work to ensure that proper Roman numerals are generated, but the rest of the property is tractable. You can use FsCheck's Prop.forAll to express the property that when tryParseRoman is called with any of the generated numerals, the return value is equal to the expected value. </p> <p> The expected value is the sum of the value of each of the symbols in the input string, <code>s</code>. The <code>string</code> type implements the interface <code>char seq</code>, so you can map each of the characters by invoking tryParseRoman. That gives you a <cdata>seq&lt;int option&gt;</cdata>, because tryParseRoman returns <code>int option</code> values. You can use <code>Seq.choose id</code> to throw away any None values there may be, and then <code>Seq.sum</code> to calculate the sum of the integers. Finally, you can pipe the sum into the Some case constructor to turn <code>expected</code> into an <code>int option</code>, which matches the type of <code>actual</code>. </p> <p> Now that you have <code>expected</code> and <code>actual</code> values, you can assert that these two values must be equal to each other. This property states that for all strictly descending numerals, the return value must be the sum of the constituent symbols. </p> <p> <strong>Subtractiveness</strong> </p> <p> The principal rule for Roman numerals is that of additivity, but if you only apply the above rules, you'd allow numerals such as IIII for 4, LXXXX for 90, etc. While there's historical precedent for such notation, it's not allowed in 'modern' Roman numerals. If there are more than three repeated characters, you should instead prefer subtractive notation: IV for 4, XC for 90, and so on. </p> <p> Subtractive notation is, however, only allowed within adjacent groups. Logically, you could write 1999 as MIM, but this isn't allowed. The symbol I can only be used to subtract from V and X, X can only subtract from L and C, and C can only subtract from D and M. </p> <p> Within these constraints, you have to describe the property of subtractive notation. Here's one way to do it: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">OptionBuilder</span>()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Bind</span>(v,&nbsp;<span style="color:navy;">f</span>)&nbsp;=&nbsp;<span style="color:#4ec9b0;">Option</span>.<span style="color:navy;">bind</span>&nbsp;<span style="color:navy;">f</span>&nbsp;v &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Return</span>&nbsp;v&nbsp;=&nbsp;<span style="color:navy;">Some</span>&nbsp;v <span style="color:blue;">let</span>&nbsp;opt&nbsp;=&nbsp;<span style="color:#4ec9b0;">OptionBuilder</span>() [&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``certain&nbsp;symbols&nbsp;in&nbsp;ascending&nbsp;are&nbsp;subtractive``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">subtractive</span>&nbsp;(subtrahend&nbsp;:&nbsp;<span style="color:#4ec9b0;">char</span>)&nbsp;(minuends&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>)&nbsp;=&nbsp;<span style="color:blue;">gen</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;minuend&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;minuends &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;subtrahend,&nbsp;minuend&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;symbols&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">oneof</span>&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">subtractive</span>&nbsp;<span style="color:#a31515;">&#39;I&#39;</span>&nbsp;<span style="color:#a31515;">&quot;VX&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">subtractive</span>&nbsp;<span style="color:#a31515;">&#39;X&#39;</span>&nbsp;<span style="color:#a31515;">&quot;LC&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">subtractive</span>&nbsp;<span style="color:#a31515;">&#39;C&#39;</span>&nbsp;<span style="color:#a31515;">&quot;DM&quot;</span>&nbsp;] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;symbols&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;(subtrahend,&nbsp;minuend)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;originalRoman&nbsp;=&nbsp;<span style="color:#4ec9b0;">String</span>&nbsp;[|&nbsp;subtrahend;&nbsp;minuend&nbsp;|] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>&nbsp;originalRoman &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;roundTrippedRoman&nbsp;=&nbsp;actual&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Option</span>.<span style="color:navy;">bind</span>&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">toRoman</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">opt</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;m&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>&nbsp;(<span style="color:navy;">string</span>&nbsp;minuend) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;s&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">tryParseRoman</span>&nbsp;(<span style="color:navy;">string</span>&nbsp;subtrahend) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;m&nbsp;-&nbsp;s&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Some</span>&nbsp;originalRoman&nbsp;=!&nbsp;roundTrippedRoman</pre> </p> <p> Like the previous property, this looks like a mouthful, but isn't too bad. I'll walk you through it. </p> <p> Initially, ignore the OptionBuilder type and the opt value; we'll return to them shortly. The property itself starts by defining a function called <code>subtractive</code>, which has the type <code>char -&gt; string -&gt; Gen&lt;char * char&gt;</code>. The first argument is a symbol representing <a href="https://en.wikipedia.org/wiki/Subtraction">the subtrahend</a>; that is: the number being subtracted. The next argument is a sequence of minuends; that is: numbers from which the subtrahend will be subtracted. </p> <p> The <code>subtractive</code> function is implemented with a <code>gen</code> computation expression. It first uses a <code>let!</code> binding to define that a singular minuend is a random value drawn from <code>minuends</code>. As usual, Gen.elements is the workhorse: it defines a generator that randomly draws from a sequence of values, and because <code>minuends</code> is a string, and the string type implements <code>char seq</code>, it can be used with Gen.elements to define a generator of char values. While <code>Gen.elements minuends</code> returns a Gen&lt;char&gt; value, the use of a <code>let!</code> binding within a computation expression causes <code>minuend</code> to have the type <code>char</code>. </p> <p> The second line of code in <code>subtractive</code> returns a tuple of two char values: the subtrahend first, and the minuend second. Normally, when subtracting with the arithmetic minus operator, you'd write a difference as <em>minuend - subtrahend</em>; the minuends comes first, followed by the subtrahend. The Roman subtractive notation, however, is written with the subtrahend before the minuend, which is the reason that the <code>subtractive</code> function returns the symbols in that order. It's easier to think about that way. </p> <p> The <code>subtractive</code> function enables you to define generators of Roman numerals using subtractive notation. Since I can only be used before V and X, you can define a generator using <code>subtractive 'I' "VX"</code>. This is a Gen&lt;char * char&gt; value. Likewise, you can define <code>subtractive 'X' "LC"</code> and <code>subtractive 'C' "DM"</code> and use Gen.oneOf to define a generator that randomly selects one of these generators, and uses the selected generator to produce a value. As always, the last step is to convert the generator into an Arbitrary with Arb.fromGen, so that <code>symbols</code> has the type Arbitrary&lt;char * char&gt;. </p> <p> Equipped with an Arbitrary, you can again use Prop.forAll to express the desired property. First, <code>originalRoman</code> is created from the subtrahend and minuend. Due to the way <code>symbols</code> is defined, <code>originalRoman</code> will have values like "IV", "XC", and so on. </p> <p> The property then proceeds to invoke tryParseRoman. It also uses the <code>actual</code> value to produce a round-tripped value. Not only should the parser correctly understand subtractive notation, but the integer-to-Roman conversion should also prefer this notation. </p> <p> The last part of the property is the assertion. Here, you need <code>opt</code>, which is a <a href="https://msdn.microsoft.com/en-us/library/dd233182.aspx">computation builder</a> for option values. </p> <p> In the assertion, you need to calculate the expected value. Both minuend and subtrahend are char values; in order to find their corresponding decimal values, you'll need to call tryParseRoman. The problem is that tryParseRoman returns an <code>int option</code>. For example, <code>tryParseRoman "I"</code> returns <code>Some 1</code>, so you may need to subtract <code>Some 1</code> from <code>Some 5</code>. The most readable way I've found is to use a computation expression. Using <code>let!</code> bindings, both <code>m</code> and <code>s</code> are <code>int</code> values, which you can easily subtract using the normal <code>-</code> operator. </p> <p> <code>expected</code> and <code>actual</code> are both <code>int option</code> values, so can be compared using Unquote's <em>must equal</em> operator. </p> <p> Finally, the property also asserts that the original value must be equal to the round-tripped value. If not, you could have an implementation that correctly parses "IV" as 4, but converts 4 to "IIII". </p> <p> <strong>Limited repetition</strong> </p> <p> The previous property only ensures that subtractive notation is used in simple cases, like IX or CD. It doesn't verify that composite numerals are correctly written. As an example, the converter should convert 1893 to MDCCCXCIII, not MDCCCLXXXXIII. The second alternative is incorrect because it uses LXXXX to represent 90, instead of XC. </p> <p> The underlying property is that any given symbol can only be repeated at most three times. A symbol can appear more than thrice in total, as demonstrated by the valid numeral MDCCCXCIII, in which C appears four times. For any group of repeated characters, however, a character must only be repeated once, twice, or thrice. </p> <p> This also explain why the maximum Roman numeral is MMMCMXCIX, or 3,999. </p> <p> In order to express this property in code, you first need a function to group characters. Here, I've chosen to reuse <a href="http://stackoverflow.com/a/35055165/126014">one of my own creation</a>: </p> <p> <pre><span style="color:green;">//&nbsp;seq&lt;&#39;a&gt;&nbsp;-&gt;&nbsp;&#39;a&nbsp;list&nbsp;list&nbsp;when&nbsp;&#39;a&nbsp;:&nbsp;equality</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">group</span>&nbsp;xs&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">folder</span>&nbsp;x&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[[x]] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;(h<span style="color:navy;">::</span>t)<span style="color:navy;">::</span>ta&nbsp;<span style="color:blue;">when</span>&nbsp;h&nbsp;=&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(x<span style="color:navy;">::</span>h<span style="color:navy;">::</span>t)<span style="color:navy;">::</span>ta &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;acc&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[x]<span style="color:navy;">::</span>acc &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">foldBack</span>&nbsp;<span style="color:navy;">folder</span>&nbsp;xs&nbsp;[]</pre> </p> <p> This function will, for example, group "MDCCCXCIII" like this: </p> <p> <pre>&gt; "MDCCCXCIII" |&gt; group;; val it : char list list = [['M']; ['D']; ['C'; 'C'; 'C']; ['X']; ['C']; ['I'; 'I'; 'I']]</pre> </p> <p> All you need to do is to find the length of all such sub-lists, and assert that the maximum is at most 3: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``there&nbsp;can&nbsp;be&nbsp;no&nbsp;more&nbsp;than&nbsp;three&nbsp;identical&nbsp;symbols&nbsp;in&nbsp;a&nbsp;row``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;romanRange&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;i&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">toRoman</span>&nbsp;i &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">group</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&gt;&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:#4ec9b0;background:yellow;">List</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">map</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">List</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">length</span><span style="background:yellow;">)&nbsp;</span><span style="background:yellow;">&gt;&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">List</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">max</span><span style="background:yellow;">)</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">exists</span><span style="background:yellow;">&nbsp;(</span><span style="color:blue;background:yellow;">fun</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">x</span><span style="background:yellow;">&nbsp;</span><span style="color:blue;background:yellow;">-&gt;</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">x</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;=</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">3</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> Since <code>actual</code> is a <code>string option</code>, you need to express the assertion within the Maybe (Option) monad. First, you can use Option.map to map any value (should there be one) to find the maximum length of any repeated character group. This returns an <code>int option</code>. </p> <p> Finally, you can pipe that <code>int option</code> into Option.exists, which will evaluate to false if there's no value, or if the boolean expression <code>x &lt;= 3</code> evaluates to false. </p> <p> <strong>Input range</strong> </p> <p> At this point, you're almost done. The only remaining properties you'll need to specify is that the maximum value is 3,999, and the minimum value is 1. Negative numbers, or zero, are not allowed: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``negative&nbsp;numbers&nbsp;and&nbsp;zero&nbsp;are&nbsp;not&nbsp;supported``</span>&nbsp;i&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;i&nbsp;=&nbsp;-(<span style="color:navy;">abs</span>&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">toRoman</span>&nbsp;i &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">isNone</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> In this property, the function argument <code>i</code> can be <em>any</em> number, but calling <code>abs</code> ensures that it's positive (or zero), and the unary <code>-</code> operator then converts that positive value to a negative value. </p> <p> Notice that the new <code>i</code> value shadows the input argument of the same name. This is <a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">a common trick when writing properties</a>. It prevents me from accidentally using the input value provided by FsCheck. While the input argument is useful as a seed value, it isn't guaranteed to model the particular circumstances of this property. Here, you only care to assert what happens if the input is negative or zero. Specifically, you always want the return value to be None. </p> <p> Likewise for too large input values: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``numbers&nbsp;too&nbsp;big&nbsp;are&nbsp;not&nbsp;supported``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">choose</span>&nbsp;(4000,&nbsp;<span style="color:#4ec9b0;">Int32</span>.MaxValue)&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span>&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;i&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Numeral</span>.<span style="color:navy;">toRoman</span>&nbsp;i &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="color:#4ec9b0;background:yellow;">Option</span><span style="background:yellow;">.</span><span style="color:navy;background:yellow;">isNone</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> Here, Gen.choose is used to define an Arbitrary&lt;int&gt; that only produces numbers between 4000 and Int32.MaxValue (including both boundary values). </p> <p> This test is similar to the one that exercises negative values, so you could combine them to a single function if you'd like. I'll leave this as an exercise, though. </p> <p> <strong>Implementation</strong> </p> <p> The interesting part of this exercise is, I think, how to define the properties. There are many ways you can implement the functions to pass all properties. Here's one of them: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;System <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryParseRoman</span>&nbsp;candidate&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">add</span>&nbsp;x&nbsp;=&nbsp;<span style="color:#4ec9b0;">Option</span>.<span style="color:navy;">map</span>&nbsp;((+)&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;&nbsp;9)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;V&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;&nbsp;4)&nbsp;xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;&nbsp;1)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;V&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;&nbsp;5)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;90)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;L&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;40)&nbsp;xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;10)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;L&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;&nbsp;50)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;M&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;900)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;D&#39;</span><span style="color:navy;">::</span>xs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;400)&nbsp;xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;100)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;D&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;&nbsp;500)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;M&#39;</span><span style="color:navy;">::</span>xs&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:navy;">add</span>&nbsp;1000)&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">None</span> &nbsp;&nbsp;&nbsp;&nbsp;candidate&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:navy;">Some</span>&nbsp;0) <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">toRoman</span>&nbsp;i&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;1000&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;M&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;1000) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;900&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;M&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;900) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;500&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;D&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;500) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;400&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;D&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;400) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;100&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;100) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;90&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;C&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;90) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;50&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;L&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;50) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;40&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;L&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;40) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;10&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;10) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;&nbsp;9&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;X&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;&nbsp;9) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;&nbsp;5&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;V&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;&nbsp;5) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;&nbsp;4&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;V&#39;</span><span style="color:navy;">::</span><span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span>acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;&nbsp;4) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;x&nbsp;<span style="color:blue;">when</span>&nbsp;x&nbsp;&gt;=&nbsp;&nbsp;&nbsp;&nbsp;1&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;(<span style="color:#a31515;">&#39;I&#39;</span><span style="color:navy;">::</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc)&nbsp;(x&nbsp;-&nbsp;&nbsp;&nbsp;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;0&nbsp;&lt;&nbsp;i&nbsp;&amp;&amp;&nbsp;i&nbsp;&lt;&nbsp;4000 &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;[]&nbsp;i&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">rev</span>&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">toArray</span>&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">String</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">Some</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> Both functions use <a href="http://blog.ploeh.dk/2015/12/22/tail-recurse">tail-recursive inner <code>imp</code> functions</a> in order to accumulate the appropriate answer. </p> <p> One of the nice properties (that I didn't test for) of this implementation is that the tryParseRoman function is a <a href="http://martinfowler.com/bliki/TolerantReader.html">Tolerant Reader</a>. While toRoman would never create such a numeral, tryParseRoman correctly understands some alternative renderings: </p> <p> <pre>&gt; "MDCCCLXXXXIII" |&gt; tryParseRoman;; val it : int option = Some 1893 &gt; 1893 |&gt; toRoman;; val it : String option = Some "MDCCCXCIII"</pre> </p> <p> In other words, the implementation follows <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>. tryParseRoman is liberal in what it accepts, while toRoman is conservative in what it returns. </p> <p> <strong>Summary</strong> </p> <p> Some problems look, at first glance, as obvious candidates for example-driven development. In my experience, this particularly happen when obvious examples abound. It's not difficult to come up with examples of Roman numerals, so it seems intuitive that you should just start writing some test cases with various examples. In my experience, though, that doesn't guarantee that you're led towards a good implementation. </p> <p> The more a problem description is based on examples, the harder it can be to identify the underlying properties. Still, they're often there, once you start looking. As I've <a href="http://blog.ploeh.dk/2015/01/10/diamond-kata-with-fscheck">previously reported</a>, using property-based test-driven development enables you to proceed in a more incremental fashion, because properties describe only parts of the desired solution. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. SUT Double http://blog.ploeh.dk/2016/06/15/sut-double 2016-06-15T18:01:00+00:00 Mark Seemann <div id="post"> <p> <em>While it's possible to rely on Extract and Override for unit testing, Dependency Injection can make code and tests simpler and easier to maintain.</em> </p> <p> In object-oriented programming, many people still struggle with the intersection of design and testability. If a unit test is <em>an automated test of a unit in isolation of its dependencies</em>, then how do you isolate an object from its dependencies, most notably databases, web services, and the like? </p> <p> One technique, described in <a href="http://amzn.to/1ygZ4Jv">The Art of Unit Testing</a>, is called <em>Extract and Override</em>. The idea is that you write a class, but use a <a href="https://en.wikipedia.org/wiki/Template_method_pattern">Template Method</a>, <a href="https://en.wikipedia.org/wiki/Factory_method_pattern">Factory Method</a>, or other sort of virtual class method to expose extensibility points that a unit test can use to achieve the desired isolation. </p> <p> People sometimes ask me whether that isn't good enough, and why <a href="http://amzn.to/12p90MG">I advocate the (ostensibly) more complex technique of Dependency Injection</a>? </p> <p> It's easiest to understand the advantages and disadvantages if we have an example to discuss. </p> <p> <strong>Example: Extract and Override</strong> </p> <p> Imagine that you're creating a reservation system for a restaurant. Clients (web sites, smart phone apps, etcetera) display a user interface where you can fill in details about your reservation: your name, email address, the number of guests, and the date of your reservation. When you submit your reservation request, the client POSTs a JSON document to a web service. </p> <p> In this example, the web service is implemented by a Controller class, and the Post method handles the incoming request: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Capacity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpActionResult</span>&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;requestedDate; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>(!<span style="color:#2b91af;">DateTime</span>.TryParse(reservationDto.Date,&nbsp;<span style="color:blue;">out</span>&nbsp;requestedDate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.BadRequest(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservedSeats&nbsp;=&nbsp;<span style="color:blue;">this</span>.ReadReservedSeats(requestedDate); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>(<span style="color:blue;">this</span>.Capacity&nbsp;&lt;&nbsp;reservationDto.Quantity&nbsp;+&nbsp;reservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.StatusCode(<span style="color:#2b91af;">HttpStatusCode</span>.Forbidden); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.SaveReservation(requestedDate,&nbsp;reservationDto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.Ok(); }</pre> </p> <p> The implementation is simple: It first attempts to validate the incoming document, and returns an error message if the document is invalid. Second, it reads the number of already reserved seats via a helper method, and rejects the request if the remaining capacity is insufficient. On the other hand, if the remaining capacity is sufficient, it saves the reservation and returns 200 OK. </p> <p> The Post method relies on two helper methods that handles communication with the database: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">int</span>&nbsp;ReadReservedSeats(<span style="color:#2b91af;">DateTime</span>&nbsp;date) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SELECT&nbsp;COALESCE(SUM([Quantity]),&nbsp;0)&nbsp;FROM&nbsp;[dbo].[Reservations] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WHERE&nbsp;YEAR([Date])&nbsp;=&nbsp;YEAR(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;MONTH([Date])&nbsp;=&nbsp;MONTH(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;DAY([Date])&nbsp;=&nbsp;DAY(@Date)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;connStr&nbsp;=&nbsp;<span style="color:#2b91af;">ConfigurationManager</span>.ConnectionStrings[<span style="color:#a31515;">&quot;booking&quot;</span>] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConnectionString; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>(connStr)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;conn)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Open(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;(<span style="color:blue;">int</span>)cmd.ExecuteScalar(); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;SaveReservation( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;dateTime, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INSERT&nbsp;INTO&nbsp;[dbo].[Reservations]&nbsp;([Date],&nbsp;[Name],&nbsp;[Email],&nbsp;[Quantity]) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VALUES&nbsp;(@Date,&nbsp;@Name,&nbsp;@Email,&nbsp;@Quantity)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;connStr&nbsp;=&nbsp;<span style="color:#2b91af;">ConfigurationManager</span>.ConnectionStrings[<span style="color:#a31515;">&quot;booking&quot;</span>] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConnectionString; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>(connStr)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;conn)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;reservationDto.Date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Name&quot;</span>,&nbsp;reservationDto.Name)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Email&quot;</span>,&nbsp;reservationDto.Email)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Quantity&quot;</span>,&nbsp;reservationDto.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Open(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.ExecuteNonQuery(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> In this example, both helper methods are <code>public</code>, but they could have been <code>protected</code> without changing any conclusions; by being <code>public</code>, though, they're easier to override using <a href="http://www.moqthis.com">Moq</a>. The important detail is that both of these methods are overridable. In C#, you declare that with the <code>virtual</code> keyword; in Java, methods are overridable by default. </p> <p> Both of these methods use elemental ADO.NET to communicate with the database. You can also use an ORM, or any other database access technique you prefer - it doesn't matter for this discussion. What matters is that the methods can be overridden by unit tests. </p> <p> Here's a unit test of the happy path: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PostReturnsCorrectResultAndHasCorrectStateOnAcceptableRequest() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;json&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2016-05-31&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Mark&nbsp;Seemann&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;mark@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">ReservationsController</span>&gt;&nbsp;{&nbsp;CallBase&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;sut &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(s&nbsp;=&gt;&nbsp;s.ReadReservedSeats(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Returns(0); &nbsp;&nbsp;&nbsp;&nbsp;sut &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(s&nbsp;=&gt;&nbsp;s.SaveReservation(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31),&nbsp;json)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Verifiable(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Object.Post(json); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">OkResult</span>&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;sut.Verify(); }</pre> </p> <p> This example uses the Extract and Override technique, but instead of creating a test-specific class that derives from ReservationsController, it uses Moq to create a dynamic <a href="http://www.martinfowler.com/bliki/TestDouble.html">Test Double</a> for the System Under Test (<a href="https://en.wikipedia.org/wiki/System_under_test">SUT</a>) - a SUT Double. </p> <p> Since both <code>ReadReservedSeats</code> and <code>SaveReservation</code> are virtual, Moq can override them with test-specific behaviour. In this test, it defines the behaviour of <code>ReadReservedSeats</code> in such a way that it returns <code>0</code> when the input is a particular date. </p> <p> While the test is simple, it has a single blemish. It ought to follow the Arrange Act Assert pattern, so it shouldn't have to configure the <code>SaveReservation</code> method <em>before</em> it calls the the <code>Post</code> method. After all, the <code>SaveReservation</code> method is a Command, and you should use <a href="http://blog.ploeh.dk/2013/10/23/mocks-for-commands-stubs-for-queries">Mocks for Commands</a>. In other words, the test ought to verify the interaction with the <code>SaveReservation</code> method in the Assert phase; not configure it in the Arrange phase. </p> <p> Unfortunately, if you don't configure the <code>SaveReservation</code> method before calling <code>Post</code>, Moq will use the base implementation, which will attempt to interact with the database. The database isn't available in the unit test context, so without that override, the base method will throw an exception, causing the test to fail. </p> <p> Still, that's a minor issue. In general, the test is easy to follow, and the design of the ReservationsController class itself is also straightforward. Are there any downsides? </p> <p> <strong>Example: shared connection</strong> </p> <p> From a design perspective, the above version is fine, but you may find it inefficient that <code>ReadReservedSeats</code> and <code>SaveReservation</code> both open and close a connection to the database. Wouldn't it be more efficient if they could share a single connection? </p> <p> If (<a href="https://ericlippert.com/2012/12/17/performance-rant">by measuring</a>) you decide that you'd like to refactor the ReservationsController class to use a shared connection, your first attempt might look like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpActionResult</span>&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;requestedDate; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:#2b91af;">DateTime</span>.TryParse(reservationDto.Date,&nbsp;<span style="color:blue;">out</span>&nbsp;requestedDate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.BadRequest(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">this</span>.OpenDbConnection()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservedSeats&nbsp;=&nbsp;<span style="color:blue;">this</span>.ReadReservedSeats(conn,&nbsp;requestedDate); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">this</span>.Capacity&nbsp;&lt;&nbsp;reservationDto.Quantity&nbsp;+&nbsp;reservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.StatusCode(<span style="color:#2b91af;">HttpStatusCode</span>.Forbidden); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.SaveReservation(conn,&nbsp;requestedDate,&nbsp;reservationDto); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.Ok(); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>&nbsp;OpenDbConnection() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;connStr&nbsp;=&nbsp;<span style="color:#2b91af;">ConfigurationManager</span>.ConnectionStrings[<span style="color:#a31515;">&quot;booking&quot;</span>] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConnectionString; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>(connStr); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">try</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Open(); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">catch</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Dispose(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;conn; } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">int</span>&nbsp;ReadReservedSeats(<span style="color:#2b91af;">SqlConnection</span>&nbsp;conn,&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;date) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SELECT&nbsp;COALESCE(SUM([Quantity]),&nbsp;0)&nbsp;FROM&nbsp;[dbo].[Reservations] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WHERE&nbsp;YEAR([Date])&nbsp;=&nbsp;YEAR(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;MONTH([Date])&nbsp;=&nbsp;MONTH(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;DAY([Date])&nbsp;=&nbsp;DAY(@Date)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;conn)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;(<span style="color:blue;">int</span>)cmd.ExecuteScalar(); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;SaveReservation( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">SqlConnection</span>&nbsp;conn, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;dateTime, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INSERT&nbsp;INTO&nbsp;[dbo].[Reservations]&nbsp;([Date],&nbsp;[Name],&nbsp;[Email],&nbsp;[Quantity]) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VALUES&nbsp;(@Date,&nbsp;@Name,&nbsp;@Email,&nbsp;@Quantity)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;conn)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;reservationDto.Date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Name&quot;</span>,&nbsp;reservationDto.Name)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Email&quot;</span>,&nbsp;reservationDto.Email)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Quantity&quot;</span>,&nbsp;reservationDto.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.ExecuteNonQuery(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> You've changed both <code>ReadReservedSeats</code> and <code>SaveReservation</code> to take an extra parameter: the connection to the database. That connection is created by the <code>OpenDbConnection</code> method, but you also have to make that method overridable, because otherwise, it'd attempt to connect to a database during unit testing, and thereby causing the tests to fail. </p> <p> You can still unit test using the Extract and Overide technique, but the test becomes more complicated: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PostReturnsCorrectResultAndHasCorrectStateOnAcceptableRequest() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;json&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2016-05-31&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Mark&nbsp;Seemann&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;mark@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">ReservationsController</span>&gt;&nbsp;{&nbsp;CallBase&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;sut.Setup(s&nbsp;=&gt;&nbsp;s.OpenDbConnection()).Returns((<span style="color:#2b91af;">SqlConnection</span>)<span style="color:blue;">null</span>); &nbsp;&nbsp;&nbsp;&nbsp;sut &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(s&nbsp;=&gt;&nbsp;s.ReadReservedSeats( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">It</span>.IsAny&lt;<span style="color:#2b91af;">SqlConnection</span>&gt;(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Returns(0); &nbsp;&nbsp;&nbsp;&nbsp;sut &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(s&nbsp;=&gt;&nbsp;s.SaveReservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">It</span>.IsAny&lt;<span style="color:#2b91af;">SqlConnection</span>&gt;(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31),&nbsp;json)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Verifiable(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Object.Post(json); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">OkResult</span>&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;sut.Verify(); }</pre> </p> <p> Not only must you override <code>ReadReservedSeats</code> and <code>SaveReservation</code>, but you must also supply a <a href="http://xunitpatterns.com/Dummy%20Object.html">Dummy Object</a> for the connection object, as well as override <code>OpenDbConnection</code>. Still manageable, perhaps, but the design indisputably deteriorated. </p> <p> You can summarise the flaw by a single design smell: <a href="http://c2.com/cgi/wiki?FeatureEnvySmell">Feature Envy</a>. Both the <code>ReadReservedSeats</code> and the <code>SaveReservation</code> methods take an argument of the type SqlConnection. On the other hand, they don't use any instance members of the ReservationsController class that currently hosts them. These methods seem like they ought to belong to SqlConnection instead of ReservationsController. That's not possible, however, since SqlConnection is a class from the Base Class Library, but you can, instead, create a new Repository class. </p> <p> <strong>Example: Repository</strong> </p> <p> A common design pattern is the Repository pattern, although the way it's commonly implemented today has diverged somewhat from the original description in <a href="http://bit.ly/patternsofeaa">PoEAA</a>. Here, I'm going to apply it like people often do. You start by defining a new class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IDisposable</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">SqlConnection</span>&gt;&nbsp;lazyConn; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;SqlReservationsRepository() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.lazyConn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">SqlConnection</span>&gt;(<span style="color:blue;">this</span>.OpenSqlConnection); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>&nbsp;OpenSqlConnection() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;connStr&nbsp;=&nbsp;<span style="color:#2b91af;">ConfigurationManager</span>.ConnectionStrings[<span style="color:#a31515;">&quot;booking&quot;</span>] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConnectionString; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>(connStr); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">try</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Open(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">catch</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;conn.Dispose(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;conn; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">int</span>&nbsp;ReadReservedSeats(<span style="color:#2b91af;">DateTime</span>&nbsp;date) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SELECT&nbsp;COALESCE(SUM([Quantity]),&nbsp;0)&nbsp;FROM&nbsp;[dbo].[Reservations] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WHERE&nbsp;YEAR([Date])&nbsp;=&nbsp;YEAR(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;MONTH([Date])&nbsp;=&nbsp;MONTH(@Date) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AND&nbsp;DAY([Date])&nbsp;=&nbsp;DAY(@Date)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;<span style="color:blue;">this</span>.lazyConn.Value)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;(<span style="color:blue;">int</span>)cmd.ExecuteScalar(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;SaveReservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;dateTime, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INSERT&nbsp;INTO&nbsp;[dbo].[Reservations]&nbsp;([Date],&nbsp;[Name],&nbsp;[Email],&nbsp;[Quantity]) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VALUES&nbsp;(@Date,&nbsp;@Name,&nbsp;@Email,&nbsp;@Quantity)&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(sql,&nbsp;<span style="color:blue;">this</span>.lazyConn.Value)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;reservationDto.Date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Name&quot;</span>,&nbsp;reservationDto.Name)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Email&quot;</span>,&nbsp;reservationDto.Email)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.Add( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Quantity&quot;</span>,&nbsp;reservationDto.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cmd.ExecuteNonQuery(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Dispose() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Dispose(<span style="color:blue;">true</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">GC</span>.SuppressFinalize(<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">protected</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Dispose(<span style="color:blue;">bool</span>&nbsp;disposing) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(disposing) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.lazyConn.Value.Dispose(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The new SqlReservationsRepository class contains the two <code>ReadReservedSeats</code> and <code>SaveReservation</code> methods, and because you've now moved them to a class that contains a shared database connection, the methods don't need the connection as a parameter. </p> <p> The ReservationsController can use the new SqlReservationsRepository class to do its work, while keeping connection management efficient. In order to make it <em>testable</em>, however, you must make it overridable. The SqlReservationsRepository class' methods are already virtual, but that's not the class you're testing. The System Under Test is the ReservationsController class, and you have to make its use of SqlReservationsRepository overridable as well. </p> <p> If you wish to avoid Dependency Injection, you can use a <a href="https://en.wikipedia.org/wiki/Factory_method_pattern">Factory Method</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpActionResult</span>&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;requestedDate; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:#2b91af;">DateTime</span>.TryParse(reservationDto.Date,&nbsp;<span style="color:blue;">out</span>&nbsp;requestedDate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.BadRequest(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;repo&nbsp;=&nbsp;<span style="color:blue;">this</span>.CreateRepository()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservedSeats&nbsp;=&nbsp;repo.ReadReservedSeats(requestedDate); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">this</span>.Capacity&nbsp;&lt;&nbsp;reservationDto.Quantity&nbsp;+&nbsp;reservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.StatusCode(<span style="color:#2b91af;">HttpStatusCode</span>.Forbidden); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;repo.SaveReservation(requestedDate,&nbsp;reservationDto); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.Ok(); &nbsp;&nbsp;&nbsp;&nbsp;} } &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>&nbsp;CreateRepository() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>(); }</pre> </p> <p> The Factory Method in the above example is the <code>CreateRepository</code> method, which is virtual, and thereby overridable. You can override it in a unit test like this: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PostReturnsCorrectResultAndHasCorrectStateOnAcceptableRequest() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;json&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2016-05-31&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Mark&nbsp;Seemann&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;mark@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;repo&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">SqlReservationsRepository</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;repo &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservedSeats(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Returns(0); &nbsp;&nbsp;&nbsp;&nbsp;repo &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(r&nbsp;=&gt;&nbsp;r.SaveReservation(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31),&nbsp;json)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Verifiable(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">ReservationsController</span>&gt;&nbsp;{&nbsp;CallBase&nbsp;=&nbsp;<span style="color:blue;">true</span>&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;sut.Setup(s&nbsp;=&gt;&nbsp;s.CreateRepository()).Returns(repo.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Object.Post(json); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">OkResult</span>&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;repo.Verify(); }</pre> </p> <p> You'll notice that not only did the complexity increase of the System Under Test, but the test itself became more complicated as well. In the previous version, at least you only needed to create a single <code>Mock&lt;T&gt;</code>, but now you have to create two different test doubles and connect them. This is a typical example demonstrating the shortcomings of the Extract and Override technique. It doesn't scale well as complexity increases. </p> <p> <strong>Example: Dependency Injection</strong> </p> <p> In 1994 we <a href="http://amzn.to/XBYukB">we were taught</a> to <em>favor object composition over class inheritance</em>. You can do that, and still keep your code loosely coupled with <a href="http://amzn.to/12p90MG">Dependency Injection</a>. Instead of relying on virtual methods, inject a polymorphic object into the class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>&nbsp;:&nbsp;<span style="color:#2b91af;">ApiController</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;ReservationsController(<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;repository) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(repository&nbsp;==&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(repository)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Capacity&nbsp;=&nbsp;12; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Repository&nbsp;=&nbsp;repository; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Capacity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;Repository&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpActionResult</span>&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;requestedDate; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:#2b91af;">DateTime</span>.TryParse(reservationDto.Date,&nbsp;<span style="color:blue;">out</span>&nbsp;requestedDate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.BadRequest(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Repository.ReadReservedSeats(requestedDate); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">this</span>.Capacity&nbsp;&lt;&nbsp;reservationDto.Quantity&nbsp;+&nbsp;reservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.StatusCode(<span style="color:#2b91af;">HttpStatusCode</span>.Forbidden); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Repository.SaveReservation(requestedDate,&nbsp;reservationDto); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.Ok(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> In this version of ReservationsController, an IReservationsRepository object is injected into the object via the constructor and saved in a class field for later use. When the <code>Post</code> method executes, it calls <code>Repository.ReadReservedSeats</code> and <code>Repository.SaveReservation</code> without further ado. </p> <p> The IReservationsRepository interface is defined like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;ReadReservedSeats(<span style="color:#2b91af;">DateTime</span>&nbsp;date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;SaveReservation(<span style="color:#2b91af;">DateTime</span>&nbsp;dateTime,&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;reservationDto); }</pre> </p> <p> Perhaps you're surprised to see that it merely defines the two <code>ReadReservedSeats</code> and <code>SaveReservation</code> methods, but makes no attempt at making the interface disposable. </p> <p> Not only is IDisposable an implementation detail, but it also keeps the implementation of ReservationsController simple. Notice how it doesn't attempt to control the lifetime of the injected repository, which may or may not be disposable. In a few paragraphs, we'll return to this matter, but first, witness how the unit test became simpler as well: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PostReturnsCorrectResultAndHasCorrectStateOnAcceptableRequest() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;json&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2016-05-31&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Mark&nbsp;Seemann&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;mark@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;repo&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;repo &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservedSeats(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Returns(0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(repo.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Post(json); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">OkResult</span>&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;repo.Verify( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;r&nbsp;=&gt;&nbsp;r.SaveReservation(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2016,&nbsp;5,&nbsp;31),&nbsp;json)); }</pre> </p> <p> With this test, you can finally use the Arrange Act Assert structure, instead of having to configure the <code>SaveReservation</code> method call in the Arrange phase. This test arranges the <a href="http://xunitpatterns.com/test%20fixture%20-%20xUnit.html">Test Fixture</a> by creating a Test Double for the IReservationsRepository interface and injecting it into the ReservationsController. You only need to configure the <code>ReadReservedSeats</code> method, because there's no default behaviour that you need to suppress. </p> <p> You may be wondering about the potential memory leak when the SqlReservationsRepository is in use. After all, ReservationsController doesn't dispose of the injected repository. </p> <p> You address this concern when you compose the dependency graph. This example uses ASP.NET Web API, which has <a href="http://blog.ploeh.dk/2012/09/28/DependencyInjectionandLifetimeManagementwithASP.NETWebAPI">an extensibility point for this exact purpose</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PureCompositionRoot</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IHttpControllerActivator</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IHttpController</span>&nbsp;Create( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">HttpRequestMessage</span>&nbsp;request, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">HttpControllerDescriptor</span>&nbsp;controllerDescriptor, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Type</span>&nbsp;controllerType) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>(controllerType&nbsp;==&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">ReservationsController</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;repo&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;request.RegisterForDispose(repo); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(repo); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentException</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Unexpected&nbsp;controller&nbsp;type:&nbsp;&quot;</span>&nbsp;+&nbsp;controllerType, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(controllerType)); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The call to <code>request.RegisterForDispose</code> tells the ASP.NET Web API framework to dispose of the concrete <code>repo</code> object once it has handled the request and served the response. </p> <p> <strong>Conclusion</strong> </p> <p> At first glance, it seems like the overall design would be easier if you make your SUT testable via inheritance, but once things become more complex, favouring composition over inheritance gives you more freedom to design according to well-known design patterns. </p> <p> In case you want to dive into the details of the code presented here, it's <a href="https://github.com/ploeh/SUTDouble">available on GitHub</a>. You can follow the progression of the code in the repository's commit history. </p> <p> If you're interested in learning more about advanced unit testing techniques, you can watch <a href="http://bit.ly/advanced-unit-testing">my popular Pluralsight course</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. TIE fighter FsCheck properties http://blog.ploeh.dk/2016/05/17/tie-fighter-fscheck-properties 2016-05-17T10:55:00+00:00 Mark Seemann <div id="post"> <p> <em>Use the F# TIE fighter operator combination to eliminate a hard-to-name intermediary value.</em> </p> <p> A doctrine of <a href="http://amzn.to/XCJi9X">Clean Code</a> is the <em>Boy Scout Rule: leave the code cleaner than you found it</em>. Attempting to live by that rule, I'm always looking for ways to improve my code. </p> <p> Writing properties with <a href="https://fscheck.github.io/FsCheck/">FsCheck</a> is enjoyable, but I've been struggling with expressing ad-hoc Arbitraries in a clean style. Although I've already <a href="http://blog.ploeh.dk/2016/03/01/ad-hoc-arbitraries-now-with-pipes">twice written about this</a>, I've recently found another improvement. </p> <p> <strong>Cleaner, but not clean enough</strong> </p> <p> <a href="http://blog.ploeh.dk/2016/03/01/ad-hoc-arbitraries-now-with-pipes">Previously</a>, I've described how you can use the backward pipe operator to avoid enclosing a multi-line expression in brackets: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nc&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;nc&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual</pre> </p> <p> This property uses the custom Arbitrary <code>nc</code> to express a property about <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Conway's Game of Life</a>. The value <code>nc</code> is a <code>Arbitrary&lt;int&gt;</code>, which is a generator of integer values between 4 and 8 (both included). </p> <p> There's a problem with that code, though: I don't care about <code>nc</code>; I care about the values it creates. That's the important value in my property, and this is the reason I reserved the descriptive name <code>neighborCount</code> for that role. The property cares about the neighbour count: <em>for any neighbour count in the range 4-8, it should hold that</em> blah blah blah, and so on... </p> <p> Unfortunately, I was still left with the need to pass an <code>Arbitrary&lt;'a&gt;</code> to <code>Prop.forAll</code>, so I named it the best I could: <code>nc</code> (short for <em>Neighbour Count</em>). Following <a href="http://amzn.to/XCJi9X">Clean Code's</a> heuristics for short symbol scope, I considered it an acceptable name, albeit a bit cryptic. </p> <p> <strong>TIE Fighters to the rescue!</strong> </p> <p> One day, I was looking at a property like the one above, and I thought: if you consider the expression <code>Prop.forAll nc</code> in isolation, you could also write it <code>nc |&gt; Prop.forAll</code>. Does it work in this context? Yes it does: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nc&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;(nc&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>)&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual</pre> </p> <p> This code compiles, and is equivalent to the first example. I deliberately put the expression <code>nc |&gt; Prop.forAll</code> in brackets to be sure that I'd made a correct replacement. Pipes are left-associative, though, so in this case, the brackets are redundant: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nc&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;nc&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual</pre> </p> <p> This still works (and fails if the system under test has a defect). </p> <p> Due to <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>, though, the value <code>nc</code> is equal to the expression <code>Gen.elements [4..8] |&gt; Arb.fromGen</code>, so you can replace it: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8] &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual</pre> </p> <p> In the above example, I've also slightly reformatted the expression, so that each expression composed with a forward pipe is on a separate line. That's not required, but I find it more readable. </p> <p> Notice how <code>Prop.forAll</code> is now surrounded by pipes: <code>|&gt; Prop.forAll &lt;|</code>. This is <a href="https://twitter.com/MattDrivenDev/status/415433890502033408">humorously called <em>TIE fighter infix</em></a>. </p> <p> <strong>Summary</strong> </p> <p> Sometimes, giving an intermediary value a descriptive name can improve code readability, but in other cases, such a value is only in the way. This was the case with the <code>nc</code> value in the first example above. Using TIE fighter infix notation enables you to get rid of a redundant, hard-to-name symbol. In my opinion, because <code>nc</code> didn't add any information to the code, I find the refactored version easier to read. I've left the code base cleaner than I found it. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. CQS and server-generated Entity IDs http://blog.ploeh.dk/2016/05/06/cqs-and-server-generated-entity-ids 2016-05-06T17:36:00+00:00 Mark Seemann <div id="post"> <p> <em>Is it a violation of Command Query Separation to update an ID property when saving an Entity to a database?</em> </p> <p> In my <a href="http://www.shareasale.com/r.cfm?u=1017843&b=611266&m=53701&afftrack=&urllink=www%2Epluralsight%2Ecom%2Fcourses%2Fencapsulation%2Dsolid">Encapsulation and SOLID</a> course on Pluralsight, I explain how the elusive object-oriented quality <em>encapsulation</em> can be approximated by the actionable principles of <a href="https://en.wikipedia.org/wiki/Command%E2%80%93query_separation">Command Query Separation</a> (CQS) and <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>. </p> <p> One of the questions that invariably arise when people first learn about CQS is how to deal with (database) server-generated IDs. While I've <a href="http://blog.ploeh.dk/2014/08/11/cqs-versus-server-generated-ids">already covered that question</a>, I recently came upon <a href="http://enterprisecraftsmanship.com/2014/11/15/cqs-with-database-generated-ids/#comment-2650708342">a variation of the question</a>: <blockquote> "The Create method is a command, then if the object passed to this method have some changes in their property, does it violate any rule? I mean that we can always get the new id from the object itself, so we don't need to return another integer. Is it good or bad practice?" </blockquote> In this article, I'll attempt to answer this question. </p> <p> <strong>Returning an ID by mutating input</strong> </p> <p> I interpret the question like this: an Entity (as described in <a href="http://amzn.to/WBCwx7">DDD</a>) can have a mutable <code>Id</code> property. A <cdata>Create</cdata> method could save the Entity in a database, and then update the input value's <code>Id</code> property with the newly created record's ID. </p> <p> As an example, consider this User class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">User</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;FirstName&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;LastName&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> In order to create a new User in your database, you could define an API like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IUserRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;Create(<span style="color:#2b91af;">User</span>&nbsp;user); }</pre> </p> <p> An implementation of <code>IUserRepository</code> based on a relational database could perform an <code>INSERT</code> into the appropriate database, get the ID of the created record, and update the User object's <code>Id</code> property. </p> <p> This test snippet demonstrates that behaviour: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;u&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">User</span>&nbsp;{&nbsp;FirstName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Jane&quot;</span>,&nbsp;LastName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Doe&quot;</span>&nbsp;}; <span style="color:#2b91af;">Assert</span>.Equal(0,&nbsp;u.Id); repository.Create(u); <span style="color:#2b91af;">Assert</span>.NotEqual(0,&nbsp;u.Id);</pre> </p> <p> When you create the <code>u</code> object, by not assigning a value to the <code>Id</code> property, it will have the default value of 0. Only after the <code>Create</code> method returns does <code>Id</code> hold a proper value. </p> <p> <strong>Evaluation</strong> </p> <p> Does this design adhere to CQS? Yes, it does. The <code>Create</code> method doesn't return a value, but rather changes the state of the system. Not only does it create a record in your database, but it also changes the state of the <code>u</code> object. Nowhere does CQS state that an operation can't change more than a single part of the system. </p> <p> Is it good design, then? I don't think that it is. </p> <p> First, the design violates another part of encapsulation: protection of invariants. The invariants of any Entity is that is has an ID. It is the single defining feature of Entities that they have enduring and stable identities. When you change the identity of an Entity, it's no longer the same Entity. Yet this design allows exactly that: </p> <p> <pre>u.Id&nbsp;=&nbsp;42; <span style="color:green;">//&nbsp;...</span> u.Id&nbsp;=&nbsp;1337;</pre> </p> <p> Second, such a design puts considerable trust in the implicit protocol between any client and the implementation of <code>IUserRepository</code>. Not only must the <code>Create</code> method save the <code>User</code> object in a data store, but it must also update the <code>Id</code>. </p> <p> What happens, though, if you replay the method call: </p> <p> <pre>repository.Create(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">User</span>&nbsp;{&nbsp;FirstName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Ada&quot;</span>,&nbsp;LastName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Poe&quot;</span>&nbsp;}); repository.Create(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">User</span>&nbsp;{&nbsp;FirstName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Ada&quot;</span>,&nbsp;LastName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Poe&quot;</span>&nbsp;});</pre> </p> <p> This will result in duplicated entries, because the repository can't detect whether this is a replay, or simply two new users with the same name. You may find this example contrived, but in these days of cloud-based storage, it's common to apply retry strategies to clients. </p> <p> If you use <a href="http://blog.ploeh.dk/2014/08/11/cqs-versus-server-generated-ids">one of the alternatives I previously outlined</a>, you will not have this problem. </p> <p> <strong>Invariants</strong> </p> <p> Even if you don't care about replays or duplicates, you should still consider the invariants of Entities. As a minimum, you shouldn't be able to change the ID of an Entity. For a class like <code>User</code>, it'll also make client developers' job easier if it can provide some useful guarantees. This is part of Postel's law applied: be conservative in what you send. In this case, at least guarantee that no values will be null. Thus, a better (but by no means perfect) User class could be: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">User</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;User(<span style="color:blue;">int</span>&nbsp;id) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(id&nbsp;&lt;=&nbsp;0) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentOutOfRangeException</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;The&nbsp;ID&nbsp;must&nbsp;be&nbsp;a&nbsp;(unique)&nbsp;positive&nbsp;value.&quot;</span>);; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.Id&nbsp;=&nbsp;id; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.firstName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.lastName&nbsp;=&nbsp;<span style="color:#a31515;">&quot;&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">string</span>&nbsp;firstName;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;FirstName &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.firstName;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">set</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">value</span>&nbsp;==&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(<span style="color:blue;">value</span>)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.firstName&nbsp;=&nbsp;<span style="color:blue;">value</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">string</span>&nbsp;lastName; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;LastName &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>.lastName;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">set</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">value</span>&nbsp;==&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(<span style="color:blue;">value</span>)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.lastName&nbsp;=&nbsp;<span style="color:blue;">value</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The constructor initialises all class fields, and ensure that <code>Id</code> is a positive integer. It can't ensure that the ID is unique, though - at least, not without querying the database, which would introduce other problems. </p> <p> How do you create a new <code>User</code> object and save it in the database, then? </p> <p> One option is this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IUserRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;Create(<span style="color:blue;">string</span>&nbsp;firstName,&nbsp;<span style="color:blue;">string</span>&nbsp;lastName); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">User</span>&nbsp;Read(<span style="color:blue;">int</span>&nbsp;id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;Update(<span style="color:#2b91af;">User</span>&nbsp;user); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;Delete(<span style="color:blue;">int</span>&nbsp;id); }</pre> </p> <p> The <code>Create</code> method doesn't use the <code>User</code> class at all, because at this time, the Entity doesn't yet exist. It only exists once it has an ID, and in this scenario, this only happens once the database has stored it and assigned it an ID. </p> <p> Other methods can still use the <code>User</code> class as either input or output, because once the Entity has an ID, its invariants are satisfied. </p> <p> There are other problems with this design, though, so I still prefer <a href="http://blog.ploeh.dk/2014/08/11/cqs-versus-server-generated-ids">the alternatives I originally sketched</a>. </p> <p> <strong>Conclusion</strong> </p> <p> Proper object-oriented design should, as a bare minimum, consider <em>encapsulation</em>. This includes protecting the invariants of objects. For Entities, it means that an Entity must always have a stable and enduring identity. Well-designed Entities guarantee that identity values are always present and immutable. This requirement tend to be at odds with most popular Object-Relational Mappers (ORM), which require that ID fields are externally assignable. In order to use ORMs, most programmers compromise on encapsulation, ending up with systems that are relational in nature, but far from object-oriented. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="http://enterprisecraftsmanship.com/">Vladimir Khorikov</a></div> <div class="comment-content"> <p> >" This includes protecting the invariants of objects. For Entities, it means that an Entity must always have a stable and enduring identity." </p> <p> Great points. </p> <p> However, how would you get from <em>void Create(string firstName, string lastName)</em> to <em>User Read(int id)</em> especially if an entity doesn't have a natural key? </p> </div> <div class="comment-date">2016-05-06 18:26 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Vladimir, thank you for writing. Exactly how you get the ID depends on the exact requirements of your application. As an example, if you're developing a web application, you may simply be creating the user, and then that's the end of the HTTP request. Meanwhile, asynchronously, a background process is performing the actual database insertion and subsequently sends an email to the user with a link to click in order to verify the email. In that link is the generated ID, so when your server receives the next request, you already know the ID. </p> <p> That may not always be a practical strategy, but I started to describe this technique, because my experience with CQRS and REST tells me that you have lots of options for communicating state and identity. Still, if you need an ID straight away, you can always pass a GUID as a correlation ID, so that you can later find the Entity you created. That's the technique I <a href="http://blog.ploeh.dk/2014/08/11/cqs-versus-server-generated-ids">originally described</a>. </p> </div> <div class="comment-date">2016-05-06 19:04 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://enterprisecraftsmanship.com/">Vladimir Khorikov</a></div> <div class="comment-content"> <p> Thank you for the reply. </p> <p> BTW, ORMs don't require you to make Ids externally assignable. You can have ORMs assign Ids for you internally, and that's a preferred design in most cases. Also, you can make the Id setters non-public and thus protect entities' encapsulation. There still are issues, of course. Namely, entities don't have established identities until they are saved to the DB or added to the Context/Session. But this issue isn't that big comparing to what you described in the post. </p> </div> <div class="comment-date">2016-05-06 19:25 UTC</div> </div> <div class="comment"> <div class="comment-author">Ethan Nelson</div> <div class="comment-content"> <p> I was skimming through the comments and had this nagging suspicion that the entire topic of, "how do I get my ID with CQS", was a false dichotomy. The premise that the "ID" stored in the database is the "identity" of the object or entity I think can be challenged. </p> <p> I prefer to look at the "ID" as purely database implementation details. Assuming the repository is SQL, the key type preferred by db admins will be the smallest-reasonable monotonically increasing ID... namely INT. But switch to Amazon DynamoDB and that conclusion flies out the window. With their totally managed and distributed indexes and hashing algorithms... GUID's would be fantastic. </p> <p> What I WISH developers would do is take the time to think about their Domain and form their own understanding of what truly defines the identity of the entity. Whatever decision they make, it translates to a "natural key" for persistence. </p> <p> Natural keys are great because a developer has promised, "these combinations of properties uniquely identify entities of type T". The result is unique indexes that can have dramatic performance benefits... outside the application benefit of being able to construct your repository like this: <pre style="overflow:auto;font-family:consolas;font-size:13px;padding:9.5px;margin-top:0px;margin-bottom:10px;line-height:1.42857;word-break:break-all;word-wrap:break-word;border:1px solid rgb(204,204,204);border-radius:4px;background-color:rgb(245,245,245)"><span style="color:blue">public</span>&nbsp;<span style="color:blue">interface</span>&nbsp;<span style="color:rgb(43,145,175)">IRepository</span>&lt;<wbr>T&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue">void</span>&nbsp;Create(T&nbsp;item); }</pre> <p> As for the "ID" field? Who cares?... that's up to the database. The developer already knows exactly how to find this T. I wouldn't even include it on the domain model object. </p> <p> This would give db admins options. We can look at the natural key size, evaluate how it is queried and order it appropriately. We can decide if the nature of typical joins merit clustering (as is common with Header -> Detail style records). If needed, we can optimize lookup of large keys through stored procedures and computed persisted checksum columns. Nullability and default values become part of design discussions on the front end. Finally, if we feel like it... we can assign a surrogate key, or use a GUID. The developer doesn't care, our dba's do. </p> <p> Thinking back to my last 6 or so development projects, I've never had to "get the id". The next operation is always a matter of query or command on unique criteria I was responsible for creating in the first place. This approach makes my dba's happy. </p> </div> <div class="comment-date">2016-05-10 15:06 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Ethan, thank you for your comment; I agree with 90 percent of what you wrote, which is also the reason I prefer the options I <a href="http://blog.ploeh.dk/2014/08/11/cqs-versus-server-generated-ids">previously outlined</a>. </p> <p> Entities don't always have 'natural IDs', though, and even when they have, experience has taught me not to use them. Is a person's email address a natural ID? It is, but people change email addresses. </p> <p> Here in Denmark, for many years many public software systems have used the <a href="https://en.wikipedia.org/wiki/Personal_identification_number_(Denmark)">Danish personal identification numbers</a> as natural keys, as they were assumed to be nationally unique and stable. These numbers encode various information about a person, such as birth-date and sex. A few inhabitants, however, get sex change operations, and then desire a new number that corresponds to their new sex. Increasingly, these requests are being granted. New ID, same person. </p> <p> I've seen such changes happen so many times during my career that I've become distrustful of 'natural IDs'. If I need to identify an Entity, I usually attach a GUID to it. </p> </div> <div class="comment-date">2016-05-10 20:04 UTC</div> </div> <div class="comment"> <div class="comment-author">Ethan Nelson</div> <div class="comment-content"> <p> With regard to entity identification, I'm not sure stability matters. Who cares if the email address changes... as long as it is unique? Uniqueness is sufficient for the application to identify the entity. I agree from a strictly database standpoint; I'm not thrilled about assigning a primary key to a natural key due to the stability issue across relationships. But that does not invalidate the key for use as an identity in my mind. Both indexes can exist, both be unique, and each with their own purpose. </p> <p> Also, it is difficult for me to envision a scenario where there is no natural key. If there are two rows where the only difference is an auto-incrementing integer, or different guids... what is the meaning of one row versus the next? That meaning cannot, by definition, be contained in the surrogate keys... they add nothing to the business meaning of the entity. </p> <p> I feel I've derailed the discussion into database modeling theory, and we are talking about CQS with "create the entity" as a stimulating use case. Suffice it to say, IMHO, if (a, b) is unique today according to business rule, than it should be my unique identity from a developer perspective. By leaving row identifiers up to DBA's and dropping them from my domain model entirely, I minimize impact to those technical implementations when the business rules change such that (a, b, c) becomes the new unique, and, preserve CQS in the process. </p> </div> <div class="comment-date">2016-05-10 23:41 UTC</div> </div> <div class="comment"> <div class="comment-author">Endy Tjahjono</div> <div class="comment-content"> <p> SQL Server has a feature called 'sequence' (from 2012 and above if I'm not mistaken). We can use sequence to generate unique int ID instead of using identity column. The sequence can be called without inserting row to the table. We can create a service that call this sequence to generate the next running number. The UI code can call this running number generator service to get a new ID, and then send the new entity complete with ID to the create service. </p> </div> <div class="comment-date">2017-02-06 07:19:00 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Async as surrogate IO http://blog.ploeh.dk/2016/04/11/async-as-surrogate-io 2016-04-11T12:38:00+00:00 Mark Seemann <div id="post"> <p> <em>Haskell defines IO as an explicit context for handling effects. In F#, you can experiment with using Async as a surrogate.</em> </p> <p> As a reaction to <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">my article on the relationship between Functional architecture and Ports and Adapters</a>, <a href="http://simon-says-architecture.com">Szymon Pobiega</a> suggested on Twitter: <blockquote> <a href="https://twitter.com/SzymonPobiega/status/713001580077965312">random idea: in c# “IO Int” translates to “Task&lt;int&gt;”. By consistently using this we can make type system help you even in c#</a> </blockquote> If you know Haskell, you might object to that statement, but I don't think Szymon meant it literally. It's an idea worth exploring, I think. </p> <p> <strong>Async as an IO surrogate</strong> </p> <p> Functions in Haskell are, by default, <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a>. Whenever you need to do something impure, like querying a database or sending an email, you need to explicitly do this in an impure context: <code>IO</code>. Don't let the name <code>IO</code> mislead you: it's not only for input and output, but for all impure actions. As an example, random number generation is impure as well, because a function that returns a new value every time isn't pure. (There are ways to model random number generation with pure functions, but let's forget about that for now.) </p> <p> F#, on the other hand, doesn't make the distinction between pure and impure functions in its type system, so nothing 'translates' from <code>IO</code> in Haskell into an equivalent type in F#. Still, it's worth looking into Szymon's suggestion. </p> <p> In 2013, C# finally caught up with F#'s support for asynchronous work-flows, and since then, many IO-bound .NET APIs have gotten asynchronous versions. The underlying API for C#'s <code>async</code>/<code>await</code> support is the <a href="https://msdn.microsoft.com/en-us/library/dd460717">Task Parallel Library</a>, which isn't particularly constrained to IO-bound operations. The use of <code>async</code>/<code>await</code>, however, makes most sense for IO-bound operations. It improves resource utilisation because it can free up threads while waiting for IO-bound operations to complete. </p> <p> It seems to be a frequent observation that once you start using asynchronous operations, they tend to be 'infectious'. A common rule of thumb is <a href="https://msdn.microsoft.com/en-us/magazine/jj991977.aspx">async all the way</a>. You can easily call synchronous methods from asynchronous methods, but it's harder the other way around. </p> <p> This is similar to the distinction between pure and impure functions. You can call pure functions from impure functions, but not the other way around. If you call an impure function from a 'pure' function in F#, the caller also automatically becomes impure. In Haskell, it's not even possible; you can call a pure function from an <code>IO</code> context, but not the other way around. </p> <p> In F#, Async&lt;'a&gt; is more <a href="http://blog.ploeh.dk/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> than using Task&lt;T&gt;. While Async&lt;'a&gt; doesn't translate directly to Haskell's <code>IO</code> either, it's worth experimenting with the analogy. </p> <p> <strong>Async I/O</strong> </p> <p> In <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">the previous Haskell example</a>, communication with the database were implemented by functions returning <code>IO</code>: </p> <p> <pre>getReservedSeatsFromDB :: ConnectionString -> ZonedTime -> IO Int saveReservation :: ConnectionString -> Reservation -> IO ()</pre> </p> <p> By applying the analogy of <code>IO</code> to <code>Async&lt;'a&gt;</code>, you can implement your F# data access module to return Async work-flows: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;DateTimeOffset&nbsp;-&gt;&nbsp;Async&lt;int&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;(date&nbsp;:&nbsp;<span style="color:#4ec9b0;">DateTimeOffset</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;Async&lt;unit&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">saveReservation</span>&nbsp;connectionString&nbsp;(reservation&nbsp;:&nbsp;<span style="color:#4ec9b0;">Reservation</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> This makes it harder to compose the desired <code>imp</code> function, because you need to deal with both asynchronous work-flows and the Either monad. In Haskell, the building blocks are already there in the shape of the <code>EitherT</code> <a href="https://en.wikipedia.org/wiki/Monad_transformer">monad transformer</a>. F# doesn't have monad transformers, but in the spirit of the <a href="http://blog.ploeh.dk/2016/03/21/composition-with-an-either-computation-expression">previous article</a>, you can define a computation expression that combines Either and Async: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">AsyncEitherBuilder</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Async&lt;Result&lt;&#39;a,&#39;c&gt;&gt;&nbsp;*&nbsp;(&#39;a&nbsp;-&gt;&nbsp;Async&lt;Result&lt;&#39;b,&#39;c&gt;&gt;)</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;-&gt;&nbsp;Async&lt;Result&lt;&#39;b,&#39;c&gt;&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Bind</span>(x,&nbsp;<span style="color:navy;">f</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">async</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&#39;&nbsp;=&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x&#39;&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">return!</span>&nbsp;<span style="color:navy;">f</span>&nbsp;s &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;f&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;f&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;&#39;a&nbsp;-&gt;&nbsp;&#39;a</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">ReturnFrom</span>&nbsp;x&nbsp;=&nbsp;x <span style="color:blue;">let</span>&nbsp;asyncEither&nbsp;=&nbsp;<span style="color:#4ec9b0;">AsyncEitherBuilder</span>&nbsp;()</pre> </p> <p> This is the minimal implementation required for the following composition. A more complete implementation would also define a Return method, as well as <a href="https://msdn.microsoft.com/en-us/library/dd233182">other useful methods</a>. </p> <p> In the Haskell example, you saw how I had to use <code>hoistEither</code> and <code>liftIO</code> to 'extract' the values within the <code>do</code> block. This is necessary because sometimes you need to pull a value out of an Either value, and sometimes you need to get the value inside an <code>IO</code> context. </p> <p> In an <code>asyncEither</code> expression, you need similar functions: </p> <p> <pre><span style="color:green;">//&nbsp;Async&lt;&#39;a&gt;&nbsp;-&gt;&nbsp;Async&lt;Result&lt;&#39;a,&#39;b&gt;&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">liftAsync</span>&nbsp;x&nbsp;=&nbsp;<span style="color:blue;">async</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&#39;&nbsp;=&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;x&#39;&nbsp;} <span style="color:green;">//&nbsp;&#39;a&nbsp;-&gt;&nbsp;Async&lt;&#39;a&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">asyncReturn</span>&nbsp;x&nbsp;=&nbsp;<span style="color:blue;">async</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;x&nbsp;}</pre> </p> <p> I chose to name the first one <code>liftAsync</code> as a counterpart to Haskell's <code>liftIO</code>, since I'm using <code>Async&lt;'a&gt;</code> as a surrogate for <code>IO</code>. The other function I named <code>asyncReturn</code> because it returns a pure value wrapped in an asynchronous work-flow. </p> <p> You can now compose the desired <code>imp</code> function from the above building blocks: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;candidate&nbsp;=&nbsp;asyncEither&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span>&nbsp;candidate&nbsp;|&gt;&nbsp;<span style="color:navy;">asyncReturn</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;i&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">liftAsync</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r&nbsp;|&gt;&nbsp;<span style="color:navy;">asyncReturn</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return!</span>&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString&nbsp;r &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">liftAsync</span>&nbsp;}</pre> </p> <p> Notice how, in contrast with the <a href="http://blog.ploeh.dk/2016/03/21/composition-with-an-either-computation-expression">previous example</a>, this expression uses <code>let!</code> and <code>return!</code> throughout, but that you have to compose each line with <code>asyncReturn</code> or <code>liftAsync</code> in order to pull out the appropriate values. </p> <p> As an example, the original, unmodified <code>Validate.reservation</code> function has the type <code>ReservationRendition -&gt; Result&lt;Reservation, Error&gt;</code>. Inside an <code>asyncEither</code> expression, however, all <code>let!</code>-bound values must be of the type <code>Async&lt;Result&lt;'a, 'b&gt;&gt;</code>, so you need to wrap the return value <code>Result&lt;Reservation, Error&gt;</code> in Async. That's what <code>asyncReturn</code> does. Since this expression is <code>let!</code>-bound, the <code>r</code> value has the type <code>Reservation</code>. It's 'double-unwrapped', if you will. </p> <p> Likewise, <code>SqlGateway.getReservedSeats</code> returns a value of the type <code>Async&lt;int&gt;</code>, so you need to pipe it into <code>liftAsync</code> in order to turn it into an <code>Async&lt;Result&lt;int, Error&gt;&gt;</code>. When that value is <code>let!</code>-bound, then, <code>i</code> has the type <code>int</code>. </p> <p> There's one change compared to the the previous examples: the type of <code>imp</code> changed. It's now <code>ReservationRendition -&gt; Async&lt;Result&lt;unit, Error&gt;&gt;</code>. While you could write an adapter function that executes this function synchronously, it's better to recall the mantra: <em>async all the way!</em> </p> <p> <strong>Async Controller</strong> </p> <p> Instead of coercing the <code>imp</code> function to be synchronous, you can change the ReservationsController that uses it to be asynchronous as well: </p> <p> <pre><span style="color:green;">//&nbsp;ReservationRendition&nbsp;-&gt;&nbsp;Task&lt;IHttpActionResult&gt;</span> <span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>(rendition&nbsp;:&nbsp;<span style="color:#4ec9b0;">ReservationRendition</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;async&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;res&nbsp;=&nbsp;<span style="color:navy;">imp</span>&nbsp;rendition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;res&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationError</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span>&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">StatusCode</span>&nbsp;<span style="color:#4ec9b0;">HttpStatusCode</span>.Forbidden &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Async</span>.<span style="color:navy;">StartAsTask</span></pre> </p> <p> Notice that the method body uses an <code>async</code> computation expression, instead of the new <code>asyncEither</code>. This is because, when returning a result, you finally need to explicitly deal with the error cases as well as the success case. Using a standard <code>async</code> expression enables you to <code>let!</code>-bind <code>res</code> to the result of invoking the asynchronous <code>imp</code> function, and pattern match against it. </p> <p> Since ASP.NET Web API support asynchronous controllers, this works when you convert the async work-flow to a Task with <code>Async.StartAsTask</code>. <em>Async all the way</em>. </p> <p> In order to get this to work, I had to overload the BadRequest, StatusCode, and Ok methods, because they are protected, and you can't use protected methods from within a closure. Here's the entire code for the Controller, including the above Post method for completeness sake: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">ReservationsController</span>(<span style="color:navy;">imp</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;<span style="color:#4ec9b0;">ApiController</span>() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;<span style="color:blue;">private</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;(msg&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">base</span>.<span style="color:navy;">BadRequest</span>&nbsp;msg&nbsp;:&gt;&nbsp;<span style="color:#4ec9b0;">IHttpActionResult</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;<span style="color:blue;">private</span>&nbsp;this.<span style="color:navy;">StatusCode</span>&nbsp;statusCode&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">base</span>.<span style="color:navy;">StatusCode</span>&nbsp;statusCode&nbsp;:&gt;&nbsp;<span style="color:#4ec9b0;">IHttpActionResult</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;<span style="color:blue;">private</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;=&nbsp;<span style="color:blue;">base</span>.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;:&gt;&nbsp;<span style="color:#4ec9b0;">IHttpActionResult</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;ReservationRendition&nbsp;-&gt;&nbsp;Task&lt;IHttpActionResult&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Post</span>(rendition&nbsp;:&nbsp;<span style="color:#4ec9b0;">ReservationRendition</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">async</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;res&nbsp;=&nbsp;<span style="color:navy;">imp</span>&nbsp;rendition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;res&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;(<span style="color:navy;">ValidationError</span>&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">BadRequest</span>&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span>&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">StatusCode</span>&nbsp;<span style="color:#4ec9b0;">HttpStatusCode</span>.Forbidden &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">return</span>&nbsp;this.<span style="color:navy;">Ok</span>&nbsp;()&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Async</span>.<span style="color:navy;">StartAsTask</span></pre> </p> <p> As you can see, apart from the overloaded methods, the Post method is all there is. It uses the injected <code>imp</code> function to perform the actual work, asynchronously translates the result into a proper HTTP response, and returns it as a <code>Task&lt;IHttpActionResult&gt;</code>. </p> <p> <strong>Summary</strong> </p> <p> The initial idea was to experiment with using Async as a surrogate for IO. If you want to derive any value from this convention, you'll need to be disciplined and make all impure functions return Async&lt;'a&gt; - even those not IO-bound, but impure for other reasons (like random number generators). </p> <p> I'm not sure I'm entirely convinced that this would be a useful strategy to adopt, but on the other hand, I'm not deterred either. At least, the outcome of this experiment demonstrates that you can combine asynchronous work-flows with the Either monad, and that the code is still manageable, with good separation of concerns. Since it <em>is</em> a good idea to access IO-bound resources using asynchronous work-flows, it's nice to know that they can be combined with sane error handling. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment"> <div class="comment-author"><a href="http://diogocastro.com">Diogo Castro</a></div> <div class="comment-content"> <p> In my opinion, the true purpose of <code>IO</code> is to allow you to write <em>pure</em> effectful functions. For example, Haskell's <code>putStrLn</code> is referentially transparent - i.e., <code>[putStrLn "hello", putStrLn "hello"]</code> has the same meaning as <code>replicate 2 $ putStrLn "hello"</code>. This means you can treat a function that returns an IO the same way you'd treat any other function. </p> <p> This is possible for two reasons: 1) <code>IO</code> actions are not run until they hit the "edge of the universe" where <code>unsafePerformIO</code> is called and 2) they don't memoize the result - if an <code>IO</code> action is executed twice, it will calculate two possibly distinct values. </p> <p> The problem with <code>Task</code> is that it's not referentially transparent. </p> <p> <code>new[] { Task.Run(() => WriteLine("hello")), Task.Run(() => WriteLine("hello")) }</code> is not the same as <code>var t = Task.Run(() => WriteLine("hello"))</code> + <code>new [] { t, t }</code>. Even if you use <code>new Task()</code> instead of <code>Task.Run</code> to delay its execution, you still wouldn't be able to factor out the task because it memoizes the result (in this case, void), so you'd still see "hello" only once on the screen. </p> <p> Since Task cannot be used to model pure functions, I don't think it's a suitable replacement for IO. </p> </div> <div class="comment-date">2016-04-16 15:55 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Diogo, thank you for writing. The point about Tasks not being referentially transparent is well taken. You are, indeed, correct. </p> <p> F# asynchronous work-flows work differently, though. Extending the experiment outlined in this article, you can attempt to 'translate' <code>putStrLn</code> to F#. Since the return type of <code>putStrLn</code> is <code>IO ()</code>, you'll need a function that returns <code>Async&lt;unit&gt;</code>. Here's my naive attempt: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">aprintfn</span>&nbsp;fmt&nbsp;=&nbsp;<span style="color:blue;">async</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:navy;">printfn</span>&nbsp;fmt&nbsp;} </pre> </p> <p> Corresponding to your two Haskell examples, consider, then, these two F# values: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;explicitSeq&nbsp;=&nbsp;<span style="color:blue;">seq</span>&nbsp;{&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:navy;">aprintfn</span>&nbsp;<span style="color:#a31515;">&quot;hello&quot;</span>;&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:navy;">aprintfn</span>&nbsp;<span style="color:#a31515;">&quot;hello&quot;</span>&nbsp;} <span style="color:blue;">let</span>&nbsp;replicatedSeq&nbsp;=&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">replicate</span>&nbsp;2&nbsp;&lt;|&nbsp;<span style="color:navy;">aprintfn</span>&nbsp;<span style="color:#a31515;">&quot;hello&quot;</span></pre> </p> <p> Both of these values have the type <code>seq&lt;Async&lt;unit&gt;&gt;</code>, and I'd expect them to be equivalent. </p> <p> In Haskell, I used <a href="http://hackage.haskell.org/package/base-4.8.2.0/docs/Data-Traversable.html#v:sequence"><code>sequence</code></a> to run your code in GHCI, turning <code>[IO ()]</code> into <code>IO [()]</code>. In F#, you can do something similar: </p> <p> <pre><span style="color:green;">//&nbsp;seq&lt;Async&lt;&#39;a&gt;&gt;&nbsp;-&gt;&nbsp;Async&lt;seq&lt;&#39;a&gt;&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">asequence</span>&nbsp;xs&nbsp;=&nbsp;<span style="color:blue;">async</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:#4ec9b0;">Async</span>.<span style="color:navy;">RunSynchronously</span>&nbsp;xs&nbsp;}</pre> </p> <p> Using this function, you can evaluate both expressions: </p> <p> <pre>> asequence explicitSeq |> Async.RunSynchronously;; hello hello val it : seq<unit> = seq [null; null] > asequence replicatedSeq |> Async.RunSynchronously;; hello hello val it : seq<unit> = seq [null; null]</pre> </p> <p> As you can see, the result is the same in both cases, like you'd expect them to be. </p> <p> I'm still not claiming that <code>Async&lt;'a&gt;</code> is a direct translation of Haskell's <code>IO</code>, but there are similarities. Admittedly, I've never considered whether F# asynchronous work-flows satisfy the monad laws, but it wouldn't surprise me if they did. </p> </div> <div class="comment-date">2016-04-20 14:55 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://diogocastro.com">Diogo Castro</a></div> <div class="comment-content"> <p> Ah, I see! It wasn't clear to me that async workflows were referentially transparent. That being the case, it seems like they are, indeed, a good substitute for <code>IO a</code>. </p> <p> It seems like the F# team got async right :) The C# team should take a page from their book... </p> </div> <div class="comment-date">2016-05-04 23:05 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Composition with an Either computation expression http://blog.ploeh.dk/2016/03/21/composition-with-an-either-computation-expression 2016-03-21T08:34:00+00:00 Mark Seemann <div id="post"> <p> <em>An simple example of a composition using an F# computation expression.</em> </p> <p> In <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">a previous article</a>, you saw how to compose an application feature from functions: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span> &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date,&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">bind</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;(i,&nbsp;r)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString)</pre> </p> <p> This is awkward because of the need to carry the result of Validate.reservation (<code>r</code>) on to Capacity.check, while also needing it for SqlGateway.getReservedSeats. In this article, you'll see a neater, alternative syntax. </p> <p> <strong>Building blocks</strong> </p> <p> To recapitulate the context, you have this <code>Result</code> discriminated union from <a href="http://fsharpforfunandprofit.com">Scott Wlaschin's</a> article about <a href="http://fsharpforfunandprofit.com/posts/recipe-part2">railway-oriented programming</a>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Result</span>&lt;&#39;TSuccess,&#39;TFailure&gt;&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;TSuccess &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;TFailure</pre> </p> <p> A type with this structure is also sometimes known as <em>Either</em>, because the result can be either a success or a failure. </p> <p> The above example also uses bind and map functions, which are implemented this way: </p> <p> <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&#39;c&gt;)&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&#39;c&gt;&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&#39;c&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">bind</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">f</span>&nbsp;s &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;f&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;f <span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;Result&lt;&#39;a,&#39;c&gt;&nbsp;-&gt;&nbsp;Result&lt;&#39;b,&#39;c&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Success</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Success</span>(<span style="color:navy;">f</span>&nbsp;s) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Failure</span>&nbsp;f&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;f</pre> </p> <p> With these building blocks, it's trivial to implement a <a href="https://msdn.microsoft.com/en-us/library/dd233182.aspx">computation expression</a> for the type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">EitherBuilder</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Bind</span>(x,&nbsp;<span style="color:navy;">f</span>)&nbsp;=&nbsp;<span style="color:navy;">bind</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">ReturnFrom</span>&nbsp;x&nbsp;=&nbsp;x</pre> </p> <p> You can add more members to EitherBuilder if you need support for more advanced syntax, but this is the minimal implementation required for the following example. </p> <p> You also need a value of the EitherBuilder type: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;either&nbsp;=&nbsp;<span style="color:#4ec9b0;">EitherBuilder</span>&nbsp;() </pre> </p> <p> All this code is entirely generic, so you could put it (or a more comprehensive version) in a reusable library. </p> <p> <strong>Composition using the Either computation expression</strong> </p> <p> Since this Either computation expression is for general-purpose use, you can also use it to compose the above <code>imp</code> function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;candidate&nbsp;=&nbsp;either&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span>&nbsp;candidate &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;i&nbsp;=&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return!</span>&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString)&nbsp;}</pre> </p> <p> Notice how the <code>r</code> value can be used by both the SqlGateway.getReservedSeats and Capacity.check functions, because they are all within the same scope. </p> <p> This example looks more like the Haskell composition in my <a href="http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters">previous article</a>. It even looks cleaner in F#, but to be fair to Haskell, in F# you don't have to explicitly deal with IO as a monadic context, so that in itself makes things look simpler. </p> <p> <strong>A variation using shadowing</strong> </p> <p> You may wonder why I chose to use <code>return!</code> with an expression using the <code>map</code> function. One reason was that I wanted to avoid having to bind a named value to the result of calling Capacity.check. Naming is always difficult, and while I find the names <code>r</code> and <code>i</code> <a href="http://blog.ploeh.dk/2015/08/17/when-x-y-and-z-are-great-variable-names">acceptable because the scope is so small</a>, I prefer composing functions without having to declare intermediary values. </p> <p> Another reason was that I wanted to show you an example that resembles the previous Haskell example as closely as possible. </p> <p> You may, however, find the following variation more readable. In order to use <code>return</code> instead of <code>return!</code>, you need to furnish a <code>Return</code> method on your computation builder: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">EitherBuilder</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Bind</span>(x,&nbsp;<span style="color:navy;">f</span>)&nbsp;=&nbsp;<span style="color:navy;">bind</span>&nbsp;<span style="color:navy;">f</span>&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;this.<span style="color:navy;">Return</span>&nbsp;x&nbsp;=&nbsp;<span style="color:navy;">Success</span>&nbsp;x</pre> </p> <p> Nothing prevents you from adding both <code>Return</code> and <code>ReturnFrom</code> (as well as other) methods to the EitherBuilder class, but I'm showing you the minimal implementation required for the example to work. </p> <p> Using this variation of EitherBuilder, you can alternatively write the <code>imp</code> function like this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;candidate&nbsp;=&nbsp;either&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span>&nbsp;candidate &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;i&nbsp;=&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString&nbsp;r&nbsp;}</pre> </p> <p> In such an alternative, you can use a <code>let!</code>-bound value to capture the result of calling Capacity.check, but that obligates you to devise a name for the value. You'll often see names like <code>r'</code>, but I always feel uneasy when I resort to such naming. Here, I circumvented the problem by <em>shadowing</em> the original <code>r</code> value with a new value also called <code>r</code>. This is possible, and even desirable, because once the reservation has been checked, you <em>should</em> use that new, checked value, and not the original value. That's what the original composition, above, does, so shadowing is both equivalent and appropriate. </p> <p> <strong>Summary</strong> </p> <p> Computation expressions give you an opportunity to compose functions in a readable way. It's an alternative to using the underlying bind and map functions. Whether you prefer one over the other is partly subjective, but I tend to favour the underlying functions until they become inconvenient. The initial version of the <code>imp</code> function in this article is an example of map and bind becoming unwieldy, and where a computation expression affords a cleaner, more readable syntax. </p> <p> The disadvantage of computation expressions is that you need to use <code>return</code> or <code>return!</code> to return a value. That forces you to write an extra line of code that isn't necessary with a simple <code>bind</code> or <code>map</code>. </p> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Functional architecture is Ports and Adapters http://blog.ploeh.dk/2016/03/18/functional-architecture-is-ports-and-adapters 2016-03-18T08:53:00+00:00 Mark Seemann <div id="post"> <p> <em>Functional architecture tends to fall into a pit of success that looks a lot like Ports and Adapters.</em> </p> <p> In object-oriented architecture, we often struggle towards the ideal of the Ports and Adapters architecture, although we often <a href="http://blog.ploeh.dk/2013/12/03/layers-onions-ports-adapters-its-all-the-same">call it something else: layered architecture, onion architecture, hexagonal architecture, and so on</a>. The goal is to decouple the business logic from technical implementation details, so that we can vary each independently. </p> <p> This creates value because it enables us to manoeuvre nimbly, responding to changes in business or technology. </p> <p> <strong>Ports and Adapters</strong> </p> <p> The idea behind the Ports and Adapters architecture is that <em>ports</em> make up the boundaries of an application. A <em>port</em> is something that interacts with the outside world: user interfaces, message queues, databases, files, command-line prompts, etcetera. While the ports constitute the interface to the rest of the world, <em>adapters</em> translate between the ports and the application model. </p> <p> <img src="/content/binary/ports-and-adapters-conceptual-diagram.png" alt="A conceptual diagram of Ports and Adapters architecture: coloured boxes in concentric circles."> </p> <p> The word <em>adapter</em> is aptly chosen, because the role of the <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter design pattern</a> is exactly to translate between two different interfaces. </p> <p> You ought to arrive at some sort of variation of Ports and Adapters if you apply Dependency Injection, as <a href="http://blog.ploeh.dk/2013/12/03/layers-onions-ports-adapters-its-all-the-same">I've previously attempted to explain</a>. </p> <p> The problem with this architecture, however, is that it seems to take a lot of explaining: <ul> <li><a href="http://amzn.to/12p90MG">My book about Dependency Injection</a> is 500 pages long.</li> <li>Robert C. Martin's <a href="http://amzn.to/19W4JHk">book about the SOLID principles, package and component design, and so on</a> is 700 pages long.</li> <li><a href="http://amzn.to/WBCwx7">Domain-Driven Design</a> is 500 pages long.</li> <li>and so on...</li> </ul> In my experience, implementing a Ports and Adapters architecture is a <a href="https://en.wikipedia.org/wiki/Sisyphus">Sisyphean task</a>. It requires much diligence, and if you look away for a moment, the boulder rolls downhill again. </p> <p> <img src="/content/binary/sisyphos-boulder-rolling-downhill.png" alt="A schematic showing a ball rolling down a slope."> </p> <p> It's possible to implement a Ports and Adapters architecture with object-oriented programming, but it takes <em>so much effort</em>. Does it have to be that difficult? </p> <p> <strong>Haskell as a learning aid</strong> </p> <p> Someone recently asked me: <em>how do I know I'm being sufficiently Functional?</em> </p> <p> I was wondering that myself, so I decided to learn Haskell. Not that Haskell is the only Functional language out there, but it enforces <a href="https://en.wikipedia.org/wiki/Pure_function">purity</a> in a way that neither F#, Clojure, nor Scala does. In Haskell, a function <em>must</em> be pure, unless its type indicates otherwise. This forces you to be deliberate in your design, and to separate pure functions from functions with (side) effects. </p> <p> If you don't know Haskell, code with side effects can only happen inside of a particular 'context' called <code>IO</code>. It's a monadic type, but that's not the most important point. The point is that you can tell by a function's type whether or not it's pure. A function with the type <code>ReservationRendition -&gt; Either Error Reservation</code> is pure, because <code>IO</code> appears nowhere in the type. On the other hand, a function with the type <code>ConnectionString -&gt; ZonedTime -&gt; IO Int</code> is impure because its return type is <code>IO Int</code>. This means that the return value is an integer, but that this integer originates from a context where it could change between function calls. </p> <p> There's a fundamental distinction between a function that returns <code>Int</code>, and one that returns <code>IO Int</code>. Any function that returns <code>Int</code> is, in Haskell, <a href="https://en.wikipedia.org/wiki/Referential_transparency">referentially transparent</a>. This means that you're guaranteed that the function will always return the same value given the same input. On the other hand, a function returning <code>IO Int</code> doesn't provide such a guarantee. </p> <p> In Haskell programming, you should strive towards maximising the amount of pure functions you write, pushing the impure code to the edges of the system. A good Haskell program has a big core of pure functions, and a shell of <code>IO</code> code. Does that sound familiar? </p> <p> It basically means that Haskell's type system <em>enforces</em> the Ports and Adapters architecture. The <em>ports</em> are all your <code>IO</code> code. The application's core is all your pure functions. The type system automatically creates a <em>pit of success</em>. </p> <p> <img src="/content/binary/pit-of-success.png" alt="A 'boulder' lying in the bottom of a schematised pit of success."> </p> <p> Haskell is a great learning aid, because it forces you to explicitly make the distinction between pure and impure functions. You can even use it as a verification step to figure out whether your F# code is 'sufficiently Functional'. F# is a <em>Functional first</em> language, but it also allows you to write object-oriented or imperative code. If you write your F# code in a Functional manner, though, it's easy to translate to Haskell. If your F# code is difficult to translate to Haskell, it's probably because it isn't Functional. </p> <p> Here's an example. </p> <p> <strong>Accepting reservations in F#, first attempt</strong> </p> <p> In my <a href="http://bit.ly/tddwithfsharp">Test-Driven Development with F#</a> Pluralsight course (a <a href="http://www.infoq.com/presentations/mock-fsharp-tdd">free, condensed version is also available</a>), I demonstrate how to implement an HTTP API that accepts reservation requests for an on-line restaurant booking system. One of the steps when handling the reservation request is to check whether the restaurant has enough remaining capacity to accept the reservation. The function looks like this: </p> <p> <pre><span style="color:green;">//&nbsp;int</span> <span style="color:green;">//&nbsp;-&gt;&nbsp;(DateTimeOffset&nbsp;-&gt;&nbsp;int)</span> <span style="color:green;">//&nbsp;-&gt;&nbsp;Reservation</span> <span style="color:green;">//&nbsp;-&gt;&nbsp;Result&lt;Reservation,Error&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">check</span>&nbsp;capacity&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;reservation.Date &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;capacity&nbsp;&lt;&nbsp;reservation.Quantity&nbsp;+&nbsp;reservedSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;reservation</pre> </p> <p> As the comment suggests, the second argument, <code>getReservedSeats</code>, is a function of the type <code>DateTimeOffset -&gt; int</code>. The <code>check</code> function calls this function to retrieve the number of already reserved seats on the requested date. </p> <p> When unit testing, you can supply a pure function as a Stub; for example: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;_&nbsp;=&nbsp;0 <span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;capacity&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;reservation</pre> </p> <p> When finally composing the application, instead of using a pure function with a hard-coded return value, you can compose with an impure function that queries a database for the desired information: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span> &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">bind</span>&nbsp;(<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString)) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString)</pre> </p> <p> Here, <code>SqlGateway.getReservedSeats connectionString</code> is a partially applied function, the type of which is <code>DateTimeOffset -&gt; int</code>. In F#, you can't tell by its type that it's impure, but I know that this is the case because I wrote it. It queries a database, so isn't referentially transparent. </p> <p> This works well in F#, where it's up to you whether a particular function is pure or impure. Since that <code>imp</code> function is composed in the application's <a href="http://blog.ploeh.dk/2011/07/28/CompositionRoot">Composition Root</a>, the impure functions SqlGateway.getReservedSeats and SqlGateway.saveReservation are only pulled in at the edge of the system. The rest of the system is nicely protected against side-effects. </p> <p> It feels Functional, but is it? </p> <p> <strong>Feedback from Haskell</strong> </p> <p> In order to answer that question, I decided to re-implement the central parts of this application in Haskell. My first attempt to check the capacity was this direct translation: </p> <p> <pre>checkCapacity :: Int -&gt; (ZonedTime -&gt; Int) -&gt; Reservation -&gt; Either Error Reservation checkCapacity capacity getReservedSeats reservation = let reservedSeats = getReservedSeats $ date reservation in if capacity &lt; quantity reservation + reservedSeats then Left CapacityExceeded else Right reservation</pre> </p> <p> This compiles, and at first glance seems promising. The type of the <code>getReservedSeats</code> function is <code>ZonedTime -&gt; Int</code>. Since <code>IO</code> appears nowhere in this type, Haskell guarantees that it's pure. </p> <p> On the other hand, when you need to implement the function to retrieve the number of reserved seats from a database, this function must, by its very nature, be impure, because the return value could change between two function calls. In order to enable that in Haskell, the function must have this type: </p> <p> <pre>getReservedSeatsFromDB :: ConnectionString -&gt; ZonedTime -&gt; IO Int</pre> </p> <p> While you can partially apply the first ConnectionString argument, the return value is <code>IO Int</code>, not <code>Int</code>. </p> <p> A function with the type <code>ZonedTime -&gt; IO Int</code> isn't the same as <code>ZonedTime -&gt; Int</code>. Even when executing inside of an IO context, you can't convert <code>ZonedTime -&gt; IO Int</code> to <code>ZonedTime -&gt; Int</code>. </p> <p> You can, on the other hand, <em>call</em> the impure function inside of an IO context, and extract the <code>Int</code> from the <code>IO Int</code>. That doesn't quite fit with the above checkCapacity function, so you'll need to reconsider the design. While it was 'Functional enough' for F#, it turns out that this design isn't <em>really</em> Functional. </p> <p> If you consider the above checkCapacity function, though, you may wonder why it's necessary to pass in a function in order to determine the number of reserved seats. Why not simply pass in this number instead? </p> <p> <pre>checkCapacity :: Int -&gt; Int -&gt; Reservation -&gt; Either Error Reservation checkCapacity capacity reservedSeats reservation = if capacity &lt; quantity reservation + reservedSeats then Left CapacityExceeded else Right reservation</pre> </p> <p> That's much simpler. At the edge of the system, the application executes in an IO context, and that enables you to compose the pure and impure functions: </p> <p> <pre>import Control.Monad.Trans (liftIO) import Control.Monad.Trans.Either (EitherT(..), hoistEither) postReservation :: ReservationRendition -&gt; IO (HttpResult ()) postReservation candidate = fmap toHttpResult $ runEitherT $ do r &lt;- hoistEither $ validateReservation candidate i &lt;- liftIO $ getReservedSeatsFromDB connStr $ date r hoistEither $ checkCapacity 10 i r &gt;&gt;= liftIO . saveReservation connStr</pre> </p> <p> (Complete source code is available <a href="https://gist.github.com/ploeh/c999e2ae2248bd44d775">here</a>.) </p> <p> Don't worry if you don't understand all the details of this composition. The highlights are these: </p> <p> The postReservation function takes a ReservationRendition (think of it as a JSON document) as input, and returns an <code>IO (HttpResult ())</code> as output. The use of <code>IO</code> informs you that this entire function is executing within the IO monad. In other words: it's impure. This shouldn't be surprising, since this is the edge of the system. </p> <p> Furthermore, notice that the function <code>liftIO</code> is called twice. You don't have to understand exactly what it does, but it's necessary to use in order to 'pull out' a value from an <code>IO</code> type; for example pulling out the <code>Int</code> from an <code>IO Int</code>. This makes it clear where the pure code is, and where the impure code is: the liftIO function is applied to the functions getReservedSeatsFromDB and saveReservation. This tells you that these two functions are impure. By exclusion, the rest of the functions (validateReservation, checkCapacity, and toHttpResult) are pure. </p> <p> It's interesting to observe how you can interleave pure and impure functions. If you squint, you can almost see how the data flows from the pure validateReservation function, to the impure getReservedSeatsFromDB function, and then both output values (<code>r</code> and <code>i</code>) are passed to the pure checkCapacity function, and finally to the impure saveReservation function. All of this happens within an <code>(EitherT Error IO) () do</code> block, so if any of these functions return <code>Left</code>, the function short-circuits right there and returns the resulting error. See e.g. Scott Wlaschin's excellent article on <a href="http://fsharpforfunandprofit.com/posts/recipe-part2">railway-oriented programming</a> for an exceptional, lucid, clear, and visual introduction to the Either monad. </p> <p> The value from this expression is composed with the built-in runEitherT function, and again with this pure function: </p> <p> <pre>toHttpResult :: Either Error () -&gt; HttpResult () toHttpResult (Left (ValidationError msg)) = BadRequest msg toHttpResult (Left CapacityExceeded) = StatusCode Forbidden toHttpResult (Right ()) = OK ()</pre> </p> <p> The entire postReservation function is impure, and sits at the edge of the system, since it handles IO. The same is the case of the getReservedSeatsFromDB and saveReservation functions. I deliberately put the two database functions in the bottom of the below diagram, in order to make it look more familiar to readers used to looking at layered architecture diagrams. You can imagine that there's a cylinder-shaped figure below the circles, representing a database. </p> <p> <img src="/content/binary/haskell-post-reservation-ports-adapters.png" alt="Haskell solution with functions placed in a Ports and Adapters diagram."> </p> <p> You can think of the validateReservation and toHttpResult functions as belonging to the <em>application model</em>. While pure functions, they translate between the external and internal representation of data. Finally, the checkCapacity function is part of the application's Domain Model, if you will. </p> <p> Most of the design from my first F# attempt survived, apart from the Capacity.check function. Re-implementing the design in Haskell has taught me an important lesson that I can now go back and apply to my F# code. </p> <p> <strong>Accepting reservations in F#, even more Functionally</strong> </p> <p> Since the required change is so little, it's easy to apply the lesson learned from Haskell to the F# code base. The culprit was the Capacity.check function, which ought to instead be implemented like this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">check</span>&nbsp;capacity&nbsp;reservedSeats&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;capacity&nbsp;&lt;&nbsp;reservation.Quantity&nbsp;+&nbsp;reservedSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Failure</span>&nbsp;<span style="color:navy;">CapacityExceeded</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;reservation</pre> </p> <p> This simplifies the implementation, but makes the composition slightly more involved: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span> &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;r.Date,&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">bind</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;(i,&nbsp;r)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString)</pre> </p> <p> This almost looks more complicated than the Haskell function. Haskell has the advantage that you can automatically use any type that implements the <code>Monad</code> typeclass inside of a <code>do</code> block, and since <code>(EitherT Error IO) ()</code> is a Monad instance, the <code>do</code> syntax is available for free. </p> <p> You could do something similar in F#, but then you'd have to implement a custom computation expression builder for the Result type. Perhaps I'll do this in <a href="http://blog.ploeh.dk/2016/03/21/composition-with-an-either-computation-expression">a later blog post</a>... </p> <p> <strong>Summary</strong> </p> <p> Good Functional design is equivalent to the Ports and Adapters architecture. If you use Haskell as a yardstick for 'ideal' Functional architecture, you'll see how its explicit distinction between pure and impure functions creates a pit of success. Unless you write your entire application to execute <em>within</em> the <code>IO</code> monad, Haskell will automatically enforce the distinction, and push all communication with the external world to the edges of the system. </p> <p> Some Functional languages, like F#, don't explicitly enforce this distinction. Still, in F#, it's easy to <em>informally</em> make the distinction and compose applications with impure functions pushed to the edges of the system. While this isn't enforced by the type system, it still feels natural. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment"> <div class="comment-author"><a href="https://baks.github.io">Arkadiusz K</a></div> <div class="comment-content"> <p>Great post as always. However, I saw that you are using <em></em>IO monad</em>. I wonder if you shouldn't use <em>IO action</em> instead? To be honest, I'm not proficient in Haskell. But according to that <a href="http://blog.jle.im/entry/io-monad-considered-harmful.html">post</a> they are encouraging to use <em>IO action</em> rather than </em>IO monad</em> if we are not talking about monadic properties.</p> </div> <div class="comment-date">2016-03-18 23:00 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Arkadiusz, thank you for writing. It's the first time I see <a href="http://blog.jle.im/entry/io-monad-considered-harmful.html">that post</a>, but if I understand the argument correctly, it argues against the (over)use of the monad <em>terminology</em> when talking about IO actions that aren't intrinsically monadic in nature. As far as I can tell, it doesn't argue against the use of the <code>IO</code> <em>type</em> in Haskell. </p> <p> It seems a reasonable point that it's unhelpful to newcomers to throw them straight into a discussion about monads, when all they want to see is a <em>hello world</em> example. </p> <p> As some of the comments to that post point out, however, you'll soon need to compose IO actions in Haskell, and it's the monadic nature of the <code>IO</code> type that enables the use of <code>do</code> blocks. (..or, if you want to be more specific, it seems that sometimes, all you need is a Functor. It hardly helps to talk about the <em>IO functor</em> instead of the <em>IO monad</em>, though...) </p> <p> I don't claim to be a Haskell expert, so there may be more subtle nuances in that article that simply went over my head. Still, within my current understanding, the discussion in this particular post of mine does relate to the IO <em>monad</em>. </p> <p> When I wrote the article, I chose my words with some caution. Notice that when I introduce the <code>IO</code> type my article, I mostly talk about it as a 'context'. </p> <p> When it comes to the discussion about the postReservation function, however, I pull out the stops and increasingly use the word <em>monad</em>. The reason is that this composition wouldn't be possible without the monadic properties of <code>IO</code>. Most of this function executes within a monadic 'stack': <code>EitherT Error IO</code>. EitherT is a monad transformer, and in order to be useful (e.g. composable in <code>do</code> blocks), the type it transforms must be a monad as well (or perhaps, as hinted above, a functor would be sufficient). </p> <p> I agree with Justin Le's article that overuse of <em>monad</em> is likely counterproductive. On the other hand, one can also fall into the opposite side. There are some educators who seem to avoid the word at any cost. That's not my policy. I try to use as precise language as possible. That means that I'll use <em>monad</em> when I talk about monads, and I'll avoid it when it isn't necessary. That's my goal, but I may not have achieved it. Do you think I used it incorrectly here? </p> </div> <div class="comment-date">2016-03-19 08:06 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://baks.github.io">Arkadiusz K</a></div> <div class="comment-content"> <p>Mark, thank you for your comprehensive explanation. I was rather just asking you about opinion on this topic. I didn't try to claim that you had used it incorrectly. Now, I see that as you wrote you are first talking about IO context and then about IO monad when it comes to use monadic properties.</p> </div> <div class="comment-date">2016-03-19 10:30 UTC</div> </div> <div class="comment"> <div class="comment-author">John Dailey</div> <div class="comment-content"> <p> Hello, thanks so much for your website and pluralsight library -- it's been a tremendous help to me getting started with F# and functional programming (and TDD to a degree). I'm having some problems with the big picture question of how to structure applications, and this post is leaving me even more confused. </p> <p> I can't understand how some of the original code, and especially the changes made in this post could possibly scale to a larger or more complicated application. Many real-life domain operations are going to need to query some data, make decisions based on that data, map that data into different domain objects, persist something, and so on. </p> <p> It seems like the end result is that we end up copying much of the meaningful internal logic of the domain out to this boundary area in the Web API project. In this code we've extracted: <ul> <li>The logic of only checking existing reservations on the same day as the day on the request</li> <li>The logic of not saving a new reservation if the request failed the capacity check</li> </ul> </p> <p> Does that logic not properly belong 'behind the wall' inside the domain? I would have expected to see a simple 'makeReserveration r' function as the public face to the domain. Suppose the restaurant also had a desktop/kiosk application in their building to manage their reservation system. Wouldn't we now be forced to duplicate this internal logic in the Composition Root of that application? </p> <p> What I instinctively want to gravitate towards is something like this: </p> <p> <pre><span style="color:blue;">let internal</span>&nbsp;<span style="color:navy;">makeReservation' getReservedSeats saveReservation check r</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">(getReservedSeats</span>&nbsp;r.Date,&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;|>&nbsp;<span style="color:navy;">bind</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;(i,&nbsp;r)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">check</span>&nbsp;10&nbsp;i&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;|>&nbsp;<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">saveReservation</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">makeReservation</span>&nbsp;= makeReservation' SqlGateway.getReservedSeats SqlGateway.saveReservation Capacity.check </pre> </p> <p> With this, the SqlGateway code needs be visible to the reservation code, putting it at the center of the app not on the edges. However, are there any practical disadvantages to this if the code is disciplined to only use it within this sort of dependency injection? What problems would I be likely to run into? </p> <p> At least as a beginner, this seems to me to have a couple advantages: <ul> <li>From the perspective of the unit tests (which test makeReservation') it is a pure function, as you noted above. Testability is no problem here.</li> <li>Code at the UI/Presentation level is nice and tight -- just make sure you can successfully transform the JSON request into a domain request, then process the command result.</li> </ul> </p> <p> But most notable to me, dependencies are 'injected' right where they were needed. Imagine a much more complicated aggregate operation like 'performOvernightCloseout' that needs to do many things like read cash register totals and send them to accounting, send timeclock info to HR, write order statistics to a reporting database, check inventory and create new orders for the following day. Of course we'd break this down into a hierarchy of functions, with top level orchestration calling down into increasingly specific functions, no function individually being too complicated. </p> <p> The original demo application would have to approach this scenario by passing in a huge number of functions to the operation root, which then parcels them out level by level where they are needed. This would seem very brittle since any change in a low level function would require changes to the parameter list all the way up. </p> <p> With newer code shown in this post, it would seem impossible. There might be a handful of small functions that can be in the central library, but the bulk of the logic is going to need to be extracted into a now-500-line imp method in the Web API project, or whatever service is launching this operation. </p> </div> <div class="comment-date">2016-03-24 23:30 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> John, thank you for writing. I'm not sure I can answer everything to your satisfaction in single response, but we can always view this as a beginning of longer interchange. Some of the doubts you express are ambiguous, so I'm not sure how to interpret them, but I'll do my best to be as explicit as possible. </p> <p> In the beginning of your comment, you write: <blockquote> "Many real-life domain operations are going to need to query some data, make decisions based on that data, map that data into different domain objects, persist something, and so on." </blockquote> This is exactly the reason why I like the reservation request example so much, because it does all of that. It queries data, because it reads the number of already reserved seats from a database. It makes decisions based on that data, because it decides whether or not it can accept the reservation based on the number of remaining seats. It maps from an external JSON (or XML) data format to a Reservation record type (belonging to the Domain Model, if you will); it also maps back to an HTTP response. Finally, the example also saves the reservation to the database, if it was accepted. </p> <p> When you say that we've extracted the logic, it's true, but I'm not sure what you mean by "behind the wall", but let's look at each of your bullet points in turn. </p> <p> <strong>Querying existing reservations</strong> </p> <p> You write that I've extracted the "logic of only checking existing reservations on the same day as the day on the request". It's a simple database query. In F#, one implementation may look like this: </p> <p> <pre><span style="color:green;">//&nbsp;In&nbsp;module&nbsp;SqlGateway:</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString&nbsp;(date&nbsp;:&nbsp;<span style="color:#4ec9b0;">DateTimeOffset</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;min&nbsp;=&nbsp;<span style="color:#4ec9b0;">DateTimeOffset</span>(date.Date&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;,&nbsp;date.Offset) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;max&nbsp;=&nbsp;<span style="color:#4ec9b0;">DateTimeOffset</span>(date.Date.<span style="color:navy;">AddDays</span>&nbsp;1.,&nbsp;date.Offset) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:#a31515;">&quot;</span> <span style="color:#a31515;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SELECT&nbsp;ISNULL(SUM(Quantity),&nbsp;0)</span> <span style="color:#a31515;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FROM&nbsp;[dbo].[Reservations]</span> <span style="color:#a31515;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WHERE&nbsp;@min&nbsp;&lt;=&nbsp;Date&nbsp;AND&nbsp;Date&nbsp;&lt;&nbsp;@max&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#4ec9b0;">SqlConnection</span>(connectionString) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#4ec9b0;">SqlCommand</span>(sql,&nbsp;conn) &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.<span style="color:navy;">AddWithValue</span>(<span style="color:#a31515;">&quot;@min&quot;</span>,&nbsp;min)&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span> &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.<span style="color:navy;">AddWithValue</span>(<span style="color:#a31515;">&quot;@max&quot;</span>,&nbsp;max)&nbsp;|&gt;&nbsp;<span style="color:navy;">ignore</span> &nbsp;&nbsp;&nbsp;&nbsp;conn.<span style="color:navy;">Open</span>() &nbsp;&nbsp;&nbsp;&nbsp;cmd.<span style="color:navy;">ExecuteScalar</span>()&nbsp;:?&gt;&nbsp;<span style="color:#4ec9b0;">int</span></pre> </p> <p> In my opinion, there's no <em>logic</em> in this function, but this is an example where terminology can be ambiguous. To me, however, <em>logic</em> implies that decisions are being made. That's not the case here. This getReservedSeats function has a cyclomatic complexity of 1. </p> <p> This function is an Adapter: it adapts the ADO.NET SDK (the <em>port</em> to SQL Server) to something the Domain Model can use. In this case, the answer is a simple integer. </p> <p> <strong>To save, or not to save</strong> </p> <p> You also write that I've extracted out the "logic of not saving a new reservation if the request failed the capacity check". Yes, that logic is extracted out to the <code>check</code> function above. Since that's a pure function, it's part of the Domain Model. </p> <p> The function that saves the reservation in the database, again, contains no logic (as I interpret the word <em>logic</em>): </p> <p> <pre><span style="color:green;">//&nbsp;In&nbsp;module&nbsp;SqlGateway:</span> <span style="color:blue;">let</span>&nbsp;saveReservation&nbsp;connectionString&nbsp;(reservation&nbsp;:&nbsp;Reservation)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;sql&nbsp;=&nbsp;<span style="color:#a31515;">&quot;</span> <span style="color:#a31515;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INSERT&nbsp;INTO&nbsp;Reservations(Date,&nbsp;Name,&nbsp;Email,&nbsp;Quantity)</span> <span style="color:#a31515;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VALUES(@Date,&nbsp;@Name,&nbsp;@Email,&nbsp;@Quantity)&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;conn&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;SqlConnection(connectionString) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;cmd&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;SqlCommand(sql,&nbsp;conn) &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.AddWithValue(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;reservation.Date)&nbsp;|&gt;&nbsp;ignore &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.AddWithValue(<span style="color:#a31515;">&quot;@Name&quot;</span>,&nbsp;reservation.Name)&nbsp;|&gt;&nbsp;ignore &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.AddWithValue(<span style="color:#a31515;">&quot;@Email&quot;</span>,&nbsp;reservation.Email)&nbsp;|&gt;&nbsp;ignore &nbsp;&nbsp;&nbsp;&nbsp;cmd.Parameters.AddWithValue(<span style="color:#a31515;">&quot;@Quantity&quot;</span>,&nbsp;reservation.Quantity)&nbsp;|&gt;&nbsp;ignore &nbsp;&nbsp;&nbsp;&nbsp;conn.Open() &nbsp;&nbsp;&nbsp;&nbsp;cmd.ExecuteNonQuery()&nbsp;|&gt;&nbsp;ignore</pre> </p> <p> Due to the way the Either monad works, however, this function is only going to be called if all previous functions returned Success (or Right in Haskell). That logic is an entirely reusable abstraction. It's also part of the glue that composes functions together. </p> <p> <strong>Reuse of composition</strong> </p> <p> Further down, you ask: <blockquote> "Suppose the restaurant also had a desktop/kiosk application in their building to manage their reservation system. Wouldn't we now be forced to duplicate this internal logic in the Composition Root of that application?" </blockquote> That's an excellent question! I'm glad you asked. </p> <p> A desktop or kiosk application is noticeably different from a web application. It may have a different application flow, and it certainly exposes a different interface to the outside world. Instead of handling incoming HTTP requests, and translating results to HTTP responses, it'll need to respond to click events and transform results to on-screen UI events. This means that validation may be different, and that you don't need to map results back to HTTP response values. Again, this may be clearer in the Haskell implementation, where the <code>toHttpResult</code> function would obviously be superfluous. Additionally, a UI may present a native date and time picker control, which can directly produce strongly typed date and time values, negating the need for validation. </p> <p> Additionally, a kiosk may need to be able to work in off-line, or occasionally connected, mode, so perhaps you'd prefer a different data store implementation. Instead of connecting directly to a database, such an application would need to write to a message queue, and read from an off-line snapshot of reservations. </p> <p> That's a sufficiently different application that it warrants creating applications from <a href="http://blog.ploeh.dk/2011/06/07/SOLIDCodeisnt">fine-grained building blocks instead of course-grained building blocks</a>. Perhaps the only part you'll end up reusing is the core business logic contained in the check[Capacity] function. <a href="http://blog.ploeh.dk/2015/01/06/composition-root-reuse">Composition Roots aren't reusable</a>. </p> <p> (As an aside, you may think that if the check[Capacity] function is the only reusable part, then what's the point? This function is only a couple of lines long, so why even bother? That'd be a fair concern, if it wasn't for the context that all of this is only example code. Think of this example as a stand-in for much more complex business logic. Perhaps the restaurant will allow a certain percentage of over-booking, because some people never show up. Perhaps it will allow different over-booking percentages on different (week) days. Perhaps it'll need to handle reservations on per-table basis, instead of per-seat. Perhaps it'll need to handle multiple seatings on the same day. There are lots of variations you can throw into this business domain that will make it much more complex, and worthy of reuse.) </p> <p> <strong>Brittleness of composition</strong> </p> <p> Finally, you refer to a more realistic scenario, called <em>performOvernightCloseout</em>, and ask if a fine-grained composition wouldn't be brittle? In my experience, it wouldn't be <em>more</em> brittle than any alternatives I've identified. Whether you 'inject' functions into other functions, or you compose them in the Composition Root, doesn't change the underlying forces that act on your code. If you make substantial changes to the dependencies involved, it will break your code, and you'll need to address that. This is true for any manual composition, including <a href="http://blog.ploeh.dk/2014/06/10/pure-di">Pure DI</a>. The only way to avoid compilation errors when you redefine your dependency graphs is to use a DI Container, but that only <a href="http://blog.ploeh.dk/2011/04/29/Feedbackmechanismsandtradeoffs">makes the feedback loop worse, because it'd change a compile-time error into a run-time error</a>. </p> <p> This is a long answer, and even so, I'm not sure I've sufficiently addressed all of your concerns. If not, please write again for clarifications. </p> </div> <div class="comment-date">2016-03-30 13:46 UTC</div> </div> <div class="comment"> <div class="comment-author">John Dailey</div> <div class="comment-content"> <p>I believe there was a bit of miscommunication. When I talked about the 'logic of only checking existing reservations on the same day', I wasn't talking about the implementation of the SqlGateway. I was talking about the 'imp' function: </p> <pre>let imp = Validate.reservation >> map (fun r -> SqlGateway.getReservedSeats connectionString r.Date, r) >> bind (fun (i, r) -> Capacity.check 10 i r) >> map (SqlGateway.saveReservation connectionString)</pre> <p> This implements the logic of 'Validation comes first. If that checks out, then get the number of reserved seats from the data store. Use the date on the request for that. Feed this into the capacity check routine and only if everything checks out should we persist the reservation. </p> <p> That's all internal logic that in my mind ought to be encapsulated within the domain. That's what I meant by 'behind the wall'. There should simply be a function 'makeReservation' with the signature ReservationRequest -> ReservationResponse. That's it. Clients should not be worrying themselves with these extra details. That 'imp' function just seems deeply inappropriate as something that lives externally to the reservations module, and that must be supplied individually by each client accessing the module. </p> <p> I guess I'm rejecting the idea of Onion/Hexagonal architecture, or at least the idea that all I/O always belongs at the outside. To me, a reservations module must depend on a reservations data store, because if you don't persist information to the db, <em>you haven't actually made the reservation</em>. Of course, I still want to depend on interfaces, not implementations (perhaps naming the reference ReservationDataStore.saveReservation). The data store IS a separate layer and should have a well defined interface. </p> <p> I just don't understand why anyone would find this design more desirable than the straightforward UI -> Logic -> Storage chain of dependencies. Clearly there's some sort of appeal -- lots of smart people show excitement for it. But it's a bit mystifying to me. </p> </div> <div class="comment-date">2016-04-06 01:45 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> John, thank you for writing. It's OK to reject the concept of <em>ports and adapters</em>; it's not a one-size-fits-all architecture. While I <em>often</em> use it, I don't <em>always</em> use it. One of my most successful customer engagements involved building a set of REST APIs on top of an existing IT infrastructure. All the business logic was already implemented elsewhere, so I deliberately chose a simpler, more coupled architecture, because there'd be no benefit from a full <em>ports and adapters</em> architecture. The code bases involved still run in production four years later, even though they're constantly being changed and enhanced. </p> <p> If, on the other hand, you choose to adopt the ports and adapters architecture, you'll need to do it right so that you can harvest the benefits. </p> <p> A Façade function with the type <code>ReservationRequest -&gt; ReservationResponse</code> only makes sense in an HTTP request/response scenario, because both the ReservationRequest and ReservationResponse types are specific to the web context. You can't reuse such a function in a different setting. </p> <p> You still have such a function, because that's the function that handles the web request, but you'll need to <em>compose</em> it from smaller components, unless you want to write a <a href="http://martinfowler.com/eaaCatalog/transactionScript.html">Transaction Script</a> within that function (which is also sometimes OK). </p> <p> Granted, there's some 'logic' involved in this composition, in the sense that it might be possible to create a faulty composition, but that logic has to go <em>somewhere</em>. </p> </div> <div class="comment-date">2016-04-08 9:57 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://github.com/MartinRykfors">Martin Rykfors</a></div> <div class="comment-content"> <p> Hi Mark, thanks for this post and the discussion. </p> <p> I feel that I share some of the concerns that John is bringing up. The <code>postReservation</code> function is pretty simple now, but eventually we might want to make external calls based on some conditional logic from the domain model. </p> <p> Let's say there is a new feature request. The restaurant has struck a deal with the local caravan dealership, allowing them to rent a caravan to park outside the restaurant in order to increase the seating capacity for one evening. Of course, sometimes there are no caravans available, so we'll need to query the caravan database to see if there is a big enough caravan available that evening: </p> <p> <pre> findCaravan :: Int -> ZoneTime -> IO (Maybe Caravan) findCaravan minimumCapacity date = ... -- database stuff </pre> </p> <p> How would we integrate this? We could put a call to <code>findCaravan</code> inside the IO context of <code>postReservation</code>, but since we only want to query the caravan database if the first capacity check failed, we'll need to add branching logic to <code>postReservation</code>. We want to have tests for this logic, but since we are coupled to the IO context, we are in trouble. </p> <p> The original issue was that the F# version of <code>checkCapacity</code> could be made impure by injecting the impure <code>getReservedSeats</code> function to it, which is something that Haskell disallows. What if we changed the Haskell version of <code>checkCapacity</code> to the following instead? </p> <p> <pre> checkCapacity :: Monad m => Int -> (ZonedTime -> m Int) -> Reservation -> EitherT Error m Reservation checkCapacity capacity getReservedSeats reservation = EitherT $ do reservedSeats <- getReservedSeats $ date reservation if capacity < quantity reservation + reservedSeats then return $ Left CapacityExceeded else return $ Right reservation </pre> </p> <p> Since we know that <code>getReservedSeats</code> will be effectful, we declare it so in its type signature. We change <code>postReservation</code> to become: </p> <p> <pre> postReservation :: ReservationRendition -> IO (HttpResult ()) postReservation candidate = fmap toHttpResult $ runEitherT $ hoistEither $ validateReservation candidate >>= checkCapacity 10 (getReservedSeatsFromDB connStr) >>= liftIO . saveReservation connStr </pre> </p> <p> (Or something like that. Monad transformers are the current step on my Haskell learning journey, so I'm not sure this will typecheck.) </p> <p> This way, we can once more inject an IO typed function as the <code>getReservedSeats</code> argument of <code>checkCapacity</code> like we did in F#. If we want to test it, we can create a stub database call by using the monad instance of <code>Identity</code>, for example: </p> <p> <pre> stubGetReservedSeats :: ZonedTime -> Identity Int stubGetReservedSeats date = return 8 </pre> </p> <p> The new feature request also becomes easy to implement. Just pass <code>findCaravan</code> as a new argument to <code>checkCapacity</code>, and make the type of that argument to be <code>ZonedTime -> m (Maybe Caravan)</code>. The <code>checkCapacity</code> function can now do the initial capacity check, and if that fails, call <code>findCaravan</code> to see if it is still possible to allow the booking by renting a suitable caravan. The branching logic that is needed is now inside <code>checkCapacity</code> and can be tested outside of the IO context. </p> <p> I don't know if this would work well in practice, or if it would cause more trouble than what it's worth. What do you think? </p> </div> <div class="comment-date">2016-06-09 10:56 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Martin, thank you for writing. Your question prompted me to write a <a href="http://blog.ploeh.dk/2016/07/04/conditional-composition-of-functions">new article</a>. It takes a slightly different approach to the one you suggest. </p> <p> I haven't tried defining <code>checkCapacity</code> in the way you suggest, but that may be an exercise for another day. In general, though, I'm hesitant to introduce constraints not <em>required</em> by the implementation. In this case, I think it'd be a fair question to ask why the <code>getReservedSeats</code> argument has to have the type <code>ZonedTime -&gt; m Int</code>? Nothing in its implementation seems to indicate the need for that, so it looks like a leaky abstraction to me. </p> </div> <div class="comment-date">2016-07-04 7:06 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://giuliohome.wordpress.com/">Giulio</a></div> <div class="comment-content"> <p> I'd like to follow up my <a href="https://twitter.com/giuliohome/status/822373175983935488">twitter comments</a> here </p><p> Very interesting article! I'm sorry that some Haskell details are quite difficult for me at the moment and the F# code looks a bit too simplified or maybe just incomplete </p><p> So I'd prefer your <a href="http://blog.ploeh.dk/2016/04/11/async-as-surrogate-io/">approach here</a> : it better translates Haskell to F# but I still have to study its implications. Anyway the concrete benefits of Async are evident, like the ones of F# computation expressions (to wrap exceptions etc...) </p><p> On the other hand I fail to see the benefit of purity vs other IoC patterns. Besides async and exceptions wrapping, purity seems an unnecessarily strict enforcement of Haskell (isn't it?): original F# was testable in isolation </p><p> What's missing if I only mock/stub/inject the dependency? When you say <blockquote>"ability to reason is eroded"</blockquote> and <blockquote>"it breaks encapsulation"</blockquote> about DI, do you mean something related to <a href="http://stackoverflow.com/questions/1005473/must-dependency-injection-come-at-the-expense-of-encapsulation">this topic</a>? </p><p> Thank you so much for your interesting insights!</p></div> <div class="comment-date">2017-01-20 10:45 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> Giulio, thank you for writing. Why do you think that the only goal of pure functions is testability? </div> <div class="comment-date">2017-01-20 18:23 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="https://giuliohome.wordpress.com/">Giulio</a></div> <div class="comment-content"> <p> <blockquote>Why do you think that the only goal of pure functions is testability?</blockquote> My bad, actually I don't know. I assume that purity's goal is to avoid side effects. It is very clear that this can be useful when I have a <em>hidden</em> side effect in the function, but it is <em>less obvious</em> (to me) what it does mean when the <i>side effect</i> is as evident as an argument (?). </p> <p> Anyway, I'll follow your implicit suggestion that with a pure function I can reach a more important goal than testability. Ok, practical example. Today I spent one hour to fix a bug that I introduced with a previous enhancement. My goal is to avoid this next time, by learning a better programming style from your article, but first I need to understand what went wrong. I can spot 3 problems: <ul> <li>1) I didn't see the bug when I wrote the code changes</li> <li>2) I didn't go through that part of my app when I tested the upgrade</li> <li>3) I needed a whole hour to fix that stupid bug, because I could not immediately find the point where it was</li> </ul> When I think that testability is important, it's because I hope it saves me from the issues 2 and 3, but let's take for granted that with a pure function I can avoid even point 1. </p> <p> Now unfortunately I struggle to interpret Haskell's details and I can't find enough F# to recover the whole meaning, but the main point I get is that a part of code (that could be bugged because <i>the ability to reason is eroded</i> there) has been moved from a finally 'pure' function to a new 'impure' one, that is introduced by a special 'liftIO' keyword (sorry for my poor recap) </p> <p> Finally my question is, but what about that all the possible bugs (often at the interface between persistence and model layers) that have simply been migrated around this critical IO land? are they going to counterbalance the above said benefits? where is exactly the net, overall improvement? </p> </div> <div class="comment-date">2017-01-20 20:02:00 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> First, I should make it specific that when I discuss object-oriented code in general, and encapsulation specifically, I tend to draw heavily on the work of Bertrand Meyer in <a href="http://amzn.to/1claOin">Object-Oriented Software Construction</a>. Some people may regard Alan Kay as the original inventor of object-orientation, but C# and Java's interpretation of object-oriented programming seems closer to Meyer's than to Kay's. </p> <p> To Meyer, the most important motivation for object-orientation was reuse. He wanted to be able to reuse software that other people had written, instead of, as had been the norm until then, importing other people's source code into new code bases. I'm sure you can see how that goal was important. </p> <p> In order to make binary libraries reusable, Meyer wanted to establish some techniques that would enable developers to (re)use the code without having access to the source code. His vision was that every object should come with a set of invariants, pre-, and post-conditions. He even invented a programming language, Eiffel, that included the ability to define such rules as programmable and enforceable checks. Furthermore, such rules could be exported as automatic documentation. This concept, which Meyer called <em>contracts</em>, is what we now know as <em>encapsulation</em> (I have <a href="http://www.shareasale.com/r.cfm?u=1017843&b=611266&m=53701&afftrack=&urllink=www%2Epluralsight%2Ecom%2Fcourses%2Fencapsulation%2Dsolid">a Pluralsight course that explains this in more details</a>). </p> <p> While the idea of formal contracts seemed like a good idea, neither Java nor C# have them. In C# and Java, you can still implement invariants with guard clauses, assertions, and so on, but there's no formal, externally visible way to communicate such invariants. </p> <p> In other words, in modern object-oriented languages, you can't reason about the code from its API alone. Meyer's goal was that you should be able to reuse an object that someone else had written, <em>without reading the source code</em>. In a nutshell: if you have to read the source code, it means that encapsulation is broken. </p> <p> One of the other concepts that Meyer introduced was <a href="http://en.wikipedia.org/wiki/Command-query_separation">Command-Query Separation</a> (CQS). To Meyer, it was important to be able to distinguish between operations that had side-effects, and operations that didn't. </p> <p> In my experience, a big category of defects are caused by unintended side-effects. One developer doesn't realise that the method call (s)he invokes has a side-effect that another developer put there. Languages like C# and Java have few constraints, so any method could potentially hide a side-effect. The only way you can know is by reading the code. </p> <p> What if, then, there was a programming language where you could tell, at a glance, whether an operation had a side-effect? Such programming languages exist. Haskell is one of them. In Haskell, all functions are pure by default. This means that they are deterministic and have no side-effects. If you want to write an impure function, you must explicitly declare that in the function's type (which is done with the <code>IO</code> type). </p> <p> In Haskell, you don't have to read the source code in order to know whether a function is impure or not. You can simply look at the type! </p> <p> This is one of the many benefits of pure functions: we know, by definition, that they have no side-effects. This property holds not only for the function itself, but for all functions that that pure function calls. This follows from the definition of purity. A pure function can call other pure functions, but it can't call impure functions, because, if it did, it would become impure itself. Haskell enforces this rule, so when you're looking at a pure function, you know that that entire subgraph of your code is pure. Otherwise, it wouldn't compile. </p> <p> Not only does that relationship force us to push impure functions to the boundary of applications, as described in the present article. It also makes <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">functions intrinsically testable</a>. There are other advantages of pure functions besides these, such as easy composability, but in general, pure functions are attractive because it becomes easier to figure out what a function does without having to understand all the details. In other words, pure functions are better abstractions. </p> <p> To be clear: functional programming doesn't protect you from defects, but in my experience, it helps you to avoid entire classes of bugs. I still write automated tests of my F# code, but I write fewer tests than when I wrote C# code. Despite that, I also seem to be producing fewer defects, and I rarely need to debug the functions. </p> <p> What's the overall net improvement? I don't know. How would you measure that? </p> </div> <div class="comment-date">2017-01-20 21:53 UTC</div> </div> <div class="comment"> <div class="comment-author">Abel Quiros</div> <div class="comment-content"> <p>Consider a logic that, depending on a condition, performs one IO operation or another. In Java (I don’t know F# or Haskell):</p> <pre>int foo(boolean condition, IntSupplier io1, IntSupplier io2) { if (condition) { return io1.getAsInt(); } else { return io2.getAsInt(); } }</pre> <p>Would you recommend pre-fetching the two values prior to calling the <code>foo</code> function even though only one would ultimately be required?</p> </div> <div class="comment-date">2017-03-12 20:47 UTC</div> </div> <div class="comment"> <div class="comment-author"><a href="http://blog.ploeh.dk">Mark Seemann</a></div> <div class="comment-content"> <p> Abel, thank you for writing. There's a discussion closely related to that question in the <em>remarks</em> sections of my <a href="http://blog.ploeh.dk/2017/02/02/dependency-rejection">dependency rejection</a> article. It's the remark from 2017-02-18 19:54 UTC (I really ought to get around to <a href="https://github.com/ploeh/ploeh.github.com/issues/267">add permalinks to comments</a>...). </p> <p> Additionally, there's a discussion on this page that lead to <a href="http://blog.ploeh.dk/2016/07/04/conditional-composition-of-functions">a new article</a> that goes into some further details. </p> </div> <div class="comment-date">2017-03-13 06:42 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Ad-hoc Arbitraries - now with pipes http://blog.ploeh.dk/2016/03/01/ad-hoc-arbitraries---now-with-pipes 2016-03-01T08:46:00+00:00 Mark Seemann <div id="post"> <p> <em>Slightly improved syntax for defining ad-hoc, in-line Arbitraries for FsCheck.Xunit.</em> </p> <p> Last year, I described <a href="http://blog.ploeh.dk/2015/09/08/ad-hoc-arbitraries-with-fscheckxunit">how to define and use ad-hoc, in-line Arbitraries with FsCheck.Xunit</a>. The final example code looked like this: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nc&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;nc&nbsp;(<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual)</pre> </p> <p> There's one tiny problem with this way of expressing properties using Prop.forAll: the use of brackets to demarcate the anonymous multi-line function. Notice how the opening bracket appears to the left of the <code>fun</code> keyword, while the closing bracket appears eleven lines down, after the end of the expression <code>Dead =! actual</code>. </p> <p> While it isn't pretty, I haven't found it that bad for readability, either. When you're writing the property, however, this syntax is cumbersome. </p> <p> <strong>Issues writing the code</strong> </p> <p> After having written <code>Prop.forAll nc</code>, you start writing <code>(fun neighborCount -> )</code>. After you've typed the closing bracket, you have to move your cursor one place to the left. When entering new lines, you have to keep diligent, making sure that the closing bracket is always to the right of your cursor. </p> <p> If you ever need to use a nested level of brackets within that anonymous function, your vigilance will be tested to its utmost. When I wrote the above property, for example, when I reached the point where I wanted to call the calculateNextState function, I wrote: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;State&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;calculateNextState&nbsp;(</pre> </p> <p> The editor detects that I'm writing an opening bracket, but since I have the closing bracket to the immediate right of the cursor, this is what happens: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;State&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;calculateNextState&nbsp;()</pre> </p> <p> What normally happens when you type an opening bracket is that the editor automatically inserts a closing bracket to the right of your cursor, but in this case, it doesn't do that. There's already a closing bracket immediately to the right of the cursor, and the editor assumes that this bracket belongs to the opening bracket I just typed. What it really should have done was this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;())</pre> </p> <p> The editor doesn't have a chance, though, because at this point, I'm still typing, and the code doesn't compile. None of the alternatives compile at this point. You can't really blame the editor. </p> <p> To make a long story short, enclosing a multi-line anonymous function in brackets is a source of errors. If only there was a better alternative. </p> <p> <strong>Backward pipe</strong> </p> <p> The solution started to dawn on me because I've been learning Haskell for the last half year, and in Haskell, you often use the <code>$</code> operator to get rid of unwanted brackets. <a href="http://stackoverflow.com/q/7183903/126014">F# has an equivalent operator</a>: <code>&lt;|</code>, the backward pipe operator. </p> <p> I rarely use the <code>&lt;|</code> operator in F#, but in this case, it works really well: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>(QuietOnSuccess&nbsp;=&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Any&nbsp;live&nbsp;cell&nbsp;with&nbsp;&gt;&nbsp;3&nbsp;live&nbsp;neighbors&nbsp;dies``</span>&nbsp;(cell&nbsp;:&nbsp;<span style="color:#4ec9b0;">int</span>&nbsp;*&nbsp;<span style="color:#4ec9b0;">int</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nc&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[4..8]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;nc&nbsp;&lt;|&nbsp;<span style="color:blue;">fun</span>&nbsp;neighborCount&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;liveNeighbors&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">findNeighbors</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">take</span>&nbsp;neighborCount &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">toList</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">State</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">calculateNextState</span>&nbsp;(cell&nbsp;<span style="color:navy;">::</span>&nbsp;liveNeighbors&nbsp;|&gt;&nbsp;<span style="color:navy;">shuffle</span>&nbsp;|&gt;&nbsp;<span style="color:navy;">set</span>)&nbsp;cell &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Dead</span>&nbsp;=!&nbsp;actual</pre> </p> <p> Notice that there's no longer any need for a closing bracket after <code>Dead =! actual</code>. </p> <p> <strong>Summary</strong> </p> <p> When the last argument passed to a function is another function, you can replace the brackets with a single application of the <code>&lt;|</code> operator. I only use this operator sparingly, but in the case of in-line ad-hoc FsCheck Arbitraries, I find it useful. </p> <p> <strong>Addendum 2016-05-17:</strong> You can <a href="http://blog.ploeh.dk/2016/05/17/tie-fighter-fscheck-properties">get rid of the <code>nc</code> value with <em>TIE fighter infix</em> notation</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: other properties http://blog.ploeh.dk/2016/02/19/types--properties--software-other-properties 2016-02-19T08:59:00+00:00 Mark Seemann <div id="post"> <p> <em>Even simple functions may have properties that can be expressed independently of the implementation.</em> </p> <p> This article is the eighth in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In previous articles, you've seen an extensive example of how to solve <a href="http://www.codingdojo.org/cgi-bin/index.pl?KataTennis">the Tennis Kata</a> with type design and Property-Based Testing. Specifically, in <a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">the third article</a>, you saw the introduction of a function called <code>other</code>. In that article we didn't cover that function with automatic tests, but this article rectifies that omission. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Implementation duplication</strong> </p> <p> The <code>other</code> function is used to find the opponent of a player. The implementation is trivial: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">other</span>&nbsp;=&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerOne</span> </pre> </p> <p> Unless you're developing software where lives, or extraordinary amounts of money, are at stake, you probably need not test such a trivial function. I'm still going to show you how you could do this, mostly because it's a good example of how Property-Based Testing can sometimes help you test the <em>behaviour</em> of a function, instead of the implementation. </p> <p> Before I show you that, however, I'll show you why example-based unit tests are inadequate for testing this function. </p> <p> You could test this function using examples, and because the space of possible input is so small, you can even cover it in its entirety: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Fact</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``other&nbsp;than&nbsp;playerOne&nbsp;returns&nbsp;correct&nbsp;result``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;=!&nbsp;<span style="color:navy;">other</span>&nbsp;<span style="color:navy;">PlayerOne</span> [&lt;<span style="color:#4ec9b0;">Fact</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``other&nbsp;than&nbsp;playerTwo&nbsp;returns&nbsp;correct&nbsp;result``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;=!&nbsp;<span style="color:navy;">other</span>&nbsp;<span style="color:navy;">PlayerTwo</span></pre> </p> <p> The <code>=!</code> operator is a custom operator defined by <a href="http://www.swensensoftware.com/unquote">Unquote</a>, an assertion library. You can read the first expression as <em>player two must equal other than player one</em>. </p> <p> The problem with these tests, however, is that they don't state anything not already stated by the implementation itself. Basically, what these tests state is that <em>if the input is PlayerOne, the output is PlayerTwo, and if the input is PlayerTwo, the output is PlayerOne</em>. </p> <p> That's exactly what the implementation does. </p> <p> The only important difference is that the implementation of <code>other</code> states this relationship more succinctly than the tests. </p> <p> It's as though the tests duplicate the information already in the implementation. How does that add value? </p> <p> Sometimes, this can be the only way to cover functionality with tests, but in this case, there's an alternative. </p> <p> <strong>Behaviour instead of implementation</strong> </p> <p> Property-Based Testing inspires you to think about the observable behaviour of a system, rather than the implementation. Which properties do the <code>other</code> function have? </p> <p> If you call it with a Player value, you'd expect it to return a Player value that's not the input value: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``other&nbsp;returns&nbsp;a&nbsp;different&nbsp;player``</span>&nbsp;player&nbsp;=&nbsp;player&nbsp;&lt;&gt;!&nbsp;<span style="color:navy;">other</span>&nbsp;player</pre> </p> <p> The <code>&lt;&gt;!</code> operator is another custom operator defined by Unquote. You can read the expression as <em>player must not equal other player</em>. </p> <p> This property alone doesn't completely define the behaviour of <code>other</code>, but combined with the next property, it does: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``other&nbsp;other&nbsp;returns&nbsp;same&nbsp;player``</span>&nbsp;player&nbsp;=&nbsp;player&nbsp;=!&nbsp;<span style="color:navy;">other</span>&nbsp;(<span style="color:navy;">other</span>&nbsp;player)</pre> </p> <p> If you call <code>other</code>, and then you take the output of that call and use it as input to call <code>other</code> again, you should get the original Player value. This property is an excellent example of what <a href="http://fsharpforfunandprofit.com/posts/property-based-testing-2">Scott Wlaschin calls <em>there and back again</em></a>. </p> <p> These two properties, together, describe the behaviour of the <code>other</code> function, without going into details about the implementation. </p> <p> <strong>Summary</strong> </p> <p> You've seen a simple example of how to describe the properties of a function without resorting to duplicating the implementation in the tests. You may not always be able to do this, but it always feels right when you can. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: finite state machine http://blog.ploeh.dk/2016/02/18/types--properties--software-finite-state-machine 2016-02-18T08:24:00+00:00 Mark Seemann <div id="post"> <p> <em>How to compose the tennis game code into a finite state machine.</em> </p> <p> This article is the seventh in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/17/types-properties-software-initial-state">previous article</a>, you saw how to define the initial state of a tennis game. In this article, you'll see how to define the tennis game as a finite state machine. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Scoring a sequence of balls</strong> </p> <p> Previously, you saw how to score a game in an ad-hoc fashion: </p> <p> <pre>&gt; let firstBall = score newGame PlayerTwo;; val firstBall : Score = Points {PlayerOnePoint = Love; PlayerTwoPoint = Fifteen;} &gt; let secondBall = score firstBall PlayerOne;; val secondBall : Score = Points {PlayerOnePoint = Fifteen; PlayerTwoPoint = Fifteen;}</pre> </p> <p> You'll quickly get tired of that, so you may think that you can do something like this instead: </p> <p> <pre>&gt; newGame |&gt; (fun s -> score s PlayerOne) |&gt; (fun s -> score s PlayerTwo);; val it : Score = Points {PlayerOnePoint = Fifteen; PlayerTwoPoint = Fifteen;}</pre> </p> <p> That does seem a little clumsy, though, but it's not really what you need to be able to do either. What you'd probably appreciate more is to be able to calculate the score given a sequence of wins. A sequence of wins is a list of Player values; for instance, [PlayerOne; PlayerOne; PlayerTwo] means that player one won the first two balls, and then player two won the third ball. </p> <p> Since we already know the initial state of a game, and how to calculate the score for each ball, it's a one-liner to calculate the score for a sequence of balls: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreSeq</span>&nbsp;wins&nbsp;=&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">fold</span>&nbsp;<span style="color:navy;">score</span>&nbsp;newGame&nbsp;wins </pre> </p> <p> This function has the type <code>Player seq -&gt; Score</code>. It uses <code>newGame</code> as the initial state of a left fold over the wins. The aggregator function is the <code>score</code> function. </p> <p> You can use it with a sequence of wins, like this: </p> <p> <pre>&gt; scoreSeq [PlayerOne; PlayerOne; PlayerTwo];; val it : Score = Points {PlayerOnePoint = Thirty; PlayerTwoPoint = Fifteen;}</pre> </p> <p> Since player one won the first two balls, and player two then won the third ball, the score at that point is thirty-fifteen. </p> <p> <strong>Properties for the state machine</strong> </p> <p> Can you think of any properties for the <code>scoreSeq</code> function? </p> <p> There are quite a few, it turns out. It may be a good exercise if you pause reading for a couple of minutes, and try to think of some properties. </p> <p> If you have a list of properties, you can compare them with the ones I though of. </p> <p> Before we start, you may find the following helper functions useful: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">isPoints</span>&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">Points</span>&nbsp;_&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span>&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">false</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">isForty</span>&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">Forty</span>&nbsp;&nbsp;_&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span>&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">false</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">isDeuce</span>&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">Deuce</span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span>&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">false</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">isAdvantage</span>&nbsp;=&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">Advantage</span>&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span>&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">false</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">isGame</span>&nbsp;=&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">Game</span>&nbsp;_&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span>&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">false</span></pre> </p> <p> These can be useful to express some properties, but I don't think that they are generally useful, so I put them together with the test code. </p> <p> <strong>Limited win sequences</strong> </p> <p> If you look at short win sequences, you can already say something about them. For instance, it takes at least four balls to finish a game, so you know that if you have fewer wins than four, the game must still be on-going: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;with&nbsp;less&nbsp;then&nbsp;four&nbsp;balls&nbsp;isn&#39;t&nbsp;over``</span>&nbsp;(wins&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>&nbsp;<span style="color:#4ec9b0;">list</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;=&nbsp;wins&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">truncate</span>&nbsp;3&nbsp;|&gt;&nbsp;<span style="color:navy;">scoreSeq</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isGame</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> The built-in function <a href="https://msdn.microsoft.com/en-us/library/ee370325.aspx">Seq.truncate</a> returns only the <em>n</em> (in this case: 3) first elements of a sequence. If the sequence is shorter than that, then the entire sequence is returned. The use of <code>Seq.truncate 3</code> guarantees that the sequence passed to scoreSeq is no longer than three elements long, no matter what <a href="http://fscheck.github.io/FsCheck">FsCheck</a> originally generated. </p> <p> Likewise, you can't reach <em>deuce</em> in less than six balls: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;with&nbsp;less&nbsp;than&nbsp;six&nbsp;balls&nbsp;can&#39;t&nbsp;be&nbsp;Deuce``</span>&nbsp;(wins&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>&nbsp;<span style="color:#4ec9b0;">list</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;wins&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">truncate</span>&nbsp;5&nbsp;|&gt;&nbsp;<span style="color:navy;">scoreSeq</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isDeuce</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> This is similar to the previous property. There's one more in that family: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;with&nbsp;less&nbsp;than&nbsp;seven&nbsp;balls&nbsp;can&#39;t&nbsp;have&nbsp;any&nbsp;player&nbsp;with&nbsp;advantage``</span> &nbsp;&nbsp;&nbsp;&nbsp;(wins&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>&nbsp;<span style="color:#4ec9b0;">list</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;wins&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">truncate</span>&nbsp;6&nbsp;|&gt;&nbsp;<span style="color:navy;">scoreSeq</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isAdvantage</span><span style="background:yellow;">)&nbsp;@&gt;</span></pre> </p> <p> It takes at least seven balls before the score can reach a state where one of the players have the advantage. </p> <p> <strong>Longer win sequences</strong> </p> <p> Conversely, you can also express some properties related to win sequences of a minimum size. This one is more intricate to express with FsCheck, though. You'll need a sequence of Player values, but the sequence should be guaranteed to have a minimum length. There's no built-in in function for that in FsCheck, so you need to define it yourself: </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;Gen&lt;Player&nbsp;list&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">genListLongerThan</span>&nbsp;n&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;playerGen&nbsp;=&nbsp;<span style="color:#4ec9b0;">Arb</span>.generate&lt;<span style="color:#4ec9b0;">Player</span>&gt; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;nPlayers&nbsp;=&nbsp;playerGen&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">listOfLength</span>&nbsp;(n&nbsp;+&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;morePlayers&nbsp;=&nbsp;playerGen&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">listOf</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">map2</span>&nbsp;(@)&nbsp;nPlayers&nbsp;morePlayers</pre> </p> <p> This function takes an exclusive minimum size, and returns an FsCheck generator that will generate Player lists longer than <code>n</code>. It does that by combining two of FsCheck's built-in generators. Gen.listOfLength generates lists of the requested length, and Gen.listOf generates all sorts of lists, including the empty list. </p> <p> Both <code>nPlayers</code> and <code>morePlayers</code> are values of the type Gen&lt;Player list&gt;. Using Gen.map2, you can concatenate these two list generators using F#'s built-in list concatenation operator <code>@</code>. (All operators in F# are also functions. The <code>(@)</code> function has the type <code>'a list -&gt; 'a list -&gt; 'a list</code>.) </p> <p> With the genListLongerThan function, you can now express properties related to longer win sequences. As an example, if the players have played more than four balls, the score can't be a Points case: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;with&nbsp;more&nbsp;than&nbsp;four&nbsp;balls&nbsp;can&#39;t&nbsp;be&nbsp;Points``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;moreThanFourBalls&nbsp;=&nbsp;<span style="color:navy;">genListLongerThan</span>&nbsp;4&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;moreThanFourBalls&nbsp;(<span style="color:blue;">fun</span>&nbsp;wins&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreSeq</span>&nbsp;wins &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isPoints</span><span style="background:yellow;">)&nbsp;@&gt;</span>)</pre> </p> <p> As <a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">previously explained</a>, Prop.forAll expresses a property that must hold for all lists generated by <code>moreThanFourBalls</code>. </p> <p> Correspondingly, if the players have played more than five balls, the score can't be a Forty case: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;with&nbsp;more&nbsp;than&nbsp;five&nbsp;balls&nbsp;can&#39;t&nbsp;be&nbsp;Forty``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;moreThanFiveBalls&nbsp;=&nbsp;<span style="color:navy;">genListLongerThan</span>&nbsp;5&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;moreThanFiveBalls&nbsp;(<span style="color:blue;">fun</span>&nbsp;wins&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreSeq</span>&nbsp;wins &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isForty</span><span style="background:yellow;">)&nbsp;@&gt;</span>)</pre> </p> <p> This property is, as you can see, similar to the previous example. </p> <p> <strong>Shortest completed game</strong> </p> <p> Do you still have your list of tennis game properties? Let's see if you thought of this one. The shortest possible way to complete a tennis game is when one of the players wins four balls in a row. This property should hold for all (two) players: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;where&nbsp;one&nbsp;player&nbsp;wins&nbsp;all&nbsp;balls&nbsp;is&nbsp;over&nbsp;in&nbsp;four&nbsp;balls``</span>&nbsp;(player)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fourWins&nbsp;=&nbsp;<span style="color:#4ec9b0;">Seq</span>.<span style="color:navy;">init</span>&nbsp;4&nbsp;(<span style="color:blue;">fun</span>&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;player) &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreSeq</span>&nbsp;fourWins &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:navy;">Game</span>&nbsp;player &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> This test first defines <code>fourWins</code>, of the type <code>Player seq</code>. It does that by using the <code>player</code> argument created by FsCheck, and repeating it four times, using Seq.init. </p> <p> The <code>=!</code> operator is a custom operator defined by <a href="http://www.swensensoftware.com/unquote">Unquote</a>, an assertion library. You can read the expression as <em>expected must equal actual</em>. </p> <p> <strong>Infinite games</strong> </p> <p> The tennis scoring system turns out to be a rich domain. There are still more properties. We'll look at a final one, because it showcases another way to use FsCheck's API, and then we'll call it a day. </p> <p> An interesting property of the tennis scoring system is that a game isn't guaranteed to end. It may, theoretically, continue forever, if players alternate winning balls: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``A&nbsp;game&nbsp;where&nbsp;players&nbsp;alternate&nbsp;never&nbsp;ends``</span>&nbsp;firstWinner&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;alternateWins&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;firstWinner &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">constant</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;p&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[p;&nbsp;<span style="color:navy;">other</span>&nbsp;p]) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">listOf</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">concat</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;alternateWins&nbsp;(<span style="color:blue;">fun</span>&nbsp;wins&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreSeq</span>&nbsp;wins &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">|&gt;</span><span style="background:yellow;">&nbsp;(</span><span style="color:navy;background:yellow;">not</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">&lt;&lt;</span><span style="background:yellow;">&nbsp;</span><span style="color:navy;background:yellow;">isGame</span><span style="background:yellow;">)&nbsp;@&gt;</span>)</pre> </p> <p> This property starts by creating <code>alternateWins</code>, an Arbitrary&lt;Player list&gt;. It creates lists with alternating players, like [PlayerOne; PlayerTwo], or [PlayerTwo; PlayerOne; PlayerTwo; PlayerOne; PlayerTwo; PlayerOne]. It does that by using the <code>firstWinner</code> argument (of the type Player) as a constant starting value. It's only constant within each test, because <code>firstWinner</code> itself varies. </p> <p> Using that initial Player value, it then proceeds to map that value to a list with two elements: the player, and the other player, using the <code>other</code> function. This creates a Gen&lt;Player list&gt;, but piped into Gen.listOf, it becomes a Gen&lt;Player list list&gt;. An example of the sort of list that would generate could be [[PlayerTwo; PlayerOne]; [PlayerTwo; PlayerOne]]. Obviously, you need to flatten such lists, which you can do with List.concat; you have to do it inside of a Gen, though, so it's <code>Gen.map List.concat</code>. That gives you a Gen&lt;Player list&gt;, which you can finally turn into an Arbitrary&lt;Player list&gt; with Arb.fromGen. </p> <p> This enables you to use Prop.forAll with alternateWins to express that regardless of the size of such alternating win lists, the game never reaches a Game state. </p> <p> <strong>Summary</strong> </p> <p> In this article, you saw how to implement a finite state machine for calculating tennis scores. It's a left fold over the <code>score</code> function, always starting in the initial state where both players are at love. You also saw how you can express various properties that hold for a game of tennis. </p> <p> In this article series, you've seen an extensive example of how to design with types and properties. You may have noticed that out of the six example articles so far, only the first one was about designing with types, and the next five articles were about Property-Based Testing. It looks as though not much is gained from designing with types. </p> <p> On the contrary, much is gained by designing with types, but you don't see it, exactly because it's efficient. If you don't <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">design with types</a> in order to <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">make illegal states unrepresentable</a>, you'd have to write even more automated tests in order to test what happens when input is invalid. Also, if illegal states are representable, it would have been much harder to write Property-Based Tests. Instead of trusting that FsCheck generates only valid values based exclusively on the types, you'd have to write custom Generators or Arbitraries for each type of input. That would have been even more work than you've seen in this articles series. </p> <p> Designing with types makes Property-Based Testing a comfortable undertaking. Together, they enable you to develop software that you can trust. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: initial state http://blog.ploeh.dk/2016/02/17/types--properties--software-initial-state 2016-02-17T08:51:00+00:00 Mark Seemann <div id="post"> <p> <em>How to define the initial state in a tennis game, using F#.</em> </p> <p> This article is the sixth in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/16/types-properties-software-composition">previous article</a>, you saw how to compose a function that returns a new score based on a previous score, and information about which player won the ball. In this article, you'll see how to define the initial state of a tennis game. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Initial state</strong> </p> <p> You may recall from <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">the article on designing with types</a> that a Score is a discriminated union. One of the union cases is the Points case, which you can use to model the case where both players have either Love, Fifteen, or Thirty points. </p> <p> The game starts with both players at love. You can define this as a value: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;newGame&nbsp;=&nbsp;<span style="color:navy;">Points</span>&nbsp;{&nbsp;PlayerOnePoint&nbsp;=&nbsp;<span style="color:navy;">Love</span>;&nbsp;PlayerTwoPoint&nbsp;=&nbsp;<span style="color:navy;">Love</span>&nbsp;} </pre> </p> <p> Since this is a value (that is: not a function), it would make no sense to attempt to test it. Thus, you can simply add it, and move on. </p> <p> You can use this value to calculate ad-hoc scores from the beginning of a game, like this: </p> <p> <pre>&gt; let firstBall = score newGame PlayerTwo;; val firstBall : Score = Points {PlayerOnePoint = Love; PlayerTwoPoint = Fifteen;} &gt; let secondBall = score firstBall PlayerOne;; val secondBall : Score = Points {PlayerOnePoint = Fifteen; PlayerTwoPoint = Fifteen;}</pre> </p> <p> You'll soon get tired of defining <code>firstBall</code>, <code>secondBall</code>, <code>thirdBall</code>, and so on, so a more general way to handle and calculate scores is warranted. </p> <p> <strong>To be continued...</strong> </p> <p> In this article, you saw how to define the initial state for a tennis game. There's nothing to it, but armed with this value, you now have half of the requirements needed to turn the tennis score function into a finite state machine. You'll see how to do that <a href="http://blog.ploeh.dk/2016/02/18/types-properties-software-finite-state-machine">in the next article</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: composition http://blog.ploeh.dk/2016/02/16/types--properties--software-composition 2016-02-16T14:23:00+00:00 Mark Seemann <div id="post"> <p> <em>In which a general transition function is composed from specialised transition functions.</em> </p> <p> This article is the fifth in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">previous article</a>, you witnessed the continued walk-through of <a href="http://www.codingdojo.org/cgi-bin/index.pl?KataTennis">the Tennis Kata</a> done with Property-Based Test-Driven Development. In these articles, you saw how to define small, specific functions that model the transition out of particular states. In this article, you'll see how to compose these functions to a more general function. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Composing the general function</strong> </p> <p> If you recall the <a href="http://blog.ploeh.dk/2016/02/11/types-properties-software-state-transition-properties">second article in this series</a>, what you need to implement is a state transition of the type <code>Score -&gt; Player -&gt; Score</code>. What you have so far are the following functions: <ul> <li><code>scoreWhenPoints : PointsData -&gt; Player -&gt; Score</code></li> <li><code>scoreWhenForty : FortyData -&gt; Player -&gt; Score</code></li> <li><code>scoreWhenDeuce : Player -&gt; Score</code></li> <li><code>scoreWhenAdvantage : Player -&gt; Player -&gt; Score</code></li> <li><code>scoreWhenGame : Player -&gt; Score</code></li> </ul> You've seen the development of <a href="http://blog.ploeh.dk/2016/02/11/types-properties-software-state-transition-properties">scoreWhenDeuce</a>, <a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">scoreWhenAdvantage</a>, and <a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">scoreWhenForty</a> in previous articles, but you haven't seen scoreWhenGame or scoreWhenPoints. The development of these remaining functions follow similar principles, and use similar techniques. If you're interested in the details, you can always peruse <a href="https://github.com/ploeh/KataTennis">the source code repository</a>. </p> <p> These five functions are all the building blocks you need to implement the desired function of the type <code>Score -&gt; Player -&gt; Score</code>. You may recall that Score is a discriminated union defined as: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;= |&nbsp;<span style="color:navy;">Points</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">PointsData</span> |&nbsp;<span style="color:navy;">Forty</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">FortyData</span> |&nbsp;<span style="color:navy;">Deuce</span> |&nbsp;<span style="color:navy;">Advantage</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">Player</span> |&nbsp;<span style="color:navy;">Game</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">Player</span></pre> </p> <p> Notice how these cases align with the five functions above. That's not a coincidence. The driving factor behind the design of these five function was to match them with the five cases of the Score type. <a href="http://blog.ploeh.dk/2015/08/10/type-driven-development">In another article series, I've previously shown this technique, applied to a different problem.</a> </p> <p> You can implement the desired function by clicking the pieces together: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">score</span>&nbsp;current&nbsp;winner&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;current&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Points</span>&nbsp;p&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">scoreWhenPoints</span>&nbsp;p&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Forty</span>&nbsp;f&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;f&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Deuce</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">scoreWhenDeuce</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Advantage</span>&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">scoreWhenAdvantage</span>&nbsp;a&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Game</span>&nbsp;g&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">scoreWhenGame</span>&nbsp;g</pre> </p> <p> There's not a lot to it, apart from matching on <code>current</code>. If, for example, <code>current</code> is a Forty value, the match is the Forty case, and <code>f</code> represents the FortyData value in that case. The destructured <code>f</code> can be passed as an argument to scoreWhenForty, together with <code>winner</code>. The scoreWhenForty function returns a Score value, so the <code>score</code> function has the type <code>Score -&gt; Player -&gt; Score</code> - exactly what you want! </p> <p> Here's an example of using the function: </p> <p> <pre>&gt; score Deuce PlayerOne;; val it : Score = Advantage PlayerOne</pre> </p> <p> When the score is deuce and player one wins the ball, the resulting score is advantage to player one. </p> <p> <strong>Properties for the score function</strong> </p> <p> Can you express some properties for the score function? Yes and no. You can't state particularly interesting properties about the function in isolation, but you <em>can</em> express meaningful properties about sequences of scores. We'll return to that in a later article. For now, let's focus on the function in itself. </p> <p> You can, for example, state that the function can handle all input: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``score&nbsp;returns&nbsp;a&nbsp;value``</span>&nbsp;(current&nbsp;:&nbsp;<span style="color:#4ec9b0;">Score</span>)&nbsp;(winner&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;=&nbsp;<span style="color:navy;">score</span>&nbsp;current&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>&nbsp;<span style="color:green;">//&nbsp;Didn&#39;t&nbsp;crash&nbsp;-&nbsp;this&nbsp;is&nbsp;mostly&nbsp;a&nbsp;boundary&nbsp;condition&nbsp;test</span></pre> </p> <p> This property isn't particularly interesting. It's mostly a <a href="https://en.wikipedia.org/wiki/Smoke_testing_(software)">smoke test</a> that I added because I thought that it might flush out boundary issues, if any exist. That doesn't seem to be the case. </p> <p> You <em>can</em> also add properties that examine each case of input: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``score&nbsp;Points&nbsp;returns&nbsp;correct&nbsp;result``</span>&nbsp;points&nbsp;winner&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">score</span>&nbsp;(<span style="color:navy;">Points</span>&nbsp;points)&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:navy;">scoreWhenPoints</span>&nbsp;points&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> Such a property is unlikely to be of much use, because it mostly reproduces the implementation details of the score function. Unless you're writing high-stakes software (e.g. for medical purposes), such properties are likely to have little or negative value. After all, tests are also code; <a href="http://blog.ploeh.dk/2013/04/02/why-trust-tests">do you trust the test code more than the production code?</a> Sometimes, you may, but if you look at the source code for the score function, it's easy to review. </p> <p> You can write four other properties, similar to the one above, but I'm going to skip them here. They <em>are</em> in <a href="https://github.com/ploeh/KataTennis">the source code repository</a>, though, so you're welcome to look there if you want to see them. </p> <p> <strong>To be continued...</strong> </p> <p> In this article, you saw how to compose the five specific state transition functions into an overall state transition function. This is only a single function that calculates a score based on another score. In order to turn this function into a finite state machine, you must define an initial state and a way to transition based on a sequence of events. </p> <p> <a href="http://blog.ploeh.dk/2016/02/17/types-properties-software-initial-state">In the next article, you'll see how to define the initial state</a>, and <a href="http://blog.ploeh.dk/2016/02/18/types-properties-software-finite-state-machine">in the article beyond that, you'll see how to move through a sequence of transitions</a>. </p> <p> If you're interested in learning more about designing with types, you can watch my <a href="http://bit.ly/type-driven-development">Type-Driven Development with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: properties for the Forties http://blog.ploeh.dk/2016/02/15/types--properties--software-properties-for-the-forties 2016-02-15T09:08:00+00:00 Mark Seemann <div id="post"> <p> <em>An example of how to constrain generated input with FsCheck.</em> </p> <p> This article is the fourth in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">previous article</a>, you saw how to express properties for a simple state transition: finding the next tennis score when the current score is advantage to a player. These properties were simple, because they had to hold for all input of the given type (Player). In this article, you'll see how to constrain the input that <a href="http://fscheck.github.io/FsCheck">FsCheck</a> generates, in order to express properties about tennis scores where one of the players have forty points. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Winning the game</strong> </p> <p> When one of the players have forty points, there are three possible outcomes of the next ball: <ul> <li>If the player with forty points wins the ball, (s)he wins the game.</li> <li>If the other player has thirty points, and wins the ball, the score is <em>deuce</em>.</li> <li>If the other player has less than thirty points, and wins the ball, his or her points increases to the next level (from <em>love</em> to fifteen, or from fifteen to thirty).</li> </ul> The first property is the easiest to express, because it doesn't depend on the other player's points. </p> <p> You may recall that when one of the players have forty points, you express that score with the FortyData record type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">FortyData</span>&nbsp;=&nbsp;{&nbsp;Player&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>;&nbsp;OtherPlayerPoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;} </pre> </p> <p> Since Point is defined as <code>Love | Fifteen | Thirty</code>, it's clear that the Player who has forty points has a higher score than the other player - regardless of the OtherPlayerPoint value. This means that if that player wins, (s)he wins the game. It's easy to express that property with FsCheck: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;player:&nbsp;40&nbsp;when&nbsp;player&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(current&nbsp;:&nbsp;<span style="color:#4ec9b0;">FortyData</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;current.Player &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:navy;">Game</span>&nbsp;current.Player &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> Notice that this test function takes a single function argument called <code>current</code>, of the type FortyData. Since <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">illegal states are unrepresentable</a>, FsCheck can only generate legal values of that type. </p> <p> The scoreWhenForty function is a function that explicitly models what happens when the score is in the Forty case. The first argument is the data associated with the current score: <code>current</code>. The second argument is the winner of the next ball. In this test case, you want to express the case where the winner is the player who already has forty points: <code>current.Player</code>. </p> <p> The expected outcome of this is always that <code>current.Player</code> wins the game. </p> <p> The <code>=!</code> operator is a custom operator defined by <a href="http://www.swensensoftware.com/unquote">Unquote</a>, an assertion library. You can read the expression as <em>expected must equal actual</em>. </p> <p> In order to pass this property, you must implement a scoreWhenForty function. This is the simplest implementation that could possible work: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;winner&nbsp;=&nbsp;<span style="color:navy;">Game</span>&nbsp;winner </pre> </p> <p> While this passes all tests, it's obviously not a complete implementation. </p> <p> <strong>Getting even</strong> </p> <p> Another outcome of a Forty score is that if the other player has thirty points, and wins the ball, the new score is <em>deuce</em>. Expressing this as a property is only slightly more involved: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;player:&nbsp;40&nbsp;-&nbsp;other:&nbsp;30&nbsp;when&nbsp;other&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(current&nbsp;:&nbsp;<span style="color:#4ec9b0;">FortyData</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;current&nbsp;=&nbsp;{&nbsp;current&nbsp;<span style="color:blue;">with</span>&nbsp;OtherPlayerPoint&nbsp;=&nbsp;<span style="color:navy;">Thirty</span>&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;(<span style="color:navy;">other</span>&nbsp;current.Player) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Deuce</span>&nbsp;=!&nbsp;actual</pre> </p> <p> In this test case, the other player specifically has thirty points. There's no variability involved, so you can simply set OtherPlayerPoint to Thirty. </p> <p> Notice that instead of creating a new FortyData value from scratch, this property takes <code>current</code> (which is generated by FsCheck) and uses a copy-and-update expression to explicitly bind OtherPlayerPoint to Thirty. This ensures that all other values of <code>current</code> can vary, but OtherPlayerPoint is fixed. </p> <p> The first line of code shadows the <code>current</code> argument by binding the result of the copy-and-update expression to a new value, also called <code>current</code>. Shadowing means that the original <code>current</code> argument is no longer available in the rest of the scope. This is exactly what you want, because the function argument isn't guaranteed to model the test case where the other player has forty points. Instead, it can be any FortyData value. You can think of the argument provided by FsCheck as a seed used to arrange the <a href="https://en.wikipedia.org/wiki/Test_fixture">Test Fixture</a>. </p> <p> The property proceeds to invoke the scoreWhenForty function with the current score, and indicating that the <code>other</code> player wins the ball. You saw the <code>other</code> function in <a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">the previous article</a>, but it's so small that it can be repeated here without taking up much space: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">other</span>&nbsp;=&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerOne</span> </pre> </p> <p> Finally, the property asserts that <em>deuce must equal actual</em>. In other words, the expected result is <em>deuce</em>. </p> <p> This property fails until you fix the implementation: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;winner&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;current.Player&nbsp;=&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Game</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Deuce</span></pre> </p> <p> This is a step in the right direction, but still not the complete implementation. If the other player has only love or fifteen points, and wins the ball, the new score shouldn't be deuce. </p> <p> <strong>Incrementing points</strong> </p> <p> The last test case is where it gets interesting. The situation you need to test is that one of the players has forty points, and the other player has <em>either</em> love or fifteen points. This feels like the previous case, but has more variable parts. In the previous test case (above), the other player <em>always</em> had thirty points, but in this test case, the other player's points can vary within a constrained range. </p> <p> Perhaps you've noticed that so far, you haven't seen any examples of using FsCheck's API. Until now, we've been able to express properties from values generated by FsCheck without constraints. This is no longer possible, but fortunately, FsCheck comes with an excellent API that enables you to configure it. Here, you'll see how to configure it to create values from a proper subset of all possible values: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;player:&nbsp;40&nbsp;-&nbsp;other:&nbsp;&lt;&nbsp;30&nbsp;when&nbsp;other&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(current&nbsp;:&nbsp;<span style="color:#4ec9b0;">FortyData</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;opp&nbsp;=&nbsp;<span style="color:#4ec9b0;">Gen</span>.<span style="color:navy;">elements</span>&nbsp;[<span style="color:navy;">Love</span>;&nbsp;<span style="color:navy;">Fifteen</span>]&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Arb</span>.<span style="color:navy;">fromGen</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Prop</span>.<span style="color:navy;">forAll</span>&nbsp;opp&nbsp;(<span style="color:blue;">fun</span>&nbsp;otherPlayerPoint&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;current&nbsp;=&nbsp;{&nbsp;current&nbsp;<span style="color:blue;">with</span>&nbsp;OtherPlayerPoint&nbsp;=&nbsp;otherPlayerPoint&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;(<span style="color:navy;">other</span>&nbsp;current.Player) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">incrementPoint</span>&nbsp;current.OtherPlayerPoint &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Option</span>.<span style="color:navy;">map</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;np&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;current&nbsp;<span style="color:blue;">with</span>&nbsp;OtherPlayerPoint&nbsp;=&nbsp;np&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Option</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">Forty</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;<span style="color:navy;">Some</span>&nbsp;actual)</pre> </p> <p> That's only nine lines of code, and some of it is similar to the previous property you saw (above). Still, F# code can have a high degree of information density, so I'll walk you through it. </p> <p> FsCheck's API is centred around <em>Generators</em> and <em>Arbitraries</em>. Ultimately, when you need to configure FsCheck, you'll need to define an Arbitrary&lt;'a&gt;, but you'll often use a Gen&lt;'a&gt; value to do that. In this test case, you need to tell FsCheck to use only the values Love and Fifteen when generating Point values. </p> <p> This is done in the first line of code. <code>Gen.elements</code> creates a Generator that creates random values by drawing from a sequence of possible values. Here, we pass it a list of two values: Love and Fifteen. Because both of these values are of the type Point, the result is a value of the type Gen&lt;Point&gt;. This Generator is piped to Arb.fromGen, which turns it into an Arbitrary (for now, you don't have to worry about exactly what that means). Thus, <code>opp</code> is a value of type Arbitrary&lt;Point&gt;. </p> <p> You can now take that Arbitrary and state that, for all values created by it, a particular property must hold. This is what Prop.forAll does. The first argument passed is <code>opp</code>, and the second argument is a function that expresses the property. When the test runs, FsCheck call this function 100 times (by default), each time passing a random value generated by <code>opp</code>. </p> <p> The next couple of lines are similar to code you've seen before. As in the case where the other player had thirty points, you can shadow the <code>current</code> argument with a new value where the other player's points is set to a value drawn from <code>opp</code>; that is, Love or Fifteen. </p> <p> Notice how the original <code>current</code> value comes from an argument to the containing test function, whereas <code>otherPlayerPoint</code> comes from the <code>opp</code> Arbitrary. FsCheck is smart enough to enable this combination, so you still get the variation implied by combining these two sources of random data. </p> <p> The <code>actual</code> value is bound to the result of calling scoreWhenForty with the current score, and indicating that the other player wins the ball. </p> <p> The expected outcome is a new Forty value that originates from <code>current</code>, but with the other player's points incremented. There is, however, something odd-looking going on with Option.map - and where did that <code>incrementPoint</code> function come from? </p> <p> In the previous article, you saw how sometimes, a test triggers the creation of a new helper function. Sometimes, such a helper function is of such general utility that it makes sense to put it in the 'production code'. Previously, it was the <code>other</code> function. Now, it's the <code>incrementPoint</code> function. </p> <p> Before I show you the implementation of the <code>incrementPoint</code> function, I would like to suggest that you reflect on it. The purpose of this function is to return the point that comes after a given point. Do you remember how, in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">the article on designing with types</a>, we quickly realised that love, fifteen, thirty, and forty are mere labels; that we don't need to do arithmetic on these values? </p> <p> There's one piece of 'arithmetic' you need to do with these values, after all: you must be able to 'add one' to a value, in order to get the next value. That's the purpose of the <code>incrementPoint</code> function: given Love, it'll return Fifteen, and so on - but with a twist! </p> <p> What should it return when given Thirty as input? Forty? That's not possible. There's no Point value higher than Thirty. Forty doesn't exist. </p> <p> The object-oriented answer would be to throw an exception, but in functional programming, we don't like such arbitrary jump statements in our code. GOTO is, after all, considered harmful. </p> <p> Instead, we can return None, but that means that we must wrap all the other return values in Some: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">incrementPoint</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Love</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Some</span>&nbsp;<span style="color:navy;">Fifteen</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Fifteen</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Some</span>&nbsp;<span style="color:navy;">Thirty</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Thirty</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> This function has the type <code>Point -&gt; Point option</code>, and it behaves like this: </p> <p> <pre>&gt; incrementPoint Love;; val it : Point option = Some Fifteen &gt; incrementPoint Fifteen;; val it : Point option = Some Thirty &gt; incrementPoint Thirty;; val it : Point option = None</pre> </p> <p> Back to the property: the <code>expected</code> value is that the other player's points are incremented, and this new points value (<code>np</code>, for <em>New Points</em>) is bound to OtherPlayerPoint in a copy-and-update expression, using <code>current</code> as the original value. In other words, this expression returns the <code>current</code> score, with the only change that OtherPlayerPoint now has the incremented Point value. </p> <p> This has to happen inside of an Option.map, though, because incrementPoint may return None. Furthermore, the new value created from <code>current</code> is of the type FortyData, but you need a Forty value. This can be achieved by piping the option value into another map that composes the Forty case constructor. </p> <p> The <code>expected</code> value has the type <code>Score option</code>, so in order to be able to compare it to <code>actual</code>, which is 'only' a <code>Score</code> value, you need to make <code>actual</code> a <code>Score option</code> value as well. This is the reason <code>expected</code> is compared to <code>Some actual</code>. </p> <p> One implementation that passes this and all previous properties is: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenForty</span>&nbsp;current&nbsp;winner&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;current.Player&nbsp;=&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Game</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;<span style="color:navy;">incrementPoint</span>&nbsp;current.OtherPlayerPoint&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">Some</span>&nbsp;p&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Forty</span>&nbsp;{&nbsp;current&nbsp;<span style="color:blue;">with</span>&nbsp;OtherPlayerPoint&nbsp;=&nbsp;p&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">None</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Deuce</span></pre> </p> <p> Notice that the implementation also uses the incrementPoint function. </p> <p> <strong>To be continued...</strong> </p> <p> In this article, you saw how, even when illegal states are unrepresentable, you may need to further constrain the input into a property in order to express a particular test case. The FsCheck combinator library can be used to do that. It's flexible and well thought-out. </p> <p> While I could go on and show you how to express properties for more state transitions, you've now seen the most important techniques. If you want to see more of the tennis state transitions, you can always check out <a href="https://github.com/ploeh/KataTennis">the source code accompanying this article series</a>. </p> <p> <a href="http://blog.ploeh.dk/2016/02/16/types-properties-software-composition">In the next article, instead, you'll see how to compose all these functions</a> into a system that implements the tennis scoring rules. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: properties for the advantage state http://blog.ploeh.dk/2016/02/12/types--properties--software-properties-for-the-advantage-state 2016-02-12T08:41:00+00:00 Mark Seemann <div id="post"> <p> <em>An example of Property-Based Test-Driven Development.</em> </p> <p> This article is the third in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/11/types-properties-software-state-transition-properties">previous article</a>, you saw how to get started with Property-Based Testing, using a Test-Driven Development tactic. In this article, you'll see the previous Tennis Kata example continued. This time, you'll see how to express properties for the state when one of the players have the advantage. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Winning the game</strong> </p> <p> When one of the players have the advantage in tennis, the result can go one of two ways: either the player with the advantage wins the ball, in which case he or she wins the game, or the other player wins, in which case the next score is <em>deuce</em>. This implies that you'll have to write at least two properties: one for each situation. Let's start with the case where the advantaged player wins the ball. Using FsCheck, you can express that property like this: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;advantage&nbsp;when&nbsp;advantaged&nbsp;player&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(advantagedPlayer&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;=&nbsp;<span style="color:navy;">scoreWhenAdvantage</span>&nbsp;advantagedPlayer&nbsp;advantagedPlayer &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:navy;">Game</span>&nbsp;advantagedPlayer &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> As explained in the previous article, <a href="http://fscheck.github.io/FsCheck">FsCheck</a> will interpret this function and discover that it'll need to generate arbitrary Player values for the <code>advantagedPlayer</code> argument. Because <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">illegal states are unrepresentable</a>, you're guaranteed valid values. </p> <p> This property calls the <code>scoreWhenAdvantage</code> function (that doesn't yet exist), passing <code>advantagedPlayer</code> as argument twice. The first argument is an indication of the current score. The scoreWhenAdvantage function only models how to transition out of the Advantage case. The data associated with the Advantage case is the player currently having the advantage, so passing in advantagedPlayer as the first argument describes the current state to the function. </p> <p> The second argument is the winner of the ball. In this test case, the same player wins again, so advantagedPlayer is passed as the second argument as well. The exact value of advantagedPlayer doesn't matter; this property holds for all (two) players. </p> <p> The <code>=!</code> operator is a custom operator defined by <a href="http://www.swensensoftware.com/unquote">Unquote</a>, an assertion library. You can read the expression as <em>expected must equal actual</em>. </p> <p> In order to pass this property, you can implement the function in the simplest way that could possibly work: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenAdvantage</span>&nbsp;advantagedPlayer&nbsp;winner&nbsp;=&nbsp;<span style="color:navy;">Game</span>&nbsp;advantagedPlayer </pre> </p> <p> Here, I've arbitrarily chosen to return <code>Game advantagedPlayer</code>, but as an alternative, <code>Game winner</code> also passes the test. </p> <p> <strong>Back to deuce</strong> </p> <p> The above implementation of scoreWhenAdvantage is obviously incorrect, because it always claims that the advantaged player wins the game, regardless of who wins the ball. You'll need to describe the other test case as well, which is slightly more involved, yet still easy: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;advantage&nbsp;when&nbsp;other&nbsp;player&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(advantagedPlayer&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:navy;">scoreWhenAdvantage</span>&nbsp;advantagedPlayer&nbsp;(<span style="color:navy;">other</span>&nbsp;advantagedPlayer) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Deuce</span>&nbsp;=!&nbsp;actual</pre> </p> <p> The first argument to the scoreWhenAdvantage function describes the current score. The test case is that the other player wins the ball. In order to figure out who the other player is, you can call the <code>other</code> function. </p> <p> Which <code>other</code> function? </p> <p> The function you only now create for this express purpose: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">other</span>&nbsp;=&nbsp;<span style="color:blue;">function</span>&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerOne</span> </pre> </p> <p> As the name suggests, this function returns the other player, for any given player. In the previous article, I promised to avoid <a href="https://en.wikipedia.org/wiki/Tacit_programming">point-free style</a>, but here I broke that promise. This function is equivalent to this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">other</span>&nbsp;player&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;player&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerTwo</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">PlayerOne</span></pre> </p> <p> I decided to place this function in the same module as the scoreWhenGame function, because it seemed like a generally useful function, more than a test-specific function. It turns out that the tennis score module, indeed, does need this function later. </p> <p> Since the <code>other</code> function is part of the module being tested, shouldn't you test it as well? For now, I'll leave it uncovered by directed tests, because it's so simple that I'm confident that it works, just by looking at it. Later, I can <a href="http://blog.ploeh.dk/2016/02/19/types-properties-software-other-properties">return to it in order to add some properties</a>. </p> <p> With the <code>other</code> function in place, the new property fails, so you need to change the implementation of scoreWhenAdvantage in order to pass all tests: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenAdvantage</span>&nbsp;advantagedPlayer&nbsp;winner&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;advantagedPlayer&nbsp;=&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Game</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">Deuce</span></pre> </p> <p> This implementation deals with all combinations of input. </p> <p> It turned out that, in this case, two properties are all we need in order to describe which tennis score comes after <em>advantage</em>. </p> <p> <strong>To be continued...</strong> </p> <p> In this article, you saw how to express the properties associated with the <em>advantage</em> state of a tennis game. These properties were each simple. You can express each of them based on any arbitrary input of the given type, as shown here. </p> <p> Even when all test input values are guaranteed to be valid, sometimes you need to manipulate an arbitrary test value in order to describe a particular test case. You'll see how to do this in <a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">the next article</a>. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: state transition properties http://blog.ploeh.dk/2016/02/11/types--properties--software-state-transition-properties 2016-02-11T08:54:00+00:00 Mark Seemann <div id="post"> <p> <em>Specify valid state transitions as properties.</em> </p> <p> This article is the second in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In the <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">previous article</a>, you saw how to design with types so that <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">illegal states are unrepresentable</a>. In this article, you'll see an example of how to express properties for transitions between legal states. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> This article continues the Tennis Kata example from the previous article. It uses <a href="http://fscheck.github.io/FsCheck">FsCheck</a> to generate test values, and <a href="http://xunit.github.io">xUnit.net</a> as the overall test framework. It also uses <a href="http://www.swensensoftware.com/unquote">Unquote</a> for assertions. </p> <p> <strong>State transitions</strong> </p> <p> While the types defined in the previous article make illegal states unrepresentable, they don't enforce any rules about how to transition from one state into another. There's yet no definition of what a <em>state transition</em> is, in the tennis domain. Let's make a definition, then. </p> <p> A state transition should be a function that takes a current Score and the winner of a ball and returns a new Score. More formally, it should have the type <code>Score -&gt; Player -&gt; Score</code>. </p> <p> (If you're thinking that all this terminology sounds like we're developing a finite state machine, you're bang on; that's exactly the case.) </p> <p> For a simple domain like tennis, it'd be possible to define properties directly for such a function, but I often prefer to define a smaller function for each case, and test the properties of each of these functions. When you have all these small functions, you can easily combine them into the desired state transition function. This is the strategy you'll see in use here. </p> <p> <strong>Deuce property</strong> </p> <p> The tennis types defined in the previous article guarantee that when you ask FsCheck to generate values, you will get only legal values. This makes it easy to express properties for transitions. Let's write the property first, and let's start with the simplest state transition: the transition out of <em>deuce</em>. </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Property</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Given&nbsp;deuce&nbsp;when&nbsp;player&nbsp;wins&nbsp;then&nbsp;score&nbsp;is&nbsp;correct``</span> &nbsp;&nbsp;&nbsp;&nbsp;(winner&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;:&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;=&nbsp;<span style="color:navy;">scoreWhenDeuce</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:navy;">Advantage</span>&nbsp;winner &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> This test exercises a transition function called <code>scoreWhenDeuce</code>. The case of <em>deuce</em> is special, because there's no further data associated with the state; when the score is deuce, it's deuce. This means that when calling scoreWhenDeuce, you don't have to supply the current state of the game; it's implied by the function itself. </p> <p> You do need, however, to pass a Player argument in order to state which player won the ball. Instead of coming up with some hard-coded examples, the test simply requests Player values from FsCheck (by requiring the <code>winner</code> function argument). </p> <p> Because the Player type makes illegal states unrepresentable, you're guaranteed that only valid Player values will be passed as the <code>winner</code> argument. </p> <p> (In this particular example, Player can only be the values PlayerOne and PlayerTwo. FsCheck will, because of its default settings, run the property function 100 times, which means that it will generate 100 Player values. With an even distribution, that means that it will generate approximately 50 PlayerOne values, and 50 PlayerTwo values. Wouldn't it be easier, and faster, to use a <code>[&lt;Theory&gt;]</code> that deterministically generates only those two values, without duplication? Yes, in this case it would; <a href="http://blog.ploeh.dk/2015/02/23/property-based-testing-without-a-property-based-testing-framework">This sometimes happens, and it's okay</a>. In this example, though, I'm going to keep using FsCheck, because I think this entire example is a good stand-in for a more complex business problem.) </p> <p> Regardless of the value of the <code>winner</code> argument, the property should hold that the return value of the <code>scoreWhenDeuce</code> function is that the winner now has the advantage. </p> <p> The <code>=!</code> operator is a custom operator defined by Unquote. You can read it as a <em>must equal</em> operator: <em>expected must equal actual</em>. </p> <p> <strong>Red phase</strong> </p> <p> When you apply Test-Driven Development, you should follow the Red/Green/Refactor cycle. In this example, we're doing just that, but at the moment the code doesn't even compile, because there's no scoreWhenDeuce function. </p> <p> In the Red phase, it's important to observe that the test fails as expected before moving on to the Green phase. In order to make that happen, you can create this temporary implementation of the function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenDeuce</span>&nbsp;winner&nbsp;=&nbsp;<span style="color:navy;">Deuce</span> </pre> </p> <p> With this definition, the code compiles, and when you run the test, you get this test failure: </p> <p> <pre>Test 'Ploeh.Katas.TennisProperties.Given deuce when player wins then score is correct' failed: FsCheck.Xunit.PropertyFailedException : Falsifiable, after 1 test (0 shrinks) (StdGen (427100699,296115298)): Original: PlayerOne ---- Swensen.Unquote.AssertionFailedException : Test failed: Advantage PlayerOne = Deuce false</pre> </p> <p> This test failure is the expected failure, so you should now feel confident that the property is correctly expressed. </p> <p> <strong>Green phase</strong> </p> <p> With the Red phase properly completed, it's time to move on to the Green phase: make the test(s) pass. For deuce, this is trivial: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenDeuce</span>&nbsp;winner&nbsp;=&nbsp;<span style="color:navy;">Advantage</span>&nbsp;winner </pre> </p> <p> This passes 'all' tests: </p> <p> <pre>Output from Ploeh.Katas.TennisProperties.Given deuce when player wins then score is correct: Ok, passed 100 tests.</pre> </p> <p> The property holds. </p> <p> <strong>Refactor</strong> </p> <p> When you follow the Red/Green/Refactor cycle, you should now refactor the implementation, but there's little to do at this point </p> <p> You could, in fact, perform an <em>eta reduction</em>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">scoreWhenDeuce</span>&nbsp;=&nbsp;<span style="color:navy;">Advantage</span> </pre> </p> <p> This would be <a href="http://blog.ploeh.dk/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> in Haskell, but not quite so much in F#. In my experience, many people find the <a href="https://en.wikipedia.org/wiki/Tacit_programming">point-free style</a> less readable, so I'm not going to pursue this type of refactoring for the rest of this article series. </p> <p> <strong>To be continued...</strong> </p> <p> In this article, you've seen how to express the single property that when the score is deuce, the winner of the next ball will have the advantage. Because illegal states are unrepresentable, you can declaratively state the type of value(s) the property requires, and FsCheck will have no choice but to give you valid values. </p> <p> <a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">In the next article, you'll see how to express properties for slightly more complex state transitions</a>. In this article, I took care to spell out each step in the process, but in the next article, I promise to increase the pace. </p> <p> If you're interested in learning more about Property-Based Testing, you can watch my <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software: designing with types http://blog.ploeh.dk/2016/02/10/types--properties--software-designing-with-types 2016-02-10T12:27:00+00:00 Mark Seemann <div id="post"> <p> <em>Design types so that illegal states are unrepresentable.</em> </p> <p> This article is the first in <a href="http://blog.ploeh.dk/2016/02/10/types-properties-software">a series of articles that demonstrate how to develop software using types and properties</a>. In this article, you'll see an example of how to design with algebraic data types, and in future articles, you'll see how to specify properties that must hold for the system. This example uses F#, but you can design similar types in Haskell. </p> <p> The source code for this article series is <a href="https://github.com/ploeh/KataTennis">available on GitHub</a>. </p> <p> <strong>Tennis</strong> </p> <p> The example used in this series of articles is <a href="http://www.codingdojo.org/cgi-bin/index.pl?KataTennis">the Tennis Kata</a>. Although a tennis match consists of multiple <em>sets</em> that again are played as several <em>games</em>, in the kata, you only have to implement the scoring system for a single game: <ul> <li>Each player can have either of these points in one game: Love, 15, 30, 40.</li> <li>If you have 40 and you win the ball, you win the game. There are, however, special rules.</li> <li>If both have 40, the players are <em>deuce</em>.</li> <ul> <li>If the game is in deuce, the winner of a ball will have advantage and game ball.</li> <li>If the player with advantage wins the ball, (s)he wins the game.</li> <li>If the player without advantage wins, they are back at deuce.</li> </ul> </ul> This problem is easy enough that it's fun to play with, but difficult enough that it's fun to play with. </p> <p> You can take on this problem in many different ways, but in this article, you'll see how F#'s type system can be used to <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">make illegal states unrepresentable</a>. Perhaps you think this is overkill for such a simple problem, but think of the Tennis Kata as a stand-in for a more complex domain problem. </p> <p> <strong>Players</strong> </p> <p> In tennis, there are two players, which we can easily model with a discriminated union: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Player</span>&nbsp;=&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span> </pre> </p> <p> When designing with types, I often experiment with values in FSI (the F# REPL) to figure out if they make sense. For such a simple type as Player, that's not strictly necessary, but I'll show you anyway, take illustrate the point: </p> <p> <pre>&gt; PlayerOne;; val it : Player = PlayerOne &gt; PlayerTwo;; val it : Player = PlayerTwo &gt; PlayerZero;; PlayerZero;; ^^^^^^^^^^ error FS0039: The value or constructor 'PlayerZero' is not defined &gt; PlayerThree;; PlayerThree;; ^^^^^^^^^^^ error FS0039: The value or constructor 'PlayerThree' is not defined</pre> </p> <p> As you can see, both <code>PlayerOne</code> and <code>PlayerTwo</code> are values inferred to be of the type Player, whereas both <code>PlayerZero</code> and <code>PlayerThree</code> are illegal expressions. </p> <p> Not only is it possible to represent all valid values, but illegal values are unrepresentable. Success. </p> <p> <strong>Naive point attempt with a type alias</strong> </p> <p> If you're unfamiliar with designing with types, you may briefly consider using a type alias to model players' points: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;=&nbsp;<span style="color:#4ec9b0;">int</span> </pre> </p> <p> This easily enables you to model some of the legal point values: </p> <p> <pre>&gt; let p : Point = 15;; val p : Point = 15 &gt; let p : Point = 30;; val p : Point = 30</pre> </p> <p> It looks good so far, but how do you model <em>love</em>? It's not really an integer. </p> <p> Still, both players start with <em>love</em>, so it's intuitive to try to model <em>love</em> as 0: </p> <p> <pre>&gt; let p : Point = 0;; val p : Point = 0</pre> </p> <p> It's a hack, but it works. </p> <p> Are illegal values unrepresentable? Hardly: </p> <p> <pre>&gt; let p : Point = 42;; val p : Point = 42 &gt; let p : Point = -1337;; val p : Point = -1337</pre> </p> <p> With a type alias, it's possible to assign every value that the 'real' type supports. For a 32-bit integer, this means that we have four legal representations (0, 15, 30, 40), and 4,294,967,291 illegal representations of a tennis point. Clearly this doesn't meet the goal of making illegal states unrepresentable. </p> <p> <strong>Second point attempt with a discriminated Union</strong> </p> <p> If you think about the problem for a while, you're likely to come to the realisation that love, 15, 30, and 40 aren't numbers, but rather <em>labels</em>. No arithmetic is performed on them. It's easy to constrain the domain of points with a discriminated union: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;=&nbsp;<span style="color:navy;">Love</span>&nbsp;|&nbsp;<span style="color:navy;">Fifteen</span>&nbsp;|&nbsp;<span style="color:navy;">Thirty</span>&nbsp;|&nbsp;<span style="color:navy;">Forty</span> </pre> </p> <p> You can play around with values of this Point type in FSI if you will, but there should be no surprises. </p> <p> A Point value isn't a score, though. A score is a representation of a state in the game, with (amongh other options) a point to each player. You can model this with a record: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">PointsData</span>&nbsp;=&nbsp;{&nbsp;PlayerOnePoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>;&nbsp;PlayerTwoPoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;} </pre> </p> <p> You can experiment with this type in FSI, like you can with the other types: </p> <p> <pre>&gt; { PlayerOnePoint = Love; PlayerTwoPoint = Love };; val it : PointsData = {PlayerOnePoint = Love; PlayerTwoPoint = Love;} &gt; { PlayerOnePoint = Love; PlayerTwoPoint = Thirty };; val it : PointsData = {PlayerOnePoint = Love; PlayerTwoPoint = Thirty;} &gt; { PlayerOnePoint = Thirty; PlayerTwoPoint = Forty };; val it : PointsData = {PlayerOnePoint = Thirty; PlayerTwoPoint = Forty;}</pre> </p> <p> That looks promising. What happens if players are evenly matched? </p> <p> <pre>&gt; { PlayerOnePoint = Forty; PlayerTwoPoint = Forty };; val it : PointsData = {PlayerOnePoint = Forty; PlayerTwoPoint = Forty;}</pre> </p> <p> That works as well, but it shouldn't! </p> <p> <em>Forty-forty</em> isn't a valid tennis score; it's called <em>deuce</em>. </p> <p> Does it matter? you may ask. Couldn't we 'just' say that forty-forty 'means' deuce and get on with it? </p> <p> You could. You should try. I've tried doing this (I've done the Tennis Kata several times, in various languages), and it turns out that it's possible, but it <em>makes the code more complicated</em>. One of the reasons is that deuce is more than a synonym for forty-forty. The score can also be deuce after <em>advantage</em> to one of the players. You can, for example, have a score progression like this: <ul> <li>...</li> <li>Forty-thirty</li> <li>Deuce</li> <li>Advantage to player one</li> <li>Deuce</li> <li>Advantage to player two</li> <li>Deuce</li> <li>...</li> </ul> While you may prefer to think of the first occurrence of <em>deuce</em> in that list as <em>forty-forty</em>, the next occurrences clearly don't correspond to <em>forty-forty</em>. </p> <p> Additionally, if you're into <a href="http://amzn.to/WBCwx7">Domain-Driven Design</a>, you prefer using the ubiquitous language of the domain. When the tennis domain language says that it's not called <em>forty-forty</em>, but <em>deuce</em>, the code should reflect that. </p> <p> <strong>Final attempt at a point type</strong> </p> <p> Fortunately, you don't have to throw away everything you've done so far. The love-love, fifteen-love, etc. values that you can represent with the above PointsData type are all valid. Only when you approach the boundary value of forty do problems appear. </p> <p> A solution is to remove the offending Forty case from Point. The updated definition of Point is this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;=&nbsp;<span style="color:navy;">Love</span>&nbsp;|&nbsp;<span style="color:navy;">Fifteen</span>&nbsp;|&nbsp;<span style="color:navy;">Thirty</span> </pre> </p> <p> You can still represent the 'early' scores using PointsData: </p> <p> <pre>&gt; { PlayerOnePoint = Love; PlayerTwoPoint = Love };; val it : PointsData = {PlayerOnePoint = Love; PlayerTwoPoint = Love;} &gt; { PlayerOnePoint = Love; PlayerTwoPoint = Thirty };; val it : PointsData = {PlayerOnePoint = Love; PlayerTwoPoint = Thirty;}</pre> </p> <p> Additionally, the illegal forty-forty state is now unrepresentable: </p> <p> <pre>&gt; { PlayerOnePoint = Forty; PlayerTwoPoint = Forty };; { PlayerOnePoint = Forty; PlayerTwoPoint = Forty };; -------------------^^^^^ error FS0039: The value or constructor 'Forty' is not defined</pre> </p> <p> This is much better, apart from the elephant in the room: you also lost the ability to model valid states where only one of the players have forty points: </p> <p> <pre>&gt; { PlayerOnePoint = Thirty; PlayerTwoPoint = Forty };; { PlayerOnePoint = Thirty; PlayerTwoPoint = Forty };; --------------------------------------------^^^^^ error FS0039: The value or constructor 'Forty' is not defined</pre> </p> <p> Did we just throw the baby out with the bathwater? </p> <p> Not really, because we weren't close to being done anyway. While we were making progress on modelling the score as a pair of Point values, remaining work includes modelling deuce, advantage and winning the game. </p> <p> <strong>Life begins at forty</strong> </p> <p> At this point, it may be helpful to recap what we have: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Player</span>&nbsp;=&nbsp;<span style="color:navy;">PlayerOne</span>&nbsp;|&nbsp;<span style="color:navy;">PlayerTwo</span> <span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;=&nbsp;<span style="color:navy;">Love</span>&nbsp;|&nbsp;<span style="color:navy;">Fifteen</span>&nbsp;|&nbsp;<span style="color:navy;">Thirty</span> <span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">PointsData</span>&nbsp;=&nbsp;{&nbsp;PlayerOnePoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>;&nbsp;PlayerTwoPoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;}</pre> </p> <p> While this enables you to keep track of the score when both players have less than forty points, the following phases of a game still remain: <ul> <li>One of the players have forty points.</li> <li>Deuce.</li> <li>Advantage to one of the players.</li> <li>One of the players won the game.</li> </ul> You can design the first of these with another record type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">FortyData</span>&nbsp;=&nbsp;{&nbsp;Player&nbsp;:&nbsp;<span style="color:#4ec9b0;">Player</span>;&nbsp;OtherPlayerPoint&nbsp;:&nbsp;<span style="color:#4ec9b0;">Point</span>&nbsp;} </pre> </p> <p> This is a record that specifically keeps track of the situation where <em>one</em> of the players have forty points. The Player element keeps track of which player has forty points, and the OtherPlayerPoint element keeps track of the other player's score. For instance, this value indicates that PlayerTwo has forty points, and PlayerOne has thirty: </p> <p> <pre>&gt; { Player = PlayerTwo; OtherPlayerPoint = Thirty };; val it : FortyData = {Player = PlayerTwo; OtherPlayerPoint = Thirty;}</pre> </p> <p> This is a legal score. Other values of this type exist, but none of them are illegal. </p> <p> <strong>Score</strong> </p> <p> Now you have two distinct types, PointsData and FortyData, that keep track of the score at two different phases of a tennis game. You still need to model the remaining three phases, and somehow turn all of these into a single type. This is an undertaking that can be surprisingly complicated in C# or Java, but is trivial to do with a discriminated union: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Score</span>&nbsp;= |&nbsp;<span style="color:navy;">Points</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">PointsData</span> |&nbsp;<span style="color:navy;">Forty</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">FortyData</span> |&nbsp;<span style="color:navy;">Deuce</span> |&nbsp;<span style="color:navy;">Advantage</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">Player</span> |&nbsp;<span style="color:navy;">Game</span>&nbsp;<span style="color:blue;">of</span>&nbsp;<span style="color:#4ec9b0;">Player</span></pre> </p> <p> This Score type states that a score is a value in one of these five cases. As an example, the game starts with both players at (not necessarily <em>in</em>) love: </p> <p> <pre>&gt; Points { PlayerOnePoint = Love; PlayerTwoPoint = Love };; val it : Score = Points {PlayerOnePoint = Love; PlayerTwoPoint = Love;}</pre> </p> <p> If, for example, PlayerOne has forty points, and PlayerTwo has thirty points, you can create this value: </p> <p> <pre>&gt; Forty { Player = PlayerOne; OtherPlayerPoint = Thirty };; val it : Score = Forty {Player = PlayerOne; OtherPlayerPoint = Thirty;}</pre> </p> <p> Notice how both values are of the same type (Score), even though they don't share the same data structure. Other example of valid values are: </p> <p> <pre>&gt; Deuce;; val it : Score = Deuce &gt; Advantage PlayerTwo;; val it : Score = Advantage PlayerTwo &gt; Game PlayerOne;; val it : Score = Game PlayerOne</pre> </p> <p> This model of the tennis score system enables you to express all legal values, while making illegal states unrepresentable. </p> <p> It's impossible to express that the score is seven-eleven: </p> <p> <pre>&gt; Points { PlayerOnePoint = Seven; PlayerTwoPoint = Eleven };; Points { PlayerOnePoint = Seven; PlayerTwoPoint = Eleven };; --------------------------^^^^^ error FS0039: The value or constructor 'Seven' is not defined</pre> </p> <p> It's impossible to state that the score is fifteen-thirty-fifteen: </p> <p> <pre>&gt; Points { PlayerOnePoint = Fifteen; PlayerTwoPoint = Thirty; PlayerThreePoint = Fifteen };; Points { PlayerOnePoint = Fifteen; PlayerTwoPoint = Thirty; PlayerThreePoint = Fifteen };; ------------------------------------------------------------^^^^^^^^^^^^^^^^ error FS1129: The type 'PointsData' does not contain a field 'PlayerThreePoint'</pre> </p> <p> It's impossible to express that both players have forty points: </p> <p> <pre>&gt; Points { PlayerOnePoint = Forty; PlayerTwoPoint = Forty };; Points { PlayerOnePoint = Forty; PlayerTwoPoint = Forty };; --------------------------^^^^^ error FS0001: This expression was expected to have type Point but here has type FortyData -> Score &gt; Forty { Player = PlayerOne; OtherPlayerPoint = Forty };; Forty { Player = PlayerOne; OtherPlayerPoint = Forty };; -----------------------------------------------^^^^^ error FS0001: This expression was expected to have type Point but here has type FortyData -> Score</pre> </p> <p> In the above example, I even attempted to express forty-forty in two different ways, but none of them work. </p> <p> <strong>Summary</strong> </p> <p> You now have a model of the domain that enables you to express valid values, but that makes illegal states unrepresentable. This is half the battle won, without any moving parts. These types govern what can be stated in the domain, but they don't provide any rules for how values can transition from one state to another. </p> <p> This is a task you can take on with Property-Based Testing. Since all values of these types are valid, it's easy to express the properties of a tennis game score. In <a href="http://blog.ploeh.dk/2016/02/11/types-properties-software-state-transition-properties">the next article, you'll see how to start that work</a>. </p> <p> If you're interested in learning more about designing with types, you can watch my <a href="http://bit.ly/type-driven-development">Type-Driven Development with F#</a> Pluralsight course. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Types + Properties = Software http://blog.ploeh.dk/2016/02/10/types--properties--software 2016-02-10T11:50:00+00:00 Mark Seemann <div id="post"> <p> <em>Combine a strong type system with Property-Based Testing to specify software.</em> </p> <p> When Kent Beck <a href="https://www.quora.com/Why-does-Kent-Beck-refer-to-the-rediscovery-of-test-driven-development">rediscovered Test-Driven Development</a> (TDD) some twenty years ago, the original context was SmallTalk programming. When the concept of TDD began to catch on, it seemed to me to proliferate particularly in dynamic language communities like Python and Ruby. </p> <p> It makes sense that if you can't get fast feedback from a compilation step, you seek another way to get feedback on your work. <a href="http://blog.ploeh.dk/2011/04/29/Feedbackmechanismsandtradeoffs">Unit testing is such a fast feedback mechanism</a>. Watching the NodeJS community from the side, it always seemed to me that TDD is an integral way of working with that stack. This makes sense. </p> <p> This also explains why e.g. C# programmers were more reluctant to adopt TDD. That your code compiles gives a smidgen of confidence. </p> <p> C# and Java programmers may joke that <em>if it compiles, it works</em>, but also realise that that's all it is: a joke. Automated testing adds another layer of safety on top of compilation. On the other hand, you may get by with writing fewer tests than in a dynamic language, because the static type system does provide <em>some</em> guarantees. </p> <p> What if you had a stronger type system than what C# or Java provides? Then you'd have to write even fewer tests. </p> <p> <strong>Type spectrum</strong> </p> <p> It can be useful to think about typing as a spectrum. </p> <p> <img src="/content/binary/type-spectrum.png" alt="A spectrum of languages, from most dynamic on the left, to most static on the right."> </p> <p> To the left, we have dynamically typed languages like JavaScript and Ruby. Python <em>can</em> be compiled, and Clojure <em>is</em> compiled, so we can think of them as being a little closer to the middle. Purists would want to point out that whether or not a language is compiled, and how statically typed it is, is two independent concerns, but the point of this spectrum is to illustrate the degree of confidence the type system gives you. </p> <p> Java and C# are statically typed, compiled languages, but a lot of things can still go wrong at run-time. Just consider all the null pointer exceptions you've encountered over the years. </p> <p> Further to the right are languages with <a href="https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_system">Hindley–Milner type systems</a>, such as F# and Haskell. These offer more safety from static typing alone. Such types are the subject of this and future articles. </p> <p> I've put Haskell a bit to the right of F# because it has higher-order types. To the right of Haskell sits dependently-typed languages like Idris. To the far right, we have a hypothetical language with such a strong types system that, indeed, <em>if it compiles, it works</em>. As far as I'm aware, no such language exist. </p> <p> <strong>Creating software with Hindley-Milner type systems</strong> </p> <p> It turns out that when you shift from a Java/C#-style type system to the stronger type system offered by F# or Haskell, you can further shift your design emphasis towards designing with types. You still need to cover behaviour with automated tests, but fewer are necessary. </p> <p> With the <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a> available in F# or Haskell, you can design your types so that <a href="http://fsharpforfunandprofit.com/posts/designing-with-types-making-illegal-states-unrepresentable">illegal states are unrepresentable</a>. In other words, all values of a particular type are guaranteed to be valid within the domain they model. This makes it much easier to test the behaviour of your system with Property-Based Testing, because you can declaratively state that you want your Property-Based Testing framework (FsCheck, QuickCheck, etc.) to provide random values of a given type. The framework will give you random values, but it can only give you valid values. </p> <p> In this series of articles, you'll see an example of this in F#: <ul> <li><a href="http://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">Types + Properties = Software: designing with types</a></li> <li><a href="http://blog.ploeh.dk/2016/02/11/types-properties-software-state-transition-properties">Types + Properties = Software: state transition properties</a></li> <li><a href="http://blog.ploeh.dk/2016/02/12/types-properties-software-properties-for-the-advantage-state">Types + Properties = Software: properties for the advantage state</a></li> <li><a href="http://blog.ploeh.dk/2016/02/15/types-properties-software-properties-for-the-forties">Types + Properties = Software: properties for the Forties</a></li> <li><a href="http://blog.ploeh.dk/2016/02/16/types-properties-software-composition">Types + Properties = Software: composition</a></li> <li><a href="http://blog.ploeh.dk/2016/02/17/types-properties-software-initial-state">Types + Properties = Software: initial state</a></li> <li><a href="http://blog.ploeh.dk/2016/02/18/types-properties-software-finite-state-machine">Types + Properties = Software: finite state machine</a></li> <li><a href="http://blog.ploeh.dk/2016/02/19/types-properties-software-other-properties">Types + Properties = Software: other properties</a></li> </ul> If you're interested in learning more, my <a href="http://blog.ploeh.dk/2015/08/10/type-driven-development">Type Driven Development</a> article showcases another example of designing with types. My Pluralsight course <a href="http://bit.ly/type-driven-development">Type-Driven Development with F#</a> dives even deeper than the article. If you need an <a href="http://bit.ly/1M1zXTf">introduction to Property-based Testing with F#</a>, that's also available. </p> <p> <strong>Summary</strong> </p> <p> The stronger a language's type system is, the more you can use static types to model your application's problem domain. With a sufficiently strong type system, you can make illegal states unrepresentable. While states are guaranteed to be legal, transitions between states need not be. You can often use Property-Based Testing to ensure that the state transitions are correct. The combination of types and properties give you a simple, but surprisingly powerful way of specifying the behaviour of the software you create. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Are shuffled plays random? http://blog.ploeh.dk/2016/02/06/are-shuffled-plays-random 2016-02-06T00:09:00+00:00 Mark Seemann <div id="post"> <p> <em>Have you ever asked your music player to shuffle all your music, only to wonder: didn't it play that song only yesterday?</em> </p> <p> When I work, I often listen to music. Sometimes, I make a deliberate choice, but often, I just ask my music player to shuffle my entire music library. </p> <p> It invariably happens that it plays a song it played only a couple of days before; I often notice when it's a song I don't like, because then it annoys me that it plays that stupid song <em>again</em>. Given that I have more than 16,000 tracks, and approximately 5,000 tracks on my laptop, it seems unlikely that the music player should pick the same track within only a few days. </p> <p> This is a well-known phenomenon known as <em>frequency illusion</em>, or more colloquially as the <em>Baader-Meinhof Phenomenon</em>. Your brain seeks patterns, even where no patterns exist. This is a really strong feeling, it turns out. </p> <p> When tracks are played in truly random fashion, it's as unlikely that it should play two tracks far apart as it should play them close together. Therefore, many media players come with shuffle algorithms that are <em>not</em> random, but designed to <em>feel</em> random. </p> <p> With my thousands of tracks, I couldn't shake the feeling that my player's shuffle algorithm was one of those non-random algorithms, so I decided to analyse my plays. </p> <p> <strong>Getting the data in shape</strong> </p> <p> Fortunately, I've been <a href="http://www.last.fm/user/ploeh">scrobbling my plays to Last.fm</a> since 2007, so I had comprehensive statistics to work with. </p> <p> There's <a href="http://benjaminbenben.com/lastfm-to-csv/">a third-party service where you can download all your Last.fm scrobbles</a> as a CSV file, so I did that; all 53,700 scrobbles. </p> <p> This file contains all my scrobbles since 2007, but I had to clean it up: <ul> <li>I haven't used the same music player in all those years, so I decide to only look at the entries since January 1, 2014. I know I've only used one player in that period.</li> <li>It often happens that I put on a particular artist, album, or playlist, and I had to remove those, because I wanted to look at only the entries when I was listening in shuffle mode.</li> </ul> Opening the file in Excel, and manually cleaning it up according to the above criteria was easy, and took perhaps 15 minutes. What was left was approximately 8,500 rows in a CSV file. </p> <p> <strong>Analysing with F#</strong> </p> <p> Next, I created an F# script file, and read in the file's contents: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;System <span style="color:blue;">open</span>&nbsp;System.IO <span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Track</span>&nbsp;=&nbsp;{&nbsp;Artist&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>;&nbsp;Album&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>;&nbsp;Name&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>&nbsp;} <span style="color:blue;">type</span>&nbsp;<span style="color:#4ec9b0;">Scrobble</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Track&nbsp;:&nbsp;<span style="color:#4ec9b0;">Track</span>;&nbsp;Time&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>&nbsp;} <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">parse</span>&nbsp;(line&nbsp;:&nbsp;<span style="color:#4ec9b0;">string</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;line.<span style="color:navy;">Split</span>&nbsp;<span style="color:#a31515;">&#39;|&#39;</span>&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[|&nbsp;artist;&nbsp;album;&nbsp;track;&nbsp;time&nbsp;|]&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">Some</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Track&nbsp;=&nbsp;{&nbsp;Artist&nbsp;=&nbsp;artist;&nbsp;Album&nbsp;=&nbsp;album;&nbsp;Name&nbsp;=&nbsp;track&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Time&nbsp;=&nbsp;time&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">None</span> <span style="color:blue;">let</span>&nbsp;scrobbles&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">File</span>.<span style="color:navy;">ReadAllLines</span>&nbsp;<span style="color:#a31515;">@&quot;Last.fm_ploeh_shuffled.csv&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Array</span>.<span style="color:navy;">map</span>&nbsp;<span style="color:navy;">parse</span></pre> </p> <p> This script reads the file, with each line an element in an array. It then parses each line into Scrobble values. There may be parse errors, if a line contains the wrong number of elements, but this turned out to not be the case: </p> <p> <pre>&gt; let hasParseErrors = scrobbles |> Array.exists (Option.isNone);; val hasParseErrors : bool = false</pre> </p> <p> The next step was to group the scrobbles into distinct tracks, with a count of each track: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;counts&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;scrobbles &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Array</span>.<span style="color:navy;">choose</span>&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Array</span>.<span style="color:navy;">countBy</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;s.Track) &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">Array</span>.<span style="color:navy;">sortByDescending</span>&nbsp;<span style="color:navy;">snd</span></pre> </p> <p> This gave me a sorted 'hit list', showing me the songs played the most in the top of the list. Here's the top 10: </p> <p> <pre>DJ Zen, Peace Therapy, Anamatha (Intro) : ....... Abakus, That Much Closer to the Sun, Igmatik : ....... Yoji Biomehanika, Technicolor Nrg Show, seduction (sleep tonight remix): ...... Mindsphere, Patience for Heaven. CD 2: Inner Cyclone, inner cyclone : ...... Massive Attack, Mezzanine, Black Milk : ...... Dimension 5, TransAddendum, Strange Phenomena : ...... Dire Straits, Brothers in Arms, Your Latest Trick : ...... Digicult, Out Of This World, Star Travel : ...... Infected Mushroom, Classical Mushroom, Bust A Move : ...... Brother Brown feat. Frank'ee, Under The Water, Under The Water (Or[...]: ......</pre> </p> <p> None of these, with the exception of <em>Under The Water</em>, are particular favourites of mine, so I don't think these were tracks I'd definitely selected; remember: I'd made a deliberate effort to remove such tracks before starting the analysis. </p> <p> This list tells <em>me</em> something, because I know what I like and don't like, but it doesn't tell me much about the distribution of tracks. The two most 'popular' tracks had been played 7 times over the last two years, and the remaining tracks 6 times. How about the other 5,155 entries in <code>counts</code>? How were they distributed? </p> <p> It's easy to count how many tracks were played 7 times, how many were played 6 times, 5 times, etc.: </p> <p> <pre>&gt; let frequencyCounts = counts |> Array.countBy snd;; val frequencyCounts : (int * int) [] = [|(7, 2); (6, 9); (5, 43); (4, 159); (3, 561); (2, 1475); (1, 2916)|]</pre> </p> <p> It looks as though the 'popular' tracks are rare occurrences. Most tracks (2,916) were played only once, and 1,475 only twice. This is perhaps easier to understand when visualised. </p> <p> <strong>Visualisation</strong> </p> <p> While there are great visualisation packages for F#, for this ad-hoc task, I found it easier to use Excel. After all, I only had 7 observations to plot: </p> <p> <img src="/content/binary/scrobble-frequency-plot.png" alt="A scatter plot showing the frequency of plays."> </p> <p> This figure shows that almost 3,000 tracks were scrobbled only once, approximately 1,500 were scrobbled twice, 500 or so were scrobbled thrice, and so on. </p> <p> Even if we expect the distribution to be entirely uniform, we should expect to see duplicates. While I have more than 16,000 tracks on my home system, I only have some 5,000 tracks on my laptop, and these are the tracks for which I have data. Since I had 8,500 scrobbles to analyse, there <em>must</em> be duplicates, but we shouldn't expect to see two plays being more common than one, because even with an even distribution, you can't play all tracks twice when you have 5,000 tracks, but 'only' 8,500 plays. </p> <p> That there are some 500 tracks that have been played three times don't seem unlikely in this light. Furthermore, outliers are to be expected as well; while they're there, the figure also shows that they promptly taper off to insignificance. </p> <p> Although I took theoretical statistics in university (I had to), I never liked it, and got the lowest passing grade. In other words: don't trust any of this. I haven't formally analysed it with proper statistical tools. </p> <p> <strong>Conclusion</strong> </p> <p> Still, based on this little Friday exercise, I now feel renewed confidence that my music player does, indeed, pick tracks at random when I tell it to play all songs shuffled. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Make pre-conditions explicit in Property-Based Tests http://blog.ploeh.dk/2016/01/18/make-pre-conditions-explicit-in-property-based-tests 2016-01-18T12:00:00+00:00 Mark Seemann <div id="post"> <p> <em>Pre-conditions are important in Property-Based Tests.</em> </p> <p> Last week, I was giving a workshop on Property-Based Testing, and we were looking at this test: </p> <p> <pre>[&lt;Property&gt;] <span style="color:blue;">let</span>&nbsp;``Validate.reservation&nbsp;returns&nbsp;right&nbsp;result&nbsp;on&nbsp;invalid&nbsp;date`` &nbsp;&nbsp;&nbsp;&nbsp;(rendition&nbsp;:&nbsp;ReservationRendition)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Validate.reservation&nbsp;rendition &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;:&nbsp;Result&lt;Reservation,&nbsp;Error&gt;&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Failure(ValidationError(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>)) &nbsp;&nbsp;&nbsp;&nbsp;test&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">expected</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">=</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> The test case being exercised by this test is to verify what happens when the input (essentially a representation of an incoming JSON document) is invalid. </p> <p> The System Under Test is this function: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:#4ec9b0;">Validate</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">reservation</span>&nbsp;(rendition&nbsp;:&nbsp;<span style="color:#4ec9b0;">ReservationRendition</span>)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;rendition.Date&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">DateTimeOffset</span>.<span style="color:navy;">TryParse</span>&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;(<span style="color:blue;">true</span>,&nbsp;date)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Success</span>&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;date &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;rendition.Name &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;rendition.Email &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;rendition.Quantity&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">Failure</span>(<span style="color:navy;">ValidationError</span>&nbsp;<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>)</pre> </p> <p> The validation rule is simple: if the rendition.Date string can be parsed into a DateTimeOffset value, then the input is valid; otherwise, it's not. </p> <p> If you run the test, if passes: </p> <p> <pre>Output from Ploeh.Samples.BookingApi.UnitTests.ValidatorTests.Validate.reservation returns right result on invalid date: Ok, passed 100 tests.</pre> </p> <p> Only, this test isn't <em>guaranteed</em> to pass. Can you spot the problem? </p> <p> There's a tiny, but real, risk that the randomly generated rendition.Date string <em>can</em> be parsed as a DateTimeOffset value. While it's not particularly likely, it will happen once in a blue moon. This can lead to spurious <a href="http://xunitpatterns.com/false%20positive.html">false positives</a>. </p> <p> In order to see this, you can increase the number of test cases generated by FsCheck: </p> <p> <pre>[&lt;Property(MaxTest&nbsp;=&nbsp;100000)&gt;] <span style="color:blue;">let</span>&nbsp;``Validate.reservation&nbsp;returns&nbsp;right&nbsp;result&nbsp;on&nbsp;invalid&nbsp;date`` &nbsp;&nbsp;&nbsp;&nbsp;(rendition&nbsp;:&nbsp;ReservationRendition)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Validate.reservation&nbsp;rendition &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;:&nbsp;Result&lt;Reservation,&nbsp;Error&gt;&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Failure(ValidationError(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>)) &nbsp;&nbsp;&nbsp;&nbsp;test&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">expected</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">=</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> In my experience, when you ask for 100,000 test cases, the property usually fails: </p> <p> <pre>Test 'Ploeh.Samples.BookingApi.UnitTests.ValidatorTests.Validate.reservation returns right result on invalid date' failed: FsCheck.Xunit.PropertyFailedException : Falsifiable, after 9719 tests (11 shrinks) (StdGen (1054829489,296106988)): Original: {Date = "7am"; Name = "6WUx;"; Email = "W"; Quantity = -5;} Shrunk: {Date = "7am"; Name = ""; Email = ""; Quantity = 0;} ---- Swensen.Unquote.AssertionFailedException : Test failed: Failure (ValidationError "Invalid date.") = Success {Date = 15.01.2016 07:00:00 +00:00; Name = ""; Email = ""; Quantity = 0;} false</pre> </p> <p> The problem is that the generated date "7am" <em>can</em> be parsed as a DateTimeOffset value: </p> <p> <pre>&gt; open System;; &gt; DateTimeOffset.Now;; val it : DateTimeOffset = 15.01.2016 21:09:25 +00:00 &gt; "7am" |&gt; DateTimeOffset.TryParse;; val it : bool * DateTimeOffset = (true, 15.01.2016 07:00:00 +00:00)</pre> </p> <p> The above test is written with the implicit assumption that the generated value for rendition.Date will always be an invalidly formatted string. As <a href="https://www.python.org/dev/peps/pep-0020">the Zen of Python</a> reminds us, <em>explicit is better than implicit</em>. You should make that assumption explicit: </p> <p> <pre>[&lt;Property&gt;] <span style="color:blue;">let</span>&nbsp;``Validate.reservation&nbsp;returns&nbsp;right&nbsp;result&nbsp;on&nbsp;invalid&nbsp;date`` &nbsp;&nbsp;&nbsp;&nbsp;(rendition&nbsp;:&nbsp;ReservationRendition)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;not(fst(DateTimeOffset.TryParse&nbsp;rendition.Date))&nbsp;==&gt;&nbsp;<span style="color:blue;">lazy</span> &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Validate.reservation&nbsp;rendition &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;:&nbsp;Result&lt;Reservation,&nbsp;Error&gt;&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Failure(ValidationError(<span style="color:#a31515;">&quot;Invalid&nbsp;date.&quot;</span>)) &nbsp;&nbsp;&nbsp;&nbsp;test&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">expected</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">=</span><span style="background:yellow;">&nbsp;</span><span style="background:yellow;">actual</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> This version of the test uses FsCheck's built-in custom operator <code>==&gt;</code>. This operator will only evaluate the expression on the right-hand side if the boolean expression on the left-hand side evaluates to <code>true</code>. In the unlikely event that the above condition evaluates to <code>false</code>, FsCheck is going to ignore that randomly generated value, and try with a new one. </p> <p> This version of the test succeeds, even if you set MaxTest to 1,000,000. </p> <p> The <code>==&gt;</code> operator effectively discards candidate values that don't match the 'guard' condition. You shouldn't always use this feature to constrain the input, as it can be wasteful. If the condition isn't satisfied, FsCheck will attempt to generate a new value that satisfies the condition. In this particular example, only in extremely rare cases will FsCheck be forced to discard a candidate value, so this use of the feature is appropriate. </p> <p> In other cases, you should consider other options. Imagine that you want only even numbers. While you could write a 'guard' condition that ensures that only even numbers are used, that would cause FsCheck to throw away half of the generated candidates. In such cases, you should instead consider defining a custom Arbitrary, using FsCheck's API. </p> <p> Another discussion entirely is whether the current behaviour is a good idea. If we consider tests as a feedback mechanism about software design, then perhaps we should explicitly specify the expected format. At the moment, the implementation is an extremely <a href="http://martinfowler.com/bliki/TolerantReader.html">Tolerant Reader</a> (because DateTimeOffset.TryParse is a Tolerant Reader). Perhaps it'd make sense to instead use <a href="https://msdn.microsoft.com/en-us/library/system.datetimeoffset.tryparseexact">TryParseExact</a>. That's an API modelling decision, though, and might have to involve various other stakeholders than only programmers. </p> <p> If you wish to learn more about Property-Based Testing with FsCheck, consider watching <a href="http://bit.ly/1M1zXTf">my Pluralsight course</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Tail Recurse http://blog.ploeh.dk/2015/12/22/tail-recurse 2015-12-22T09:11:00+00:00 Mark Seemann <div id="post"> <p> <em>Tips on refactoring recursive functions to tail-recursive functions.</em> </p> <p> In a recent article, I described <a href="http://blog.ploeh.dk/2015/12/01/recurse">how to refactor an imperative loop to a recursive function</a>. If you're coming from C# or Java, however, you may have learned to avoid recursion, since it leads to stack overflows when the recursion is too deep. </p> <p> In some Functional languages, like F# and Haskell, such stack overflows can be prevented with <a href="https://en.wikipedia.org/wiki/Tail_call">tail recursion</a>. If the <em>last</em> function call being made is a recursive call to the function itself, the current stack frame can be eliminated because execution is effectively completed. This allows a tail-recursive function to keep recursing without ever overflowing. </p> <p> When you have a recursive function that's <em>not</em> tail recursive, however, it can sometimes be difficult to figure out how to refactor it so that it becomes tail recursive. In this article, I'll outline some techniques I've found to be effective. </p> <p> <strong>Introduce an accumulator</strong> </p> <p> It seems as though the universal trick related to recursion is to introduce an accumulator argument. This is <a href="http://blog.ploeh.dk/2015/12/01/recurse">how you refactor an imperative loop to a recursive function</a>, but it can also be used to make a non-tail-recursive function tail recursive. This will also require you to introduce an 'implementation function' that does the actual work. </p> <p> <strong>Example: discarding candidate points when finding a convex hull</strong> </p> <p> In <a href="http://blog.ploeh.dk/2015/10/19/visual-value-verification">previous articles</a> I've described various problems I had when implementing the <a href="https://en.wikipedia.org/wiki/Graham_scan">Graham Scan</a> algorithm to find the <a href="https://en.wikipedia.org/wiki/Convex_hull">convex hull</a> of a set of points. </p> <p> One of my functions examined the three last points in a sequence of points in order to determine if the next-to-last point is in the interior of the set, or if that point could potentially be on the hull. If the point is positively known to be in the interior of the set, it should be discarded. At one stage, the function was implemented this way: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryDiscard</span>&nbsp;points&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[p1;&nbsp;p2;&nbsp;p3]&nbsp;<span style="color:blue;">when</span>&nbsp;<span style="color:navy;">turn</span>&nbsp;p1&nbsp;p2&nbsp;p3&nbsp;=&nbsp;<span style="color:#4ec9b0;">Direction</span>.Right&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[p1;&nbsp;p3] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[p1;&nbsp;p2;&nbsp;p3]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[p1;&nbsp;p2;&nbsp;p3] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;p&nbsp;<span style="color:navy;">::</span>&nbsp;ps&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;p&nbsp;<span style="color:navy;">::</span>&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;ps &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;newPoints&nbsp;=&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;points &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;newPoints.Length&nbsp;&lt;&gt;&nbsp;points.Length &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Some</span>&nbsp;newPoints &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> This function was earlier called <code>check</code>, which is the name used in <a href="http://blog.ploeh.dk/2015/12/01/recurse">the article about refactoring to recursion</a>. The tryDiscard function is actually an inner function in a more complex function that's defined with the <code>inline</code> keyword, so the type of tryDiscard is somewhat complicated, but think of it as having the type <code>(int * int) list -&gt; (int * int) list option</code>. If a point was discarded, the new, reduced list of points is returned in a Some case; otherwise, None is returned. </p> <p> The tryDiscard function already has an 'implementation function' called tryDiscardImp, but while tryDiscardImp is recursive, it isn't <em>tail</em> recursive. The problem is that in the <code>p :: ps</code> case, the recursive call to tryDiscardImp isn't the tail call. Rather, the stack frame has to wait for the recursive call <code>tryDiscardImp ps</code> to complete, because only then can it cons <code>p</code> onto its return value. </p> <p> Since an 'implementation function' already exists, you can make it tail recursive by adding an accumulator argument to tryDiscardImp: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">tryDiscard</span>&nbsp;points&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[p1;&nbsp;p2;&nbsp;p3]&nbsp;<span style="color:blue;">when</span>&nbsp;<span style="color:navy;">turn</span>&nbsp;p1&nbsp;p2&nbsp;p3&nbsp;=&nbsp;<span style="color:#4ec9b0;">Direction</span>.Right&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;acc&nbsp;@&nbsp;[p1;&nbsp;p3] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[p1;&nbsp;p2;&nbsp;p3]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc&nbsp;@&nbsp;[p1;&nbsp;p2;&nbsp;p3] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;p&nbsp;<span style="color:navy;">::</span>&nbsp;ps&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;(acc&nbsp;@&nbsp;[p])&nbsp;ps &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;newPoints&nbsp;=&nbsp;<span style="color:navy;">tryDiscardImp</span>&nbsp;[]&nbsp;points &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;newPoints.Length&nbsp;&lt;&gt;&nbsp;points.Length &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:navy;">Some</span>&nbsp;newPoints &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:navy;">None</span></pre> </p> <p> As you can see, I added the <code>acc</code> argument to tryDiscardImp; it has the type <code>(int * int) list</code> (again: not really, but close enough). Instead of returning from each case, the tryDiscardImp function now appends points to <code>acc</code> until it reaches the end of the list, which is when it returns the accumulator. The <code>p :: ps</code> case now <em>first</em> appends the point in consideration to the accumulator (<code>acc @ [p]</code>), and only then recursively calls tryDiscardImp. This puts the call to tryDiscardImp in the tail position. </p> <p> The repeated use of the append operator (<code>@</code>) is terribly inefficient, though, but I'll return to this issue later in this article. For now, let's take a step back. </p> <p> <strong>Example: implementing map with recursion</strong> </p> <p> A common exercise for people new to Functional Programming is to implement a map function (C# developers will know it as <a href="https://msdn.microsoft.com/library/bb548891">Select</a>) using recursion. This function <a href="https://msdn.microsoft.com/en-us/library/ee370378.aspx">already exists in the List module</a>, but it can be enlightening to do the exercise. </p> <p> An easy, but naive implementation is only two lines of code, using pattern matching: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">mapNaive</span>&nbsp;<span style="color:navy;">f</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[] &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;h<span style="color:navy;">::</span>t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">f</span>&nbsp;h&nbsp;<span style="color:navy;">::</span>&nbsp;<span style="color:navy;">mapNaive</span>&nbsp;<span style="color:navy;">f</span>&nbsp;t</pre> </p> <p> Is mapNaive tail recursive? No, it isn't. The last operation happening is that <code>f h</code> is consed unto the return value of <code>mapNaive f t</code>. While <code>mapNaive f t</code> is a recursive call, it's not in the tail position. For long lists, this will create a stack overflow. </p> <p> How can you create a tail-recursive map implementation? </p> <p> <strong>Example: inefficient tail-recursive map implementation</strong> </p> <p> According to my introduction, adding an accumulator and an 'implementation' function should do the trick. Here's the straightforward application of that technique: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapTailRecursiveUsingAppend</span>&nbsp;<span style="color:navy;">f</span>&nbsp;xs&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;h<span style="color:navy;">::</span>t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;(acc&nbsp;@&nbsp;[<span style="color:navy;">f</span>&nbsp;h])&nbsp;t &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;[]&nbsp;xs</pre> </p> <p> The mapImp function does the actual work, and it's tail recursive. It appends the result of mapping the head of the list unto the accumulator: <code>acc @ [f h]</code>. Only then does it recursively call itself with the new accumulator and the tail of the list. </p> <p> While this version is tail recursive, it's horribly inefficient, because appending to the tail of a (linked) list is inefficient. In theory, this implementation would never result in a stack overflow, but the question is whether anyone has the patience to wait for that to happen? </p> <p> <pre>&gt; mapTailRecursiveUsingAppend ((*) 2) [1..100000];; Real: 00:02:46.104, CPU: 00:02:44.750, GC gen0: 13068, gen1: 6380, gen2: 1 val it : int list = [2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30; 32; 34; 36; ...]</pre> </p> <p> Doubling 100,000 integers this way takes nearly 3 minutes on my (admittedly mediocre) laptop. A better approach is required. </p> <p> <strong>Example: efficient tail-recursive map implementation</strong> </p> <p> The problem with mapTailRecursiveUsingAppend is that appending to a list is slow when the left-hand list is long. This is because lists are linked lists, so the append operation has to traverse the entire list and copy all the element to link to the right-hand list. </p> <p> Consing a single item unto an existing list, on the other hand, is efficient: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapTailRecursiveUsingRev</span>&nbsp;<span style="color:navy;">f</span>&nbsp;xs&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;h<span style="color:navy;">::</span>t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;(<span style="color:navy;">f</span>&nbsp;h&nbsp;<span style="color:navy;">::</span>&nbsp;acc)&nbsp;t &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;[]&nbsp;xs&nbsp;|&gt;&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">rev</span></pre> </p> <p> This function conses unto the accumulator (<code>f h :: acc</code>) instead of appending to the accumulator. The only problem with this is that <code>acc</code> is in reverse order compared to the input, so the final step must be to reverse the output of mapImp. While there's a cost to reversing a list, you pay it only once. In practice, it turns out to be efficient: </p> <p> <pre>&gt; mapTailRecursiveUsingRev ((*) 2) [1..100000];; Real: 00:00:00.017, CPU: 00:00:00.015, GC gen0: 1, gen1: 0, gen2: 0 val it : int list = [2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30; 32; 34; 36; ...]</pre> </p> <p> From nearly three minutes to 17 milliseconds! That's a nice performance improvement. </p> <p> The only problem, from a point of view where <em>learning</em> is in focus, is that it feels a bit like cheating: we've delegated an important step in the algorithm to <a href="https://msdn.microsoft.com/en-us/library/ee340277.aspx">List.rev</a>. If we think it's OK to use the library functions, we could have directly used <a href="https://msdn.microsoft.com/en-us/library/ee370378.aspx">List.map</a>. The whole point of this exercise, however, is to learn something about how to write tail-recursive functions. </p> <p> At this point, we have two options: <ul> <li>Learn how to write an efficient, tail-recursive implementation of <code>rev</code>.</li> <li>Consider alternatives.</li> </ul> Ultimately, it turns out that List.rev is <a href="https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/local.fs">implemented in a fashion similar to above, using an 'implementation function' and an accumulator</a>. Let us, therefore, turn our attention to another alternative. </p> <p> <strong>Example: efficient tail-recursive map using a difference list</strong> </p> <p> The mapTailRecursiveUsingAppend function is attractive because of its simplicity. If only there was an efficient way to append a single item to the tail of a (long) list, like <code>acc appendSingle (f h)</code>! (appendSingle is a hypothetical function that we wish existed.) </p> <p> So far, we've treated data as data, and functions as functions, but in Functional Programming, functions are data as well! </p> <p> What if we could <em>partially</em> apply a cons operation? </p> <p> Unfortunately, the cons operator (<code>::</code>) <a href="http://stackoverflow.com/q/3821472/126014">can't be used as a function</a>, so you'll have to introduce a little helper function: </p> <p> <pre><span style="color:green;">//&nbsp;&#39;a&nbsp;-&gt;&nbsp;&#39;a&nbsp;list&nbsp;-&gt;&nbsp;&#39;a&nbsp;list</span> <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">cons</span>&nbsp;x&nbsp;xs&nbsp;=&nbsp;x&nbsp;<span style="color:navy;">::</span>&nbsp;xs</pre> </p> <p> This enables you to partially apply a cons operation: </p> <p> <pre>&gt; cons 1;; val it : (int list -&gt; int list) = &lt;fun:it@4-5&gt;</pre> </p> <p> The expression <code>cons 1</code> is a <em>function</em> that awaits an <code>int list</code> argument. You can, for example, call it with the empty list, or another list: </p> <p> <pre>&gt; (cons 1) [];; val it : int list = [1] &gt; (cons 1) [2];; val it : int list = [1; 2]</pre> </p> <p> That hardly seems useful, but what happens if you start <em>composing</em> such partially applied functions? </p> <p> <pre>&gt; cons 1 &gt;&gt; cons 2;; val it : (int list -&gt; int list) = &lt;fun:it@7-8&gt;</pre> </p> <p> Notice that the result is another function with the same signature as <code>cons 1</code>. A way to read it is: <code>cons 1</code> is a function that takes a list as input, appends the list after <code>1</code>, and returns that new list. The return value of <code>cons 1</code> is passed to <code>cons 2</code>, which takes that input, appends that list after <code>2</code>, and returns that list. Got it? Try it out: </p> <p> <pre>&gt; (cons 1 &gt;&gt; cons 2) [];; val it : int list = [2; 1]</pre> </p> <p> Not what you expected? Try going through the data flow again. The input is the empty list (<code>[]</code>), which, when applied to <code>cons 1</code> produces <code>[1]</code>. That value is then passed to <code>cons 2</code>, which puts <code>2</code> at the head of <code>[1]</code>, yielding the final result of <code>[2; 1]</code>. </p> <p> This still doesn't seem to help, because it still reverses the list. True, but you can <em>reverse the composition:</em> </p> <p> <pre>&gt; (cons 2 &gt;&gt; cons 1) [];; val it : int list = [1; 2] &gt; (cons 1 &lt;&lt; cons 2) [];; val it : int list = [1; 2]</pre> </p> <p> Notice that in the first line, I reversed the composition by changing the order of partially applied functions. This, however, is equivalent to keeping the order, but using the reverse composition operator (<code>&lt;&lt;</code>). </p> <p> You can repeat this composition: </p> <p> <pre>&gt; (cons 1 &lt;&lt; cons 2 &lt;&lt; cons 3 &lt;&lt; cons 4) [];; val it : int list = [1; 2; 3; 4]</pre> </p> <p> That's exactly what you need, enabling you to write <code>acc &lt;&lt; (cons (f h))</code> in order to efficiently append a single element to the tail of a list! </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapTailRecursiveUsingDifferenceList</span>&nbsp;<span style="color:navy;">f</span>&nbsp;xs&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">cons</span>&nbsp;x&nbsp;xs&nbsp;=&nbsp;x&nbsp;<span style="color:navy;">::</span>&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;<span style="color:navy;">acc</span>&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">acc</span>&nbsp;[] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;h<span style="color:navy;">::</span>t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;(<span style="color:navy;">acc</span>&nbsp;&lt;&lt;&nbsp;(<span style="color:navy;">cons</span>&nbsp;(<span style="color:navy;">f</span>&nbsp;h)))&nbsp;t &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">mapImp</span>&nbsp;<span style="color:navy;">f</span>&nbsp;<span style="color:navy;">id</span>&nbsp;xs</pre> </p> <p> This implementation of <em>map</em> uses this technique, introduced by <a href="https://en.wikipedia.org/wiki/John_Hughes_(computer_scientist)">John Hughes</a> in 1985. <a href="http://bit.ly/real-world-haskell">Real World Haskell</a> calls the technique a <em>difference list</em>, without explaining why. </p> <p> This mapImp function's accumulator is no longer a list, but a <em>function</em>. For every item, it composes a new accumulator function from the old one, effectively appending the mapped item to the tail of the accumulated list. Yet, because <code>acc</code> isn't a list, but rather a function, the 'append' operation doesn't trigger a list traversal. </p> <p> When the recursive function finally reaches the end of the list (the <code>[]</code> case), it invokes the <code>acc</code> function with the empty list (<code>[]</code>) as the initial input. </p> <p> This implementation is also tail recursive, because the accumulator is being completely composed (<code>acc &lt;&lt; (cons (f h))</code>) <em>before</em> mapImp is recursively invoked. </p> <p> Is it efficient, then? </p> <p> <pre>&gt; mapTailRecursiveUsingDifferenceList ((*) 2) [1..100000];; Real: 00:00:00.024, CPU: 00:00:00.031, GC gen0: 1, gen1: 0, gen2: 0 val it : int list = [2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30; 32; 34; 36; ...]</pre> </p> <p> 24 milliseconds is decent. It's not as good as mapTailRecursiveUsingRev (17 milliseconds), but it's close. </p> <p> In practice, you'll probably find that mapTailRecursiveUsingRev is not only more efficient, but also easier to understand. The advantage of using the difference list technique, however, is that now mapImp has a shape that almost screams to be refactored to a fold. </p> <p> <strong>Example: implementing map with fold</strong> </p> <p> The mapImp function in mapTailRecursiveUsingDifferenceList almost has the shape required by the accumulator function in <a href="https://msdn.microsoft.com/en-us/library/ee353894.aspx">List.fold</a>. This enables you to rewrite mapImp using fold: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapUsingFold</span>&nbsp;<span style="color:navy;">f</span>&nbsp;xs&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">cons</span>&nbsp;x&nbsp;xs&nbsp;=&nbsp;x&nbsp;<span style="color:navy;">::</span>&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:navy;">mapImp</span>&nbsp;=&nbsp;<span style="color:#4ec9b0;">List</span>.<span style="color:navy;">fold</span>&nbsp;(<span style="color:blue;">fun</span>&nbsp;<span style="color:navy;">acc</span>&nbsp;h&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:navy;">acc</span>&nbsp;&lt;&lt;&nbsp;(<span style="color:navy;">cons</span>&nbsp;(<span style="color:navy;">f</span>&nbsp;h)))&nbsp;<span style="color:navy;">id</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">mapImp</span>&nbsp;xs&nbsp;[]</pre> </p> <p> As usual in Functional Programming, the ultimate goal seems to be to avoid writing recursive functions after all! </p> <p> The mapUsingFold function is as efficient as mapTailRecursiveUsingDifferenceList: </p> <p> <pre>&gt; mapUsingFold ((*) 2) [1..100000];; Real: 00:00:00.025, CPU: 00:00:00.031, GC gen0: 2, gen1: 1, gen2: 1 val it : int list = [2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30; 32; 34; 36; ...]</pre> </p> <p> Not only does 25 milliseconds seem fast, but it's also comparable with the performance of the built-in map function: </p> <p> <pre>&gt; List.map ((*) 2) [1..100000];; Real: 00:00:00.011, CPU: 00:00:00.015, GC gen0: 0, gen1: 0, gen2: 0 val it : int list = [2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30; 32; 34; 36; ...]</pre> </p> <p> Granted, List.map seems to be twice as fast, but it's also been the subject of more scrutiny than the above fun exercise. </p> <p> <strong>Summary</strong> </p> <p> In Functional Programming, recursive functions take the place of imperative loops. In order to be efficient, they must be tail recursive. </p> <p> You can make a function tail recursive by introducing an accumulator argument to an 'implementation function'. This also tends to put you in a position where you can ultimately refactor the implementation to use a fold instead of explicit recursion. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Integration Testing composed functions http://blog.ploeh.dk/2015/12/21/integration-testing-composed-functions 2015-12-21T09:05:00+00:00 Mark Seemann <div id="post"> <p> <em>When you build a system from small functions that you subsequently compose, how do you know that the composition works? Integration testing is one option.</em> </p> <p> Despite its reputation as a niche language for scientific or finance computing, F# is a wonderful language for 'mainstream' software development. You can compose applications from small, <a href="http://blog.ploeh.dk/2015/05/07/functional-design-is-intrinsically-testable">inherently testable</a> functions. </p> <p> <strong>Composition</strong> </p> <p> Once you have your functions as building blocks, you compose them. This is best done in an application's <a href="http://blog.ploeh.dk/2011/07/28/CompositionRoot">Composition Root</a> - no different from Dependency Injection in Object-Oriented Programming. </p> <p> In my <a href="http://www.infoq.com/presentations/mock-fsharp-tdd">(free) Build Stuff talk on Functional TDD with F#</a> (expanded version <a href="http://bit.ly/tddwithfsharp">available on Pluralsight</a>), I demonstrate how to compose such a function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">imp</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">Validate</span>.<span style="color:navy;">reservation</span> &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">bind</span>&nbsp;(<span style="color:#4ec9b0;">Capacity</span>.<span style="color:navy;">check</span>&nbsp;10&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">getReservedSeats</span>&nbsp;connectionString)) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:navy;">map</span>&nbsp;(<span style="color:#4ec9b0;">SqlGateway</span>.<span style="color:navy;">saveReservation</span>&nbsp;connectionString)</pre> </p> <p> How can you be sure that this composition is correct? </p> <p> <strong>Integration Testing</strong> </p> <p> The answer to that question isn't different from its Object-Oriented counterpart. When you implement small building blocks, you can test them. Call these small building blocks <em>units</em>, and it should be clear that such tests are unit tests. It doesn't matter if units are (small) classes or functions. </p> <p> Unit testing ought to give you confidence that each unit behaves correctly, but unit tests don't tell you how they integrate. Are there issue that only surface when units interact? Are units correctly composed? </p> <p> In my experience, you can develop entire systems based exclusively on unit tests, and the final application can be stable and sturdy without the need for further testing. This depends on circumstances, though. In other cases, you need further testing to gain confidence that the application is correctly composed from its building blocks. </p> <p> You can use a small set of integration tests for that. </p> <p> <strong>Example</strong> </p> <p> In my <a href="http://bit.ly/outside-in-tdd">Outside-In Test-Driven Development</a> Pluralsight course, I demonstrate how to apply the <a href="http://bit.ly/growingoos">GOOS</a> approach to an HTTP API built with <a href="http://www.asp.net/web-api">ASP.NET Web API</a>. One of the techniques I describe is <a href="http://stackoverflow.com/a/22294059/126014">how to integration test the API against its HTTP boundary</a>. </p> <p> In that course, you learn how to test an API implemented in C#, but since the tests are made against the HTTP boundary, the implementation language doesn't matter. Even so, you can also write the tests themselves in F#. Here's an example that exercises the Controller that uses the above <code>imp</code> function: </p> <p> <pre>[&lt;<span style="color:#4ec9b0;">Fact</span>;&nbsp;<span style="color:#4ec9b0;">UseDatabase</span>&gt;] <span style="color:blue;">let</span>&nbsp;<span style="color:navy;">``Post&nbsp;returns&nbsp;success``</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;client&nbsp;=&nbsp;<span style="color:navy;">createClient</span>() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;json&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#4ec9b0;">ReservationJson</span>.<span style="color:#4ec9b0;">Root</span>(<span style="color:#a31515;">&quot;2014-10-21&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Mark&nbsp;Seemann&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;4) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;response&nbsp;=&nbsp;client.<span style="color:navy;">PostAsJsonAsync</span>(<span style="color:#a31515;">&quot;reservations&quot;</span>,&nbsp;json).Result &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">test</span>&nbsp;<span style="background:yellow;">&lt;@&nbsp;</span><span style="background:yellow;">response</span><span style="background:yellow;">.</span><span style="background:yellow;">IsSuccessStatusCode</span><span style="background:yellow;">&nbsp;@&gt;</span></pre> </p> <p> This test creates a new <a href="https://msdn.microsoft.com/en-us/library/system.net.http.httpclient">HttpClient</a> object called <code>client</code>. It then creates a JSON document with some reservation data, and POSTs it to the <em>reservations</em> resource. Finally, it verifies that the response indicated a success. </p> <p> The ReservationJson type was created from a sample JSON document using the <a href="http://fsharp.github.io/FSharp.Data/library/JsonProvider.html">JSON Type Provider</a>. The createClient function is a bit more involved, but follows the same recipe I describe in my course: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:navy;">createClient</span>&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;baseAddress&nbsp;=&nbsp;<span style="color:#4ec9b0;">Uri</span>&nbsp;<span style="color:#a31515;">&quot;http://localhost:8765&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;config&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#4ec9b0;">HttpSelfHostConfiguration</span>(baseAddress) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:navy;">configure</span>&nbsp;connStr&nbsp;config &nbsp;&nbsp;&nbsp;&nbsp;config.IncludeErrorDetailPolicy&nbsp;&lt;-&nbsp;<span style="color:#4ec9b0;">IncludeErrorDetailPolicy</span>.Always &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;server&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#4ec9b0;">HttpSelfHostServer</span>(config) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;client&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#4ec9b0;">HttpClient</span>(server) &nbsp;&nbsp;&nbsp;&nbsp;client.BaseAddress&nbsp;&lt;-&nbsp;baseAddress &nbsp;&nbsp;&nbsp;&nbsp;client</pre> </p> <p> The (impure) <code>configure</code> function is a function defined by the application implementation. Among many other things, it creates the above <code>imp</code> composition. When the test passes, you can trust that <code>imp</code> is correctly composed. </p> <p> <strong>Smoke Testing</strong> </p> <p> You may already have noticed that the ``Post returns success`` test is course-grained and vague. It doesn't attempt to make strong assertions about the posterior state of the system; if the response indicates success, the test passes. </p> <p> The reason for this is that all important behaviour is already covered by unit tests. <ul> <li>Is the response specifically a 201 (Created) response? Covered by unit tests.</li> <li>Does the response have a Location header indicating the address of a newly created resource? Covered by unit test.</li> <li>What happens if the input is malformed? Covered by unit tests.</li> <li>What happens if the system can't accept the request due to business rules? Covered by unit tests.</li> <li>...and so on.</li> </ul> The purpose of the integration test isn't to instil confidence in the implementation; the purpose is to give you confidence that the application is correctly integrated (hence the name). In my experience, a few <a href="https://en.wikipedia.org/wiki/Smoke_testing_(software)">Smoke Tests</a> are enough to achieve that goal. </p> <p> Specifically, it has been my experience that the most common integration issues are related to various configuration errors: <ul> <li>Missing configuration values</li> <li>Wrong configuration values</li> <li>Network errors</li> <li>Security errors</li> <li>... etc.</li> </ul> Common to most of such errors is that they tend to manifest themselves uniformly. For example, in a database-driven application, if the database server is unreachable, <em>nothing</em> works. Therefore, only a single Smoke Test is required in order to tell you whether the application can reach the server or not. </p> <p> In other words: you should have a legion of unit tests covering specific <em>behaviour</em>, and a few integration tests covering common integration issues. You may already have recognised this principle as <a href="http://martinfowler.com/bliki/TestPyramid.html">the Test Pyramid</a>. </p> <p> <strong>Summary</strong> </p> <p> In this article, you saw an example of an integration test against an HTTP API, written in F#. The principle is universal, though. You can compose applications from units. These units can be functions written in F#, or Haskell, or Scala, or they can be classes written in C#, Java, Python, and so on. Composition can be done on functions, or on objects using Dependency Injection. </p> <p> In all of these cases, you can cover the units with unit tests. Issues with composition can be screened by a few smoke tests at the integration level. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. The Rules of Attraction: Location http://blog.ploeh.dk/2015/12/04/the-rules-of-attraction-location 2015-12-04T08:57:00+00:00 Mark Seemann <div id="post"> <p> <em>Widen your hiring pool to the entire world; you won't believe what happens next.</em> </p> <p> Every week in my Twitter stream, there's some enticing offer: <blockquote> NoWare is looking for strong F# or Haskell developers with experience in TDD, property-based testing, object-oriented design, REST, and software architecture. </blockquote> For half a second I go: "Oh, man, that's <em>so much</em> a description of me! This sounds terrific!" But then, with small letters, it says: "Must be willing to relocate to <a href="https://en.wikipedia.org/wiki/Sauda">Sauda</a>." </p> <p> Oh. </p> <p> If your company is based in London, New York, Silicon Valley, Seattle, and perhaps a few other places, I can even understand that attitude. The local 'talent pool' is already big in those places, and more people are willing to move there for work. </p> <p> In most other areas, I find that attitude towards hiring programmers lazy and self-centred. </p> <p> As an example, consider Copenhagen, where I live. The <a href="https://en.wikipedia.org/wiki/%C3%98resund_Region">greater metropolitan area is home to some 3.5 million people</a>, so you'd think it was possible to find programmers to hire. Yet, every employer here whines that it's too hard to find developers. </p> <p> While Copenhagen is nice enough, I can understand if the majority of the world's programmers aren't itching to move here. Apparently, this place is only <em>attractive</em> if you live in Afghanistan, Somalia, Nigeria, Syria, or some other horrible place. </p> <p> <strong>All digital</strong> </p> <p> The outcome of programming is software, a 100% immaterial, digital product. Source code, as well, is entirely digital. Documentation, if it exists, tends to be digital. Source control systems are networked. Developers shouldn't have physical access to production servers. </p> <p> Why do programmers need to be physically present? </p> <p> <strong>Tacit knowledge</strong> </p> <p> I think I know why programmers need to be physically present in many organisations. It's because all the knowledge of 'how things are done' is in the heads of people. <ul> <li><em>How do I configure my development machine?</em> Ask Joe; he can tell you.</li> <li><em>What's the password to the CI server?</em> Call Kathy in the satellite office. Maybe she knows.</li> <li><em>How do I find the logs related to this bug I was asked to fix?</em> Go and talk to Jane. She can show you.</li> <li><em>Why did we build our own ORM, instead of using Entity Framework?</em> Uhm, yeah, there was this lead developer, Rick... He's no longer with the company...</li> <li><em>Would anyone mind if I add an extra argument to the BloobedlyBloo method?</em> Sure, go right ahead!</li> <li><em>How are we going to address this new feature request?</em> I don't know. Let's have a meeting!</li> <li><em>What's the status of the Foo component?</em> I don't know. Let's have a meeting!</li> <li><em>Is there any way to prevent the back-office developers from accessing our database directly?</em> Probably not, but let's have a meeting!</li> </ul> I've seen plenty of organisations that work that way. Most of the communication that happens tends to be ephemeral and unorganised. If you're a remote worker, you don't have a chance. </p> <p> Perhaps organisations that work that way are effective, but it doesn't scale. Every time a new team member or stakeholder joins the project, someone needs to take time bringing him or her up to speed. The more people in the organisation, the worse it becomes. </p> <p> At one time I worked in an organisation that operated like that. I'd have meeting upon meeting about particular topics. The next week, I'd have <em>the same meetings again</em>, because a new stakeholder wanted to get involved. Nothing new happened; nothing new was decided, but it was necessary, because nothing was written down. Lots of time was wasted. </p> <p> That's when I started writing documentation and meeting minutes. Not that I enjoyed doing that, but it saved me time. The next time someone came and asked me to sit in on a meeting, I'd forward the minutes and documentation and ask people to read that instead. </p> <p> <strong>Conway's law</strong> </p> <p> Imagine that you could hire programmers from all over the world. Not only within driving distance of your offices, but from the entire world. Just think of the supply of developers you'd be able to choose from. You can literally select from among the world's best programmers, if you're willing to pay the price. Some companies, like <a href="http://stackoverflow.com">Stack Overflow</a>, <a href="https://github.com">GitHub</a>, <a href="https://basecamp.com">Basecamp</a>, and <a href="http://basho.com">Basho</a>, already do this, with great success. </p> <p> It's only going to work, though, if your organisation is set up for it. Processes must be written down. Decisions must be documented. Work must be asynchronous, in order to accommodate people in different time zones. You can't easily have meetings, so you'll need to discuss strategy, architecture, design, etc. using collaborative online tools. </p> <p> If your organisation doesn't already work like that, you'll have to transform it. Software development with distributed teams can work, but only if you enable it. The good news is that there's plenty of publicly available examples on how to do this: just find your favourite open source project, and study how it's organised. Most open source is written by programmers from all over the world, at various times, and yet it still works. </p> <p> As a modest example, my own experience with <a href="https://github.com/AutoFixture/AutoFixture">AutoFixture</a> has taught me a thing or two about running a distributed project. Contributors have submitted code from Australia, South Korea, USA, Greece, England, Denmark, Sweden, Russia, and lots of other countries. Discussions happen in the open, in <a href="https://github.com/AutoFixture/AutoFixture/issues">GitHub issues</a>. </p> <p> You'll most likely find that if you can get distributed development to work, your software architecture, too, will become more and more distributed and asynchronous in nature. This is <a href="https://en.wikipedia.org/wiki/Conway%27s_law">Conway's law</a>: <em>organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations</em>. </p> <p> If your organisation is based on ad hoc ephemeral communication and impromptu meetings, your software will have no clear communication structures either. There's <a href="https://en.wikipedia.org/wiki/Spaghetti_code">a word</a> for that kind of software architecture. </p> <p> If, on the other hand, you set yourself up for distributed development teams, you'll be forced to make the communication channels clearer - both in your organisation, and in your software. This will enable you to scale your development organisation. </p> <p> <strong>Summary</strong> </p> <p> You can <em>claim</em> that you're a 'world-class' organisation that hires only the best of the best (within a radius of 100 km). </p> <p> Or you can <em>be</em> a world-class organisation by literally having developers working all over the world. You still need to be able to find the best, though. Perhaps you should try to <a href="http://blog.ploeh.dk/2015/12/03/the-rules-of-attraction-language">let them find you instead</a>. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. The Rules of Attraction: Language http://blog.ploeh.dk/2015/12/03/the-rules-of-attraction-language 2015-12-03T15:05:00+00:00 Mark Seemann <div id="post"> <p> <em>How to attract the best developers to your organisation, with one weird trick.</em> </p> <p> In a <a href="http://broadcast.oreilly.com/2009/03/an-interview-with-anders-hejls.html">2009 interview</a>, <a href="https://en.wikipedia.org/wiki/Anders_Hejlsberg">Anders Hejlsberg</a>, the inventor of C#, Turbo Pascal, TypeScript, etc. said: <blockquote> "Well, you know, platforms live maybe 10, 15 years and then they cave in under their own weight, one way or the other." </blockquote> C# is now 15 years old; Java: 20 years. </p> <p> You don't have to believe that Anders Hejlsberg is right, though. After all, COBOL is still in use, 56 years after its invention. One of the world's most popular languages, C++, is 32 years old, and C is 43 years old. Still, it's food for thought. </p> <p> When I consult and advise, I often encounter organisations that standardise on C# or Java. When I try to tell CTOs and development leads about the benefits of adopting 'new' languages like F# (10 years), Haskell (25 years), Clojure (8 years), Erlang (29 years), or Scala (11 years), the response is always the same: <blockquote> "How will I find developers?" </blockquote> That's the easiest thing in the world! </p> <p> In the early 2000s, Java was already getting close to 10 years old, and some programmers were beginning to look for the next cool technology. They found it in Python, and for a while, Python was perceived as the cutting edge. </p> <p> In the late 2000s, C# 'alpha developers' migrated to Ruby en masse. It became so bad that <em>I'm-leaving-.NET</em> blog posts became a cliché. </p> <p> In the early 2010s, the main attraction has been Node.js, and JavaScript in general. </p> <p> Let's not forget those of us who have fallen in love with F#, Clojure, Haskell, Elixir, etc. </p> <p> The most curious developers eventually get tired of using the same programming language year in and year out. Those first-movers that migrated to Python 10 years ago are already on to the next language. The same goes for the Rubyists. </p> <p> Finding F#, Clojure, Elixir, etc. developers is the easiest thing in the world. The most important thing you can do as an organisation is to say: <blockquote> "We wish to hire F# developers!" </blockquote>, or Clojure developers, Haskell developers, etc. </p> <p> You don't have to <em>find</em> such developers; make them find you. </p> <p> Although there are few of these developers out there, they are easy to attract. This is called <a href="http://paulgraham.com/pypar.html">the Python Paradox</a>, after the early 2000 Python migration. </p> <p> Not only is it easy to attract developers for such 'new' languages, you also get the most progressive, curious, motivated, enthusiastic programmers. That's the 'talent' all companies seem to be pining for these days. </p> <p> Some programmers will even accept a decrease in income, only for the chance to get to work with a technology they love. </p> <p> You'll probably also get some difficult-to-work-with primadonnas who are gone again after three years... <a href="https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as_a_free_lunch">TANSTAAFL</a>. </p> <p> The crux of the matter is that the argument that you can't find developers for a particular cool language doesn't hold. </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. Recurse http://blog.ploeh.dk/2015/12/01/recurse 2015-12-01T09:12:00+00:00 Mark Seemann <div id="post"> <p> <em>How to refactor from a loop using mutable state to a recursive implementation.</em> </p> <p> One of the most compelling reasons to adopt Functional Programming (FP) is the emphasis on immutable values. All the dread and angst associated with state that can implicitly change while you're not looking, is gone. </p> <p> One of the most frustrating aspects of FP for people coming from other paradigms is the emphasis on immutable values. You know that you ought to be able to implement a given algorithm using immutable values, but given your background in Object-Oriented Programming, you can't quite figure out how to do it. </p> <p> In FP, loops are implemented with recursion, and mutable values are replaced with accumulator arguments. This article describes how to refactor from a procedural, mutable loop implementation to a pure, tail-recursive implementation. </p> <p> <strong>Motivation</strong> </p> <p> You want your loop implementation to be 'Functional' instead of procedural. There can be many reasons for this. Perhaps you want to learn FP. Perhaps you want to eliminate mutable state in order to make your implementation thread-safe. Perhaps you think that getting rid of mutation will make the code more readable and maintainable. Perhaps you want to port your implementation to a language that doesn't support mutability at all (like Haskell). </p> <p> <strong>Mechanics</strong> </p> <p> Start with a procedural implementation, using a mutable loop variable. This obviously only works in multi-paradigmatic languages where mutable variables are possible, even if they're not ideal. Examples include F#, Scala, and Clojure, but not Haskell, which isn't multi-paradigmatic. <ol> <li>Instead of your imperative loop, introduce a recursive function.</li> <li>Replace each mutable loop variable with an argument for the recursive function.</li> </ol> Ultimately, your refactored implementation should be tail-recursive, but you can always address that concern once you've refactored to recursion. </p> <p> <strong>Example: backspace characters</strong> </p> <p> Imagine that you receive a stream of characters from someone typing on a keyboard. Sometimes, the typist mistypes and uses the backspace key, which sends the character '\b'. Whenever you encounter '\b', you should remove the preceding character, as well as the backspace character itself. This example is based on <a href="http://stackoverflow.com/q/32801109/126014">this Stack Overflow question</a>. </p> <p> The original F# implementation is procedural, using a for loop and a single mutable variable: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;System <span style="color:blue;">open</span>&nbsp;System.Collections.Generic <span style="color:blue;">let</span>&nbsp;handleBackspaces&nbsp;textToProcess&nbsp;:&nbsp;<span style="color:#2b91af;">string</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;stack&nbsp;=&nbsp;<span style="color:#2b91af;">Stack</span>&lt;<span style="color:#2b91af;">char</span>&gt;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;c&nbsp;<span style="color:blue;">in</span>&nbsp;textToProcess&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;c&nbsp;=&nbsp;<span style="color:#a31515;">&#39;\b&#39;</span>&nbsp;<span style="color:blue;">then</span>&nbsp;stack.Pop()&nbsp;|&gt;&nbsp;ignore &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;stack.Push&nbsp;c &nbsp;&nbsp;&nbsp;&nbsp;stack&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">Seq</span>.rev&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">Seq</span>.toArray&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">String</span></pre> </p> <p> While this implementation doesn't explicitly use the <code>mutable</code> keyword, the <code>stack</code> variable is mutable because Stack&lt;T&gt; is mutable. Since <code>textToProcess</code> is a string, and string implements IEnumerable&lt;char&gt;, you can loop over each char value, pushing the value on the stack unless it's a backspace character; in that case, the previous value is instead popped and thrown away. </p> <p> According to the rules of the <em>Recurse</em> refactoring, you should introduce a recursive function instead of the loop, and add an argument that will replace the stack. To make it easy, call the recursive function <code>imp</code>, and the function argument <code>acc</code>. The name <code>acc</code> is popular; it's short for <em>accumulator</em>. This argument is used to accumulate the final value, just like <code>stack</code> in the above example. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;handleBackspaces&#39;&nbsp;textToProcess&nbsp;:&nbsp;<span style="color:#2b91af;">string</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;imp&nbsp;acc&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;<span style="color:#a31515;">&#39;\b&#39;</span>::cs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;imp&nbsp;(acc&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">List</span>.tail)&nbsp;cs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;c::cs&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;imp&nbsp;(c::acc)&nbsp;cs &nbsp;&nbsp;&nbsp;&nbsp;textToProcess&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">Seq</span>.toList&nbsp;|&gt;&nbsp;imp&nbsp;[]&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">List</span>.rev&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">List</span>.toArray&nbsp;|&gt;&nbsp;<span style="color:#2b91af;">String</span></pre> </p> <p> The <code>imp</code> function is declared with the <code>rec</code> keyword to mark it as a recursive function. It has the type <code>char list -&gt; char list -&gt; char list</code>. <code>acc</code> is a <code>char list</code>, which is also the case for the second argument implied by the <code>function</code> keyword. The function returns the accumulator if the input list is empty; otherwise, it matches on the head of the list. If the head is explicitly the backspace character, the <code>imp</code> function calls itself recursively, but replaces <code>acc</code> with the tail of <code>acc</code>; it uses List.tail to get the tail. This effectively removes the most recent character from the accumulator. In all other cases, the function also calls itself by consing <code>c</code> on <code>acc</code>. </p> <p> The <code>imp</code> function is tail-recursive, since all other values are already computed when a recursive call to <code>imp</code> takes place. </p> <p> You can <a href="http://stackoverflow.com/a/32801993/126014">further refactor this implementation to use a fold</a> instead of a recursive function. </p> <p> <strong>Example: Graham Scan</strong> </p> <p> This second example is a more complex example that further illustrates how to apply the <em>Recurse</em> refactoring. If you already feel that you understand how to apply this refactoring, you can skip reading the following example. </p> <p> Some time ago, I was attempting to implement the <a href="https://en.wikipedia.org/wiki/Graham_scan">Graham Scan</a> algorithm to find the <a href="https://en.wikipedia.org/wiki/Convex_hull">convex hull</a> for a set of points. As I've <a href="http://blog.ploeh.dk/2015/10/19/visual-value-verification">described before</a>, this turned out to be quite difficult for me. One of my problems was that I was trying to implement the algorithm in Haskell, and while I understood the algorithm, I couldn't figure out how to implement it in a functional way - and when you can't do it the functional way, you can't do it at all in Haskell. </p> <p> In F#, on the other hand, you can implement algorithms using procedural code if you must, so I decided to implement the algorithm in F# first, using a procedural approach, and then subsequently figure out how to refactor to immutable values. Once I had a pure F# implementation, I could always back-port it to Haskell. </p> <p> This is one of the reasons I think F# is a better language for learning FP if you come from an Object-Oriented background: you can gradually refactor towards more Functional implementations as you become better at FP. </p> <p> The part that caused me particular difficulty was the <em>scan</em> part, where the algorithm examines all points to identify which points to discard from the hull (because they're in the interior of the hull, and not <em>on</em> the hull). </p> <p> After sorting all candidates according to special rules, the algorithm must consider each point in turn. If that new point is 'to the right' of the previous two points, the <em>previous</em> point is in the interior of the hull and should be discarded. The previous-to-previous point could also be 'to the right' of the new point, so the algorithm needs to check again, and so on. </p> <p> My imperative solution looked like this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">inline</span>&nbsp;hullPoints&nbsp;points&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;ps&nbsp;=&nbsp;[] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;p&nbsp;<span style="color:blue;">in</span>&nbsp;points&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ps&nbsp;&lt;-&nbsp;ps&nbsp;@&nbsp;[p] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;shouldCheck&nbsp;=&nbsp;<span style="color:blue;">true</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">while</span>&nbsp;shouldCheck&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;wasDiscarded,&nbsp;newPoints&nbsp;=&nbsp;check&nbsp;ps &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;shouldCheck&nbsp;&lt;-&nbsp;wasDiscarded &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;wasDiscarded&nbsp;<span style="color:blue;">then</span>&nbsp;ps&nbsp;&lt;-&nbsp;newPoints &nbsp;&nbsp;&nbsp;&nbsp;ps</pre> </p> <p> (You can see <a href="https://github.com/ploeh/Hull">the full code base on GitHub</a>. The start is at <a href="https://github.com/ploeh/Hull/tree/5290abd3c31c162ee6c4b21b82494ce97ecf7fa5">5290abd3c31c162ee6c4b21b82494ce97ecf7fa5</a>, and the end state that this post describes is at <a href="https://github.com/ploeh/Hull/tree/e3efd1b457a46112cff6f06b8cbb100d153f0ef1">e3efd1b457a46112cff6f06b8cbb100d153f0ef1</a>.) </p> <p> Due to the <code>inline</code> keyword, the hullPoints function has a complex type, but for practical purposes, think of it as having the type <code>(int * int) seq -&gt; (int * int) list</code>. The <code>points</code> argument is a sequence of coordinates: <code>(int * int) seq</code>. </p> <p> As you can see, this implementation has a nested loop. The outer loop traverses all <code>points</code> and appends the point in consideration to the mutable list variable <code>ps</code>. At this stage, <code>p</code> is only a <em>candidate</em>. What the algorithm must now determine is whether <code>p</code> is in the interior or might be a hull point. </p> <p> In order to do that, it calls another function called <code>check</code>. The check function is another inline function, but you can think about it as having the type <code>(int * int) list -&gt; bool * (int * int) list</code>. The return type is peculiar, but the idea is that it returns true in the first tuple element if points were discarded from the input, and false if no points were discarded. The second tuple element contains the points (that may or may not have had points removed compared to the input points). (I later refactored this function to a function called tryDiscard with the type <code>(int * int) list -&gt; (int * int) list option</code>.) </p> <p> If points were discarded, the algorithm must check again, as there may be more points to discard. Only when no more points were discarded can the outer loop move on to the next candidate. </p> <p> According to the <em>Recurse</em> refactoring, you need to define a recursive function for each loop. There are two loops here, but do the inner loop first. Each mutable variable should be replaced with a function argument, but fortunately there's only one: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">inline</span>&nbsp;hullPoints&nbsp;points&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;update&nbsp;candidates&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;wasDiscarded,&nbsp;newCandidates&nbsp;=&nbsp;check&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;wasDiscarded &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;update&nbsp;newCandidates &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;candidates&nbsp;=&nbsp;[] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;p&nbsp;<span style="color:blue;">in</span>&nbsp;points&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidates&nbsp;&lt;-&nbsp;candidates&nbsp;@&nbsp;[p] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;candidates&nbsp;&lt;-&nbsp;update&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;candidates</pre> </p> <p> The new <code>update</code> function calls the <code>check</code> function. If <code>wasDiscarded</code> is true, it calls itself recursively with the new candidates; otherwise, it returns the input candidates. </p> <p> The update function is now a recursive function without mutable variables, but the containing hullPoints function still has a mutable <code>candidates</code> variable. You'll need to apply the <em>Recursive</em> refactoring again: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;hullPoints&nbsp;points&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;update&nbsp;candidates&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;wasDiscarded,&nbsp;newCandidates&nbsp;=&nbsp;check&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;wasDiscarded &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;update&nbsp;newCandidates &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;hpImp&nbsp;candidates&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;[]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;candidates &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;p&nbsp;::&nbsp;tail&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;cs&nbsp;=&nbsp;candidates&nbsp;@&nbsp;[p] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;updatedCandidates&nbsp;=&nbsp;update&nbsp;cs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;hpImp&nbsp;updatedCandidates&nbsp;tail &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;hpImp&nbsp;[]&nbsp;points</pre> </p> <p> The hpImp function replaces the remaining loop, and <code>candidates</code> is now a function argument instead of mutable variable. </p> <p> As long as <code>candidates</code> has contents, the head of the list <code>p</code> is appended to <code>candidates</code>, and <code>update</code> is invoked. Subsequently, hpImp is invoked recursively with the updated candidates and the tail of the list. </p> <p> The hullPoints function returns the value of calling hpImp with empty hull candidates, and the <code>points</code> argument. This implementation has no mutable variables. </p> <p> You can refactor this implementation to make it more readable, but that's not the point of this article. You can see what I then did in <a href="https://github.com/ploeh/Hull">the GitHub repository</a>. </p> <p> (Now that I had a pure implementation, I could also port it to Haskell, which met my original goal.) </p> <p> <strong>Summary</strong> </p> <p> For programmers used to imperative programming, it can be difficult to apply pure Functional techniques. When you have loops that update a mutable variable in each step of the loop, you can refactor to use a recursive function; the mutable variable is replaced with a function argument. When you've tried it a couple of times, you'll get the hang of it. </p> <p> Once you have a recursive function with an accumulator argument, you can often further refactor it to use a fold, instead of a recursive function. </p> <p> (This post is the December 1st entry in the <a href="https://sergeytihon.wordpress.com/2015/10/25/f-advent-calendar-in-english-2015">2015 F# Advent Calendar</a>.) </p> </div><hr> This blog is totally free, but if you like it, please <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=NZEPYW8KVZ8WL">buy me a cup of coffee</a>. To log or not to log http://blog.ploeh.dk/2015/11/30/to-log-or-not-to-log 2015-11-30T08:45:00+00:00 Mark Seemann <div id="post"> <p> <em>There's no reason to make logging any harder than it has to be. Here's a compositional approach in F#.</em> </p> <p> Logging seems to be one of those cross-cutting concerns on which people tend to spend a lot of effort. For programmers coming from an object-oriented language like C#, finding a sane approach to logging seems to be particularly difficult. </p> <p> In <a href="http://amzn.to/12p90MG">my book about Dependency Injection</a>, I made an effort to explain that logging and other cross-cutting concerns are best addressed by <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorators</a> (or dynamic interception). You can use the same design with F# functions. </p> <p> <strong>Example scenario</strong> </p> <p> Consider, as an example, an HTTP API for a restaurant booking system. This example is taken from my <a href="http://bit.ly/1Hhru7l">Test-Driven Development with F#</a> Pluralsight course, but you can see an almost identical example for free in <a href="http://www.infoq.com/presentations/mock-fsharp-tdd">this recording of a BuildStuff talk</a>. </p> <p> The application is composed of small, mostly pure functions: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;imp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Validate</span>.reservationValid &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.bind&nbsp;(<span style="color:#2b91af;">Capacity</span>.check&nbsp;10&nbsp;<span style="color:#2b91af;">SqlGateway</span>.getReservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.map&nbsp;<span style="color:#2b91af;">SqlGateway</span>.saveReservation</pre> </p> <p> Exactly what each function does isn't important in this context, but here are the types involved: <table> <thead> <tr> <th>Function</th> <th>Type</th> </tr> </thead> <tbody> <tr> <td>imp</td> <td><code>ReservationRendition -&gt; Rop.Result&lt;unit, Error&gt;</code></td> </tr> <tr> <td>Validate.reservationValid</td> <td><code>ReservationRendition -&gt; Rop.Result&lt;Reservation, Error&gt;</code></td> </tr> <tr> <td>Capacity.check</td> <td><code>int -&gt; (DateTimeOffset -&gt; int) -&gt; Reservation -&gt; Rop.Result&lt;Reservation, Error&gt;</code></td> </tr> <tr> <td>SqlGateway.saveReservation</td> <td><code>Reservation -&gt; unit</code></td> </tr> </tbody> </table> where Rop.Result&lt;'success, 'error&gt;, Rop.bind, and Rop.map are as described in <a href="http://fsharpforfunandprofit.com/rop">Scott Wlaschin's Railway Oriented Programming</a>. </p> <p> In short, the <code>imp</code> function validates the input, applies some business rules if the input was valid, and saves the reservation to a database if the business rules allow it. </p> <p> I strongly believe that in any well-designed code base, the core implementation should be independent of cross-cutting concerns such as logging. If the above reservation system is well-designed, it should be possible to retrofit logging onto it without changing the existing functions. Indeed, that turns out to be possible. </p> <p> <strong>Adding logs</strong> </p> <p> You should use an existing logging library such as <a href="http://serilog.net">Serilog</a>, <a href="https://logging.apache.org/log4net">log4net</a>, <a href="http://nlog-project.org">NLog</a>, etc. instead of rolling your own. In this example, imagine that you're using the well-known <em>SomeExternalLoggingLibrary</em>. In order to protect yourself against changes etc. in the external library, you first define your own, application-specific logging module: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:#2b91af;">BookingLog</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;logError&nbsp;=&nbsp;<span style="color:#2b91af;">SomeExternalLoggingLibrary</span>.logError &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;logInformation&nbsp;=&nbsp;<span style="color:#2b91af;">SomeExternalLoggingLibrary</span>.logInformation</pre> </p> <p> Both functions have the type <code>fileName:string -&gt; msg:string -&gt; unit</code>. </p> <p> As a beginning, you can start by logging the final result of executing the <code>imp</code> function. Since it has the type <code>ReservationRendition -&gt; Rop.Result&lt;unit, Error&gt;</code>, if you implement a log function that both accepts and returns <code>Rop.Result&lt;unit, Error&gt;</code>, you can append that to the composition of <code>imp</code>. Start with the logging function itself: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;<span style="color:#2b91af;">BookingLog</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;logReservationsPost&nbsp;logFile&nbsp;result&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;result&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Failure(ValidationError&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;logError&nbsp;logFile&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Failure&nbsp;CapacityExceeded&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;logError&nbsp;logFile&nbsp;<span style="color:#a31515;">&quot;Capacity&nbsp;exceeded.&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Success&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;logInformation&nbsp;logFile&nbsp;<span style="color:#a31515;">&quot;Reservation&nbsp;saved.&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;result</pre> </p> <p> This function has the type <code>string -&gt; Rop.Result&lt;unit, Error&gt; -&gt; Rop.Result&lt;unit, Error&gt;</code>. It matches on the cases of <code>result</code> and logs something relevant for each case; then it returns <code>result</code> without modifying it. </p> <p> Since the logReservationsPost function both accepts and returns the same type, you can easily append it to the other functions while composing <code>imp</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;imp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Validate</span>.reservationValid &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.bind&nbsp;(<span style="color:#2b91af;">Capacity</span>.check&nbsp;10&nbsp;<span style="color:#2b91af;">SqlGateway</span>.getReservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.map&nbsp;<span style="color:#2b91af;">SqlGateway</span>.saveReservation &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">BookingLog</span>.logReservationsPost&nbsp;logFile</pre> </p> <p> Notice how BookingLog.logReservationsPost is simply added as the last line of composition. This compiles because that function returns its input. </p> <p> Running the application with various input demonstrates that logging works as intended: </p> <p> <pre>Information: Reservation saved. Error: Invalid date. Information: Reservation saved. Error: Capacity exceeded.</pre> </p> <p> You've seen that you can append high-level logging of the final value, but can you also add logging deeper in the guts of the implementation? </p> <p> <strong>Logging business behaviour</strong> </p> <p> Imagine that you need to also log what happens before and after Capacity.check is called. One option is to add a logging function with the same type as Capacity.check, that also Decorates Capacity.check, but I think it's simpler to add two functions that log the values before and after Capacity.check. </p> <p> The type of Capacity.check is <code>int -&gt; (DateTimeOffset -&gt; int) -&gt; Reservation -&gt; Rop.Result&lt;Reservation, Error&gt;</code>, but after partial application, it's only <code>Reservation -&gt; Rop.Result&lt;Reservation, Error&gt;</code>. In order to log what happens before Capacity.check is called, you can add a function that both accepts and returns a Reservation: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;logBeforeCapacityCheck&nbsp;logFile&nbsp;reservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;logInformation &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;logFile &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(sprintf &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Checking&nbsp;capacity&nbsp;for&nbsp;%s...&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(reservation.Date.ToString&nbsp;<span style="color:#a31515;">&quot;d&quot;</span>)) &nbsp;&nbsp;&nbsp;&nbsp;reservation</pre> </p> <p> This function has the type <code>string -&gt; Reservation -&gt; Reservation</code>, and is placed within the BookingLog module. The logInformation function is used to log the input, which is then returned. </p> <p> Likewise, you can also log what happens after Capacity.check is invoked. Since Capacity.check returns Rop.Result&lt;Reservation, Error&gt;, your log file must take that type as both input and output: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;logAfterCapacityCheck&nbsp;logFile&nbsp;result&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;result&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Failure(ValidationError&nbsp;msg)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;logError&nbsp;logFile&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Failure&nbsp;CapacityExceeded&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;logError&nbsp;logFile&nbsp;(sprintf&nbsp;<span style="color:#a31515;">&quot;Capacity&nbsp;exceeded.&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Success&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;logInformation &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;logFile &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(sprintf&nbsp;<span style="color:#a31515;">&quot;All&nbsp;is&nbsp;good&nbsp;for&nbsp;%s.&quot;</span>&nbsp;(r.Date.ToString&nbsp;<span style="color:#a31515;">&quot;d&quot;</span>)) &nbsp;&nbsp;&nbsp;&nbsp;result</pre> </p> <p> The logAfterCapacityCheck function has the type <code>string -&gt; Rop.Result&lt;Reservation, Error&gt; -&gt; Rop.Result&lt;Reservation, Error&gt;</code>, and is also placed within the BookingLog module. Like the logReservationsPost function, it matches on <code>result</code> and logs accordingly; then it returns <code>result</code>. Do you see a pattern? </p> <p> Because of these types, you can compose them into <code>imp</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;imp&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Validate</span>.reservationValid &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.map&nbsp;(<span style="color:#2b91af;">BookingLog</span>.logBeforeCapacityCheck&nbsp;logFile) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.bind&nbsp;(<span style="color:#2b91af;">Capacity</span>.check&nbsp;10&nbsp;<span style="color:#2b91af;">SqlGateway</span>.getReservedSeats) &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">BookingLog</span>.logAfterCapacityCheck&nbsp;logFile &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">Rop</span>.map&nbsp;<span style="color:#2b91af;">SqlGateway</span>.saveReservation &nbsp;&nbsp;&nbsp;&nbsp;&gt;&gt;&nbsp;<span style="color:#2b91af;">BookingLog</span>.logReservationsPost&nbsp;logFile</pre> </p> <p> Notice that BookingLog.logBeforeCapacityCheck and BookingLog.logAfterCapacityCheck are composed around Capacity.check. The final BookingLog.logReservationsPost is also still in effect. Running the application shows that logging still works: </p> <p> <pre>Information: Checking capacity for 27.11.2015... Information: All is good for 27.11.2015. Information: Reservation saved.</pre> </p> <p> The first two log entries are created by the logs around Capacity.check, whereas the last line is written by BookingLog.logReservationsPost. </p> <p> <strong>Conditional logging</strong> </p> <p> Some programmers are concerned about the performance implications of logging. You may wish to be able to control whether or not to log. </p> <p> The easiest way to do that is to make logging itself conditional: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;logError&nbsp;fileName&nbsp;msg&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;log &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;SomeExternalLoggingLibrary.logError&nbsp;fileName&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;() <span style="color:blue;">let</span>&nbsp;logInformation&nbsp;fileName&nbsp;msg&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;log &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;SomeExternalLoggingLibrary.logInformation&nbsp;fileName&nbsp;msg &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;()</pre> </p> <p> where <code>log</code> is a boolean value. If <code>log</code> is false, the above two functions simply return <code>()</code> (unit) without doing anything. This prevents costly IO from happening, so may already be enough of a performance optimisation. As always when performance is the topic: <em>don't assume anything; measure</em>. </p> <p> In reality, you probably want to use more granular flags than a singl