ploeh blog 2021-01-18T06:31:42+00:00 Mark Seemann danish software design https://blog.ploeh.dk Parametrised test primitive obsession code smell https://blog.ploeh.dk/2021/01/18/parametrised-test-primitive-obsession-code-smell 2021-01-18T06:30:00+00:00 Mark Seemann <div id="post"> <p> <em>Watch out for this code smell with some unit testing frameworks.</em> </p> <p> In a <a href="/2021/01/11/waiting-to-happen">previous article</a> you saw this <a href="/2019/04/01/an-example-of-state-based-testing-in-c">state-based integration test</a>: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> This was the test <em>after</em> I improved it. Still, I wasn't satisfied with it. It has several problems. Take a few moments to consider it. Can you identify any problems? Which ones? </p> <h3 id="6da86d1040e544a3a67f509c0f3aade7"> Size <a href="#6da86d1040e544a3a67f509c0f3aade7" title="permalink">#</a> </h3> <p> I know that you're not familiar with all the moving parts. You don't know how <code>ReservationDto</code> or <code>Reservation</code> are implemented. You don't know what <code>InMemoryRestaurantDatabase</code> is, or how <code>ReservationsController</code> behaves. Still, the issues I have in mind aren't specific to a particular code base. </p> <p> I feel that the method is verging on being too big. Quantifiably, it doesn't fit in an <a href="/2019/11/04/the-80-24-rule">80x24 box</a>, but that's just an arbitrary <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a> anyway. Still, I think it's grown to a size that makes me uncomfortable. </p> <p> If you aren't convinced, think of this code example as a stand-in for something larger. In the above test, a reservation contains five smaller values (<code>Id</code>, <code>At</code>, <code>Email</code>, <code>Name</code>, and <code>Quantity</code>). How would a similar test look if the object in question contains ten or twenty values? </p> <p> In the decades I've been programming and consulting, I've seen plenty of code bases. Data objects made from twenty fields are hardly unusual. </p> <p> What would a similar test look like if the <code>dto</code> and the <code>expected</code> object required twenty smaller values? </p> <p> The test would be too big. </p> <h3 id="e251bee2df414c5cb0d489f31183c664"> Primitive obsession <a href="#e251bee2df414c5cb0d489f31183c664" title="permalink">#</a> </h3> <p> A test like this one contains a mix of essential behaviour and implementation details. The behaviour that it verifies is that when you <code>Post</code> a valid <code>dto</code>, the data makes it all the way to the database. </p> <p> Exactly how the <code>dto</code> or the <code>expected</code> value are constructed is less relevant for the test. Yet it's intermingled with the test of behaviour. The signal-to-noise ratio in the test isn't all that great. What can you do to improve things? </p> <p> As given, it seems difficult to do much. The problem is <a href="/2011/05/25/DesignSmellPrimitiveObsession">primitive obsession</a>. While this is a <a href="http://xunitpatterns.com/Parameterized%20Test.html">parametrised test</a>, all the method parameters are primitives: integers and strings. The makes it hard to introduce useful abstractions. </p> <p> In C# (and probably other languages as well) parametrised tests often suffer from primitive obsession. The most common data source is an attribute (AKA <em>annotation</em>), like <a href="https://xunit.net">xUnit.net</a>'s <code>[InlineData]</code> attribute. This isn't a limitation of xUnit.net, but rather of .NET attributes. You can only create attributes with primitive values and arrays. </p> <p> What <em>is</em> a limitation of xUnit.net (and the other mainstream .NET testing frameworks, as far as I know) is that tests aren't first-class values. In <a href="https://www.haskell.org">Haskell</a>, by contrast, it's <a href="/2018/04/30/parametrised-unit-tests-in-haskell">easy to write parametrised tests using the normal language constructs</a> exactly because tests are first-class values. (I hope that the next version of xUnit.net will support tests as first-class values.) </p> <p> Imagine that instead of only five constituent fields, you'd have to write a parametrised test for objects with twenty primitive values. As long as you stick with attribute-based data sources, you'll be stuck with primitive values. </p> <p> Granted, attributes like <code>[InlineData]</code> are lightweight, but over the years, my patience with them has grown shorter. They lock me into primitive obsession, and I don't appreciate that. </p> <h3 id="9342d4306e174fe79946a131b9c6894c"> Essential test <a href="#9342d4306e174fe79946a131b9c6894c" title="permalink">#</a> </h3> <p> While tests as first-class values aren't an option in xUnit.net, you can provide other data sources for the <code>[Theory]</code> attribute than <code>[InlineData]</code>. It's not as lightweight, but it breaks the primitive obsession and re-enables normal code design techniques. It enables you to reduce the test itself to its essence. You no longer have to think in primitives, but can instead express the test unshackled by constraints. As a first pass, I'd like the test to look like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">ClassData</span>(<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;validDto,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;expected) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(validDto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> This version of the test eliminates the noise. How <code>validDto</code> and <code>expected</code> are constructed is an implementation detail that has little bearing on the behaviour being tested. </p> <p> For a reader of the code, it's should now be clearer what's at stake here: If you <code>Post</code> a <code>validDto</code> the <code>expected</code> reservation should appear in the database. </p> <p> Reducing the test code to its essentials made me realise something that hitherto had escaped me: that I could <a href="/2020/11/30/name-by-role">name the DTO by role</a>. Instead of just <code>dto</code>, I could call the parameter <code>validDto</code>. </p> <p> Granted, I could also have done that previously, but I didn't think of it. There's was so much noise in that test that I didn't stop to consider whether <code>dto</code> sufficiently communicated the role of that variable. </p> <p> The less code, the easier it becomes to think such things through, I find. </p> <p> In any case, the test code now much more succinctly expresses the essence of the desired behaviour. Notice how I started my refactoring by writing the desired test code. I've yet to implement the data source. Now that the data source expresses test data as full objects, I'm not so concerned with whether or not that's going to be possible. Of course it's possible. </p> <h3 id="469ee6ae8f264e21a544f11be2476111"> Object data source <a href="#469ee6ae8f264e21a544f11be2476111" title="permalink">#</a> </h3> <p> You can define data sources for xUnit.net as classes or methods. In C# I usually reach for the <code>[ClassData]</code> option, since an object (in C#, that is) gives me better options for further decomposition. For example, I can define a class and delegate the details to helper methods: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TheoryData</span>&lt;<span style="color:#2b91af;">ReservationDto</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithoutName(956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;2); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;here...</span></pre> </p> <p> Here, I'm taking advantage of xUnit.net's built-in <code>TheoryData&lt;T1, T2&gt;</code> base class, but that's just a convenience. All you have to do is to implement <code>IEnumerable&lt;object[]&gt;</code>. </p> <p> As you can see, the constructor adds the four test cases by calling two private helper methods. Here's the first of those: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>; <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithName( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(name), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity)); }</pre> </p> <p> The other helper method is almost identical, although it has a slight variation when it comes to the reservation name: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithoutName( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity)); }</pre> </p> <p> In total, this refactoring results in <em>more</em> code, so how is this an improvement? </p> <h3 id="9722aade54194b1a87c7312d16ed7fc8"> The paradox of decomposition <a href="#9722aade54194b1a87c7312d16ed7fc8" title="permalink">#</a> </h3> <p> In object-oriented design, decomposition tends to lead to more code. If you want to isolate and make reusable a particular piece of behaviour, you'll usually introduce an interface or a base class. Even stateless functions need a static class to define them. (To be fair, functional programming isn't entirely devoid of such overhead associated with decomposition, but the cost tends to smaller.) This leads to more code, compared with the situation before decomposition. </p> <p> This is a situation you may also encounter if you attempt to refactor to design patterns, or follow the <a href="/encapsulation-and-solid">SOLID principles</a>. You'll have more code than when you started. This often leads to resistance to such 'code bloat'. </p> <p> It's fine to resist code bloat. It's also fine to dislike 'complexity for complexity's sake'. Try to evaluate each potential code change based on advantages and disadvantages. I'm not insisting that the above refactoring is objectively better. I did feel, however, that I had a problem that I ought to address, and that this was a viable alternative. The result is more code, but each piece of code is smaller and simpler. </p> <p> You can, conceivably, read the test method itself to get a feel for what it tests, even if you don't know all the implementation details. You can read the four statements in the <code>PostValidReservationWhenDatabaseIsEmptyTestCases</code> constructor without, I hope, understanding all the details about the two helper methods. And you <em>can</em> read <code>AddWithName</code> without understanding how <code>AddWithoutName</code> works, and vice versa, because these two methods don't depend on each other. </p> <h3 id="27b546f17ba44571a11ed44db9d45a09"> Conclusion <a href="#27b546f17ba44571a11ed44db9d45a09" title="permalink">#</a> </h3> <p> In this article, I've described how the use of code annotations for parametrised tests tend to pull in the direction of primitive obsession. This is a force worth keeping an eye on, I think. </p> <p> You saw how to refactor to class-based test data generation. This enables you to use objects instead of primitives, thus opening your design palette. You can now use all your usual object-oriented or functional design skills to factor the code in a way that's satisfactory. </p> <p> Was it worth it in this case? Keep in mind that the original problem was already marginal. While the code didn't fit in a 80x24 box, it was only 33 lines of code (excluding the test data). Imagine, however, that instead of a five-field reservation, you'd be dealing with a twenty-field data class, and such a refactoring begins to look more compelling. </p> <p> Is the code now perfect? It still isn't. I'm a little put off by the similarity of <code>AddWithName</code> and <code>AddWithoutName</code>. I'm also aware that there's a trace of production code duplicated in the test case, in the way that the test code duplicates how a valid <code>ReservationDto</code> relates to a <code>Reservation</code>. I'm on the fence whether I should do anything about this. </p> <p> At the moment I'm inclined to heed <a href="https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)">the rule of three</a>. The duplication is still too insubstantial to warrant refactoring, but it's worth keeping an eye on. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Waiting to happen https://blog.ploeh.dk/2021/01/11/waiting-to-happen 2021-01-11T06:31:00+00:00 Mark Seemann <div id="post"> <p> <em>A typical future test maintenance problem.</em> </p> <p> In <a href="/2020/12/07/branching-tests">a recent article</a> I showed a unit test and parenthetically mentioned that it might have a future maintenance problem. Here's a more recent version of the same test. Can you tell what the future issue might be? </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2022-03-18&nbsp;17:30&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> To be honest, there's more than one problem with this test, but presently I'm going to focus on one of them. </p> <p> Since you don't know the details of the implementation, you may not be able to tell what the problem might be. It's not a trick question. On the other hand, you might still be able to guess, just from the clues available in the above code listing. </p> <h3 id="970cada688cc4a04abbb1c11328931f1"> Sooner or later <a href="#970cada688cc4a04abbb1c11328931f1" title="permalink">#</a> </h3> <p> Here are some clues to consider: I'm writing this article in the beginning of 2021. Consider the dates supplied via the <code>[InlineData]</code> attributes. Seen from 2021, they're all in the future. </p> <p> Notice, as well, that the <code>sut</code> takes a <code>SystemClock</code> dependency. You don't know the <code>SystemClock</code> class (it's a proprietary class in this code base), but from the name I'm sure that you can imagine what it represents. </p> <p> From the perspective of early 2021, all dates are going to be in the future for more than a year. What is going to happen, though, once the test runs after March 18, 2022? </p> <p> That test case is going to fail. </p> <p> You can't tell from the above code listing, but the system under test rejects reservations in the past. Once March 18, 2022 has come and gone, the reservation at <code>"2022-03-18 17:30"</code> is going to be in the past. The <code>sut</code> will reject the reservation, and the assertion will fail. </p> <p> You have to be careful with tests that rely on the system clock. </p> <h3 id="61e0c1225dd34c40a0ceb19270ae9007"> Test Double? <a href="#61e0c1225dd34c40a0ceb19270ae9007" title="permalink">#</a> </h3> <p> The fundamental problem is that the system clock is non-deterministic. A typical reaction to non-determinism in unit testing is to introduce a <a href="https://martinfowler.com/bliki/TestDouble.html">Test Double</a> of some sort. Instead of using the system clock, you could use a <a href="/2013/10/23/mocks-for-commands-stubs-for-queries">Stub</a> as a stand-in for the real time. </p> <p> This is possible here as well. The <code>ReservationsController</code> class actually depends on an <code>IClock</code> interface that <code>SystemClock</code> implements. You could define a test-specific <code>ConstantClock</code> implementation that would always return a constant date and time. This would actually work, but would rely on an implementation detail. </p> <p> At the moment, the <code>ReservationsController</code> only calls <code>Clock.GetCurrentDateTime()</code> a <em>single time</em> to get the current time. As soon as it has that value, it passes it to a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a>, which implements <a href="/2020/01/27/the-maitre-d-kata">the business logic</a>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); <span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError();</pre> </p> <p> A <code>ConstantClock</code> would work, but only as long as the <code>ReservationsController</code> only calls <code>Clock.GetCurrentDateTime()</code> once. If it ever began to call this method multiple times to detect the passing of time, using a constant time value would mostly likely again break the test. This seems brittle, so I don't want to go that way. </p> <h3 id="29f9100aa1104cd2a1db18146384bc17"> Relative time <a href="#29f9100aa1104cd2a1db18146384bc17" title="permalink">#</a> </h3> <p> Working with the system clock in automated tests is easier if you deal with relative time. Instead of defining the test cases as absolute dates, express them as days into the future. Here's one way to refactor the test: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> The absolute dates always were fairly arbitrary, so I just took the current date and converted the dates to a number of days into the future. Now, the first test case will always be a date 1,049 days (not quite three years) into the future, instead of November 24, 2023. </p> <p> The test is no longer a failure waiting to happen. </p> <h3 id="906499c9b3d647d08d2f48eb5991cf43"> Conclusion <a href="#906499c9b3d647d08d2f48eb5991cf43" title="permalink">#</a> </h3> <p> Treating test cases that involve time and date as relative to the current time, instead of as absolute values, is usually a good idea if the system under test depends on the system clock. </p> <p> It's always a good idea to factor as much code as you can as pure functions, like the above <code>WillAccept</code> method. Pure functions don't depend on the system clock, so here you can safely pass absolute time and date values. Pure functions are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. </p> <p> Still, as the <a href="https://martinfowler.com/bliki/TestPyramid.html">test pyramid</a> suggests, relying exclusively on unit tests isn't a good idea. The test shown in this article isn't really a unit test, but rather a <a href="/2019/04/01/an-example-of-state-based-testing-in-c">state-based integration test</a>. It relies on both the system clock and a <a href="http://xunitpatterns.com/Fake%20Object.html">Fake</a> database. Expressing the test cases for this test as relative time values effectively addresses the problem introduced by the system clock. </p> <p> There are plenty of other problems with the above test. One thing that bothers me is that the 'fix' made the line count grow. It didn't quite fit into a <a href="/2019/11/04/the-80-24-rule">80x24 box</a> before, but now it's even worse! I should do something about that, but that's a topic for <a href="/2021/01/18/parametrised-test-primitive-obsession-code-smell">another article</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Dynamic test oracles for rho problems https://blog.ploeh.dk/2021/01/04/dynamic-test-oracles-for-rho-problems 2021-01-04T06:26:00+00:00 Mark Seemann <div id="post"> <p> <em>A proof of concept of cross-branch testing for compiled languages.</em> </p> <p> <a href="https://www.hillelwayne.com">Hillel Wayne</a> recently published an article called <a href="https://buttondown.email/hillelwayne/archive/cross-branch-testing/">Cross-Branch Testing</a>. It outlines an approach to a class of problems that are hard to test. He mentions computer vision and simulations, among others. I can add that it's also <a href="/2015/10/19/visual-value-verification">difficult to write intuitive tests of convex hulls</a> and <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Conway's game of life</a>. </p> <p> Hillel Wayne calls these <em>rho problems</em>, 'just because'. I'm totally going to run with that term. </p> <p> In the article, he outlines an approach where you test an iteration of rho code against a 'last known good' snapshot. He uses <code>git worktree</code> to set up a snapshot of the reference implementation. He then writes a property that compares the refactored code's behaviour against the reference. </p> <p> The example code is in <a href="https://www.python.org">Python</a>, which is a language that I don't know. As far as I can tell, it works because Python is 'lightweight' enough that you can load and execute source code directly. I found that the approach makes much sense, but I wondered how it would apply for statically typed, compiled languages. I decided to create a proof of concept in <a href="https://fsharp.org">F#</a>. </p> <h3 id="ab0c6139b2c84e148fe1173c2508ec62"> Test cases from Python <a href="#ab0c6139b2c84e148fe1173c2508ec62" title="permalink">#</a> </h3> <p> My first problem was to port Hillel Wayne's example rho problem to F#. The function <code>f</code> doesn't have any immediate mathematical properties; nor is its behaviour intuitive. While I think that I understand what each line of code in <code>f</code> means, I don't really know Python. Since one of the properties of rho problems is that bugs can be subtle, I didn't trust myself to be able to port the Python code to F# without some test cases. </p> <p> To solve that problem, I first found an online Python interpreter and pasted the <code>f</code> function into it. I then wrote code to print the output of a function call: </p> <p> <pre>print(f'1, 2, 3, { f(1, 2, 3) }')</pre> </p> <p> This line of code produces this output: </p> <p> <pre>1, 2, 3, True</pre> </p> <p> In other words, I could produce comma-separated values of input and actual output. </p> <p> Hillel Wayne wrote properties using <a href="https://hypothesis.works">Hypothesis</a>, which, <a href="https://hypothesis.works/articles/how-many-tests">it seems</a>, by default runs each property 200 times. </p> <p> In F# I'm going to use <a href="https://fscheck.github.io/FsCheck">FsCheck</a>, so I first used <em>F# Interactive</em> with FsCheck to produce 200 Python <code>print</code> statements like the above: </p> <p> <pre>&gt; Arb.Default.Int32().Generator |&gt; Gen.three |&gt; Gen.map (fun (x, y, z) -&gt; sprintf "print(f'%i, %i, %i, { f(%i, %i, %i) }')" x y z x y z) |&gt; Gen.sample 100 200 |&gt; List.iter (printfn "%s");; print(f'-77, 67, 84, { f(-77, 67, 84) }') print(f'58, -46, 3, { f(58, -46, 3) }') print(f'21, 13, 94, { f(21, 13, 94) }') ... </pre> </p> <p> This is a throwaway data pipeline that starts with an FsCheck integer generator, creates a triple from it, turns that triple into a Python <code>print</code> statement, and finally writes 200 of those to the console. The above code listing only shows the first three lines of output, while the rest are indicated by an ellipsis. </p> <p> I copied those 200 <code>print</code> statements over to the online Python interpreter and ran the code. That produced 200 comma-separated values like these: </p> <p> <pre>-77, 67, 84, False 58, -46, 3, False 21, 13, 94, True ...</pre> </p> <p> These can serve as test cases for porting the Python code to F#. </p> <h3 id="936410215a784d73ae9b5dcba9125a4c"> Port to F# <a href="#936410215a784d73ae9b5dcba9125a4c" title="permalink">#</a> </h3> <p> The next step is to write a parametrised test, using a provisional implementation of <code>f</code>: </p> <p> <pre>[&lt;Theory;&nbsp;MemberData(nameof&nbsp;fTestCases)&gt;] <span style="color:blue;">let</span>&nbsp;``test&nbsp;f``&nbsp;x&nbsp;y&nbsp;z&nbsp;expected&nbsp;=&nbsp;expected&nbsp;=!&nbsp;f&nbsp;x&nbsp;y&nbsp;z</pre> </p> <p> This test uses <a href="https://xunit.net">xUnit.net</a> 2.4.1 and <a href="https://github.com/SwensenSoftware/Unquote">Unquote</a> 5.0.0. As you can tell, apart from the annotations, it's a true one-liner. It calls the <code>f</code> function with the three supplied arguments <code>x</code>, <code>y</code>, and <code>z</code> and compares the return value with the <code>expected</code> value. </p> <p> The code uses the new <a href="https://docs.microsoft.com/dotnet/fsharp/language-reference/nameof">nameof</a> feature of F# 5. <code>fTestCases</code> is a function in the same module that holds the test: </p> <p> <pre><span style="color:green;">//&nbsp;unit&nbsp;-&gt;&nbsp;seq&lt;obj&nbsp;[]&gt;</span> <span style="color:blue;">let</span>&nbsp;fTestCases&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;strm&nbsp;=&nbsp;typeof&lt;Anchor&gt;.Assembly.GetManifestResourceStream&nbsp;streamName &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;rdr&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;StreamReader&nbsp;(strm) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;s&nbsp;=&nbsp;rdr.ReadToEnd&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;s.Split&nbsp;Environment.NewLine&nbsp;|&gt;&nbsp;Seq.map&nbsp;csvToTestCase</pre> </p> <p> It reads an embedded resource stream of test cases, like the above comma-separated values. Even though the values are in a text file, it's easier to embed the file in the test assembly, because it nicely dispenses with the problem of copying a text file to the appropriate output directory when the code compiles. That would, however, be an valid alternative. </p> <p> <code>Anchor</code> is a dummy type to support <code>typeof</code>, and <code>streamName</code> is just a string constant that identifies the name of the stream. </p> <p> The <code>csvToTestCase</code> function converts each line of comma-separated values to test cases for the <code>[&lt;Theory&gt;]</code> attribute: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;obj&nbsp;[]</span> <span style="color:blue;">let</span>&nbsp;csvToTestCase&nbsp;(csv&nbsp;:&nbsp;string)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;values&nbsp;=&nbsp;csv.Split&nbsp;<span style="color:#a31515;">&#39;,&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;[| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[0]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[1]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[2]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[3]&nbsp;|&gt;&nbsp;Convert.ToBoolean&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;|]</pre> </p> <p> It's not the safest code I could write, but this is, after all, a proof of concept. </p> <p> The most direct port of the Python code I could produce is this: </p> <p> <pre><span style="color:green;">//&nbsp;f&nbsp;:&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;f&nbsp;(x&nbsp;:&nbsp;int)&nbsp;(y&nbsp;:&nbsp;int)&nbsp;(z&nbsp;:&nbsp;int)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;mx&nbsp;=&nbsp;bigint&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;my&nbsp;=&nbsp;bigint&nbsp;y &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;mz&nbsp;=&nbsp;bigint&nbsp;z &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;out&nbsp;=&nbsp;0I &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;i&nbsp;<span style="color:blue;">in</span>&nbsp;[0I..9I]&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;out&nbsp;*&nbsp;mx&nbsp;+&nbsp;abs&nbsp;(my&nbsp;*&nbsp;mz&nbsp;-&nbsp;i&nbsp;*&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&#39;&nbsp;=&nbsp;mx &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;y&#39;&nbsp;=&nbsp;my &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;z&#39;&nbsp;=&nbsp;mz &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mx&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;y&#39;&nbsp;+&nbsp;1I &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;my&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;z&#39; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mz&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;x&#39; &nbsp;&nbsp;&nbsp;&nbsp;abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;10I</pre> </p> <p> As F# code goes, it's disagreeable, but it passes all 200 test cases, so this will serve as an initial implementation. The <code>out</code> variable can grow to values that overflow even 64-bit integers, so I had to convert to <a href="https://docs.microsoft.com/dotnet/api/system.numerics.biginteger">bigint</a> to get all test cases to pass. </p> <p> If I make the same mutation to the code that Hillel Wayne did (<code>abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;9I</code>) two test cases fail. This gives me some confidence that I have a degree of problem coverage comparable to his. </p> <h3 id="289e30b4098c4e54b8390af0b672caf1"> Test oracle <a href="#289e30b4098c4e54b8390af0b672caf1" title="permalink">#</a> </h3> <p> Now that a reference implementation exists, we can use it as a <a href="https://en.wikipedia.org/wiki/Test_oracle">test oracle</a> for refactorings. You can, for example, add a little test-only utility to your program portfolio: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Prod <span style="color:blue;">open</span>&nbsp;FsCheck [&lt;EntryPoint&gt;] <span style="color:blue;">let</span>&nbsp;main&nbsp;argv&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Arb.Default.Int32().Generator &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Gen.three &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Gen.sample&nbsp;100&nbsp;200 &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;List.iter&nbsp;(<span style="color:blue;">fun</span>&nbsp;(x,&nbsp;y,&nbsp;z)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;%i,&nbsp;%i,&nbsp;%i,&nbsp;%b&quot;</span>&nbsp;x&nbsp;y&nbsp;z&nbsp;(f&nbsp;x&nbsp;y&nbsp;z)) &nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;<span style="color:green;">//&nbsp;return&nbsp;an&nbsp;integer&nbsp;exit&nbsp;code</span></pre> </p> <p> Notice that the last step in the pipeline is to output the values of each <code>x</code>, <code>y</code>, and <code>z</code>, as well as the result of calling <code>f x y z</code>. </p> <p> This is a command-line executable that uses FsCheck to produce new test cases by calling the <code>f</code> function. It looks similar to the above one-off script that produced Python code, but this one instead just produces comma-separated values. You can run it from the command line to produce a new sample of test cases: </p> <p> <pre>$ ./foracle 29, -48, -78, false -8, -25, 13, false -74, 34, -68, true ...</pre> </p> <p> As above, I've used an ellipsis to indicate that in reality, 200 lines of comma-separated values scroll by. </p> <p> When you use Bash, you can even pipe the output straight to a file: </p> <p> <pre>$ ./foracle > csv.txt</pre> </p> <p> You can now take the new comma-separated values and update the test values that the above <code>test f</code> test uses. </p> <p> In other words, you use version <em>n</em> of <code>f</code> as a test oracle for version <em>n + 1</em>. When iteration <em>n + 1</em> is a function of iteration <em>n</em>, you have a so-called <em>dynamic system</em>, so I think that we can call this technique <em>dynamic test oracles</em>. </p> <p> The above <code>foracle</code> program is just a proof of concept. You could make it more flexible by making it take command-line arguments that would let you control the sample size and FsCheck's <code>size</code> parameter (the hard-coded <code>100</code> in the above code listing). </p> <h3 id="b7cee6f3ed874b3390276dd3852850a6"> Refactoring <a href="#b7cee6f3ed874b3390276dd3852850a6" title="permalink">#</a> </h3> <p> With the confidence instilled by the test cases, we can now refactor the <code>f</code> function: </p> <p> <pre><span style="color:green;">//&nbsp;f&nbsp;:&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;f&nbsp;(x&nbsp;:&nbsp;int)&nbsp;(y&nbsp;:&nbsp;int)&nbsp;(z&nbsp;:&nbsp;int)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;imp&nbsp;(x,&nbsp;y,&nbsp;z,&nbsp;out)&nbsp;i&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;out&nbsp;=&nbsp;out&nbsp;*&nbsp;x&nbsp;+&nbsp;abs&nbsp;(y&nbsp;*&nbsp;z&nbsp;-&nbsp;i&nbsp;*&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;y&nbsp;+&nbsp;1I,&nbsp;z,&nbsp;x,&nbsp;out &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;(_,&nbsp;_,&nbsp;_,&nbsp;out)&nbsp;=&nbsp;List.fold&nbsp;imp&nbsp;(bigint&nbsp;x,&nbsp;bigint&nbsp;y,&nbsp;bigint&nbsp;z,&nbsp;0I)&nbsp;[0I..9I] &nbsp;&nbsp;&nbsp;&nbsp;abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;10I</pre> </p> <p> Instead of all those mutable variables, the function is, after all, just a left fold. Phew, I feel better now. </p> <h3 id="f849e03594b448299ba4eef5a6a72b4e"> Conclusion <a href="#f849e03594b448299ba4eef5a6a72b4e" title="permalink">#</a> </h3> <p> This article demonstrated a proof of concept where you use a known good version (<em>n</em>) of the code as a test oracle for the next version (<em>n + 1</em>). In interpreted languages, you may be able to load two versions of the code base side by side, but that's rarely practical in a statically typed compiled language like F#. Instead, I used a utility program to generate test cases that can be used as a data source for a parametrised test. </p> <p> The example rho problem takes only integers as input, and returns a simple Boolean value, so in this case it's trivial to encode each test case as a line of comma-separated values. For (real) problems that may involve more complex types, it'd be better to use another serialisation format, such as JSON or XML. </p> <p> An outstanding issue is whether it's possible to implement shrinking behaviour when tests fail. Currently, the proof of concept just uses a set of serialised test cases. Normally, when a <a href="/property-based-testing-intro">property-based testing</a> framework like FsCheck detects a counter-example, it'll shrink the counter-example to values that are easier to understand than the original. This proof of concept doesn't do that. I'm not sure if a framework like FsCheck currently contains enough extensibility points to enable that sort of behaviour. I'll leave that question open for any reader interested in taking on that problem. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bb31c6eea41f4adcacf249d39a3798d2"> <div class="comment-author"><a href="https://github.com/dharmaturtle">Alex Nguyen</a></div> <div class="comment-content"> <p> Hi Mark! Thanks for another thought provoking post. </p> <p> I believe you and Hillel are writing <a href="https://en.wikipedia.org/wiki/Characterization_test">characterization tests</a>, which <a href="https://blog.ploeh.dk/2013/04/02/why-trust-tests/">you've mentioned in the past</a>. Namely, you're both using the behavior of existing code to verify the correctness of a refactor. The novel part to me is that Hillel is using code as the test oracle. Your solution serializes the oracle to a static file. The library I use for characterization tests (<a href="https://www.nuget.org/packages/ApprovalTests">ApprovalTests</a>) does this as well. </p> <p> I believe shrinking is impossible when the oracle is a static file. However with Hillel's solution the oracle may be consulted at any time, making shrinking viable. If only there was a practical way to combine the two... </p> </div> <div class="comment-date">2021-01-06 23:01 UTC</div> </div> <div class="comment" id="4aa4188124ca4f4fadf35d6881b9452e"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> A thought provoking post indeed! </p> <blockquote> In F# I'm going to use <a href="https://fscheck.github.io/FsCheck">FsCheck</a>... </blockquote> <p> I think that is a fine choice given the use case laid out in this post. In general though, I think <a href="https://github.com/hedgehogqa/fsharp-hedgehog">Hedgehog</a> is a better property-based testing library. Its killer feature is integrated shrinking, which means that all generators can also shrink and this extra power is essentially free. </p> <p> For the record (because this can be a point of confusion), Haskell has <a href="https://hackage.haskell.org/package/QuickCheck">QuickCheck</a> and <a href="https://hackage.haskell.org/package/hedgehog">(Haskell) Hedgehog</a> while F# has ports from Haskell called <a href="https://fscheck.github.io/FsCheck">FsCheck</a> and <a href="https://github.com/hedgehogqa/fsharp-hedgehog">(FSharp) Hedgehog</a>. </p> <p> <a href="https://twitter.com/jacobstanley">Jacob Stanley</a> gave <a href="https://www.youtube.com/watch?v=AIv_9T0xKEo">this excellent talk at YOW! Lambda Jam 2017</a> that explains the key idea that allows Hedgehog to have integrated shrinking. (Spoiler: A generic type that is invariant in its only type parameter is replaced by a different generic type that is monadic in its only type parameter. API design guided by functional programming for the win!) </p> <blockquote> Normally, when a property-based testing framework like FsCheck detects a counter-example, it'll shrink the counter-example to values that are easier to understand than the original. </blockquote> <p> In my experience, property-based tests written with QuickCheck / FsCheck do not normally shrink. I think this is because of the extra work required to enable shrinking. For an anecdotal example, consider <a href="https://frasertweedale.github.io/blog-fp/posts/2020-03-31-quickcheck-hedgehog.html">this post by Fraser Tweedale</a>. He believed that it would be faster to add (Haskell) Hedgehog as a dependency and create a generator for it than to add shrinking to his existing generator in QuickCheck. </p> <blockquote> In other words, you use version <em>n</em> of <code>f</code> as a test oracle for version <em>n + 1</em>. When iteration <em>n + 1</em> is a function of iteration <em>n</em>, you have a so-called <em>dynamic system</em>, so I think that we can call this technique <em>dynamic test oracles</em>. </blockquote> <p> I am confused by this paragraph. I interpret your word "When" at the start of the second sentence as a common-language synonym for the mathematical word "If". Here is roughly how I understand that paragraph, where <code>A</code> stands for "version / iteration <em>n</em> of <code>f</code>" and <code>B</code> stands for "version / iteration <em>n + 1</em> of <code>f</code>". "<code>A</code> depends on <code>B</code>. If <code>B</code> depends on <code>A</code>, then we have a dynamic system. Therefore, we have a dynamic system." I feel like the paragraph assumes (because it is obvious?) that version / iteration <em>n + 1</em> of <code>f</code> depends on version / iteration <em>n</em> of <code>f</code>. In what sense is that the case? </p> <blockquote> An outstanding issue is whether it's possible to implement shrinking behaviour when tests fail. [...] I'll leave that question open for any reader interested in taking on that problem. </blockquote> <p> I am interested! </p> <p> Quoting Mark and then Alex. </p> <blockquote> <p> Hillel Wayne [...] outlines an approach where you test an iteration of rho code against a 'last known good' snapshot. He uses <code>git worktree</code> to set up a snapshot of the reference implementation. He then writes a property that compares the refactored code's behaviour against the reference. </p> <p> The example code is in <a href="https://www.python.org">Python</a>, which is a language that I don't know. As far as I can tell, it works because Python is 'lightweight' enough that you can load and execute source code directly. I found that the approach makes much sense, but I wondered how it would apply for statically typed, compiled languages. I decided to create a proof of concept in <a href="https://fsharp.org">F#</a>. </p> </blockquote> <blockquote> I believe shrinking is impossible when the oracle is a static file. However with Hillel's solution the oracle may be consulted at any time, making shrinking viable. </blockquote> <p> I want to start by elaborating on this to make sure we are all on the same page. I think of shrinking as involving two parts. On the one hand, we have the "shrink tree", which contains the values to test during the shrinking process. On the other hand, for each input tested, we need to know if the output should cause the test to pass or fail. </p> <p> With Hedgehog, getting a shrink tree would not be too difficult. For a generator with type parameter <code>'a</code>, the current generator API returns a "random" shrink tree of type <code>'a</code> in which the root is an instance <code>a</code> of the type <code>'a</code> and the tree completely depends on <code>a</code>. It should be easy to expose an additional function that accepts inputs of type <code>'a Gen</code> and <code>'a</code> and returns <em>the</em> tree with the given <code>'a</code> as its root. </p> <p> The difficult part is being able to query the test oracle. As Mark said, this seems easy to do in a dynamically-typed language like Python. In contrast, the fundamental issue with a statically-typed language like F# is that the compiled code exists in an assembly and only one assembly of a given name can be loaded in a given process at the same time. </p> <p> This leads me to two ideas for workarounds. First, we could query the test oracle in a different process. I imagine an entry point could be generated that gives direct access to the test oracle. Then the test process could query the test oracle by executing this generated process. Second, we could generate a different assembly that exposes the test oracle. Then the test process could load this generated assembly to query the test oracle. The second approach seems like it would have a faster query time but be harder to implement. The first approach seems easier to implement but would probably have a slower query time. Maybe the query time would be fast enough though, especially if it was only queried when shrinking. </p> <p> But given such a solution, who wants to restrict access to the test oracle only to shrinking? If the test oracle is always available, then there is no need to store input-output pairs. Instead of always checking that the system under test works correctly for a previously selected set of inputs, the property-based test can check that the system under test has the expected behavior for a unique set of inputs each time the property-based test is executed. In my experience, this is the default behavior of a property-based test. </p> <p> One concern that some people might have is the idea of checking into the code repository the binary containing the test oracle. My first though is that <a href="https://blog.ploeh.dk/2014/01/29/nuget-package-restore-considered-harmful/">the size of this is likely so small that it does not matter</a>. My second thought is that the binary containing the test oracle does not have to be included in the repository. Instead, the workflow could be to (1) create the property-based test that uses the compiled test oracle, (2) refactor the system under test, (3) observe that the property-based test still passes, (4) commit the refactored code, and (5) discard the remaining changes, which will delete the property-based test and the compiled test oracle. </p> <p> Instead of completely removing that property-based test, it might be better to leave it there with input-output pairs stored in a file. Then the conversion from that state of the property-based test to the one that uses the compiled test oracle will be much smaller. </p> </div> <div class="comment-date">2021-01-07 19:27 UTC</div> </div> <div class="comment" id="39b559f49ecc426db7b3d9d1518556df"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Alex, thank you for writing. Yes, I think that calling this a Characterisation Test is correct. I wasn't aware of the <em>ApprovalTests</em> library; thank you for mentioning it. </p> <p> When I originally wrote the article, I was under the impression that shrinking might still be possible. I admit, though, that I hadn't thought things through. I think that <a href="#4aa4188124ca4f4fadf35d6881b9452e">Tyson Williams argues convincingly</a> that this isn't possible. </p> </div> <div class="comment-date">2021-01-15 13:42 UTC</div> </div> <div class="comment" id="a6ea5d7cbb75431ba6adf969599c2bd3"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I'm <a href="/2017/09/18/the-test-data-generator-functor#5bd990290ff048c2a7b55b740053831d">well aware of Hedgehog</a>, and I'm keen on the way it works. I rarely use it, however, as it so far doesn't quite seem to have the same degree of 'industrial strength' to it that FsCheck has. Additionally, I find that shrinking is less important in practice than it might seem in theory. </p> <p> I'm not sure that I understand your confusion about the term <em>dynamic</em>. You write: <blockquote> <p> "<code>A</code> depends on <code>B</code>." </p> </blockquote> Why do you write that? I don't think, in the way you've labelled iterations, that <code>A</code> depends on <code>B</code>. </p> <p> When it comes to shrinking, I think that you convincigly argues that it can't be done unless one is able to query the oracle. As long as all you have is a list of test cases, you can't do that... unless, perhaps, you were to also generate and run all the shrunk test cases when you capture the list of test cases... Again, I haven't thought this through, so there may be some obvious gotcha that I'm missing. </p> <p> I would be wary of trying to host the previous iteration in a different process. This is technically possible, but, in .NET at least, quite cumbersome. You'll have to deal with data marshalling and lifetime management of the second process. It was difficult enough in .NET framework back when <em>remoting</em> was a thing; I'm not even sure how one would go about such a problem in .NET Core - particularly if you want it to work on both Windows, Linux, and Mac. HTTP? </p> </div> <div class="comment-date">2021-01-16 13:24 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An F# demo of validation with partial data round trip https://blog.ploeh.dk/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip 2020-12-28T09:22:00+00:00 Mark Seemann <div id="post"> <p> <em>An F# port of the previous Haskell proof of concept.</em> </p> <p> This article is part of <a href="/2020/12/14/validation-a-solved-problem">a short article series</a> on <a href="/2018/11/05/applicative-validation">applicative validation</a> with a twist. The twist is that validation, when it fails, should return not only a list of error messages; it should also retain that part of the input that <em>was</em> valid. </p> <p> In the <a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">previous article</a> you saw a <a href="https://www.haskell.org">Haskell</a> proof of concept that demonstrated how to compose the appropriate <a href="/2018/10/01/applicative-functors">applicative functor</a> with a suitable <a href="/2017/11/27/semigroups">semigroup</a> to make validation work as desired. In this article, you'll see how to port that proof of concept to <a href="https://fsharp.org">F#</a>. </p> <h3 id="b2ea11fdb65343b5b60fcf2cffc62b1a"> Data definitions <a href="#b2ea11fdb65343b5b60fcf2cffc62b1a" title="permalink">#</a> </h3> <p> Like in the previous article, we're going to need some types. These are essentially direct translations of the corresponding Haskell types: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Input&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string&nbsp;option;&nbsp;DoB&nbsp;:&nbsp;DateTime&nbsp;option;&nbsp;Address&nbsp;:&nbsp;string&nbsp;option} <span style="color:blue;">type</span>&nbsp;ValidInput&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string;&nbsp;DoB&nbsp;:&nbsp;DateTime;&nbsp;Address&nbsp;:&nbsp;string&nbsp;}</pre> </p> <p> The <code>Input</code> type plays the role of the input we'd like to validate, while <code>ValidInput</code> presents validated data. </p> <p> If you're an F# fan, you can bask in the reality that F# records are terser than Haskell records. I like both languages, so I have mixed feelings about this. </p> <h3 id="2e1b689c57114b259b3b4019f4c3976d"> Computation expression <a href="#2e1b689c57114b259b3b4019f4c3976d" title="permalink">#</a> </h3> <p> Haskell's main workhorse is its type class system. F# doesn't have that, but it has <a href="https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/computation-expressions">computation expressions</a>, which in F# 5 got support for applicative functors. That's just what we need, and it turns out that there isn't a lot of code we have to write to make all of this work. </p> <p> To recap from the Haskell proof of concept: We need a <code>Result</code>-like <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> that returns a tuple for errors. One element of the tuple should be a an <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a>, which <a href="/2017/11/13/endomorphism-monoid">forms a monoid</a> (and therefore also a semigroup). The other element should be a list of error messages - <a href="/2017/10/10/strings-lists-and-sequences-as-a-monoid">another monoid</a>. In F# terms we'll write it as <code>(('b -&gt; 'b) * 'c list)</code>. </p> <p> That's a tuple, and since <a href="/2017/10/30/tuple-monoids">tuples form monoids when their elements do</a> the <code>Error</code> part of <code>Result</code> <a href="/2017/11/20/monoids-accumulate">supports accumulation</a>. </p> <p> To support an applicative computation expression, we're going to need a a way to merge two results together. This is by far the most complicated piece of code in this article, all six lines of code: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;Result&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;&#39;a&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;,((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;&nbsp;-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;&#39;d&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;,((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;&nbsp;-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;(&#39;a&nbsp;*&nbsp;&#39;d),((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;merge&nbsp;x&nbsp;y&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x,&nbsp;y&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;xres,&nbsp;Ok&nbsp;yres&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;(xres,&nbsp;yres) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;(f,&nbsp;e1s),&nbsp;Error&nbsp;(g,&nbsp;e2s)&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(f&nbsp;&gt;&gt;&nbsp;g,&nbsp;e2s&nbsp;@&nbsp;e1s) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;e,&nbsp;Ok&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;e &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;_,&nbsp;Error&nbsp;e&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;e</pre> </p> <p> The <code>merge</code> function composes two input results together. The results have <code>Ok</code> types called <code>'a</code> and <code>'d</code>, and if they're both <code>Ok</code> values, the return value is an <code>Ok</code> tuple of <code>'a</code> and <code>'d</code>. </p> <p> If one of the results is an <code>Error</code> value, it beats an <code>Ok</code> value. The only moderately complex operations is when both are <code>Error</code> values. </p> <p> Keep in mind that an <code>Error</code> value in this instance contains a tuple of the type <code>(('b -&gt; 'b) * 'c list)</code>. The first element is an endomorphism <code>'b -&gt; 'b</code> and the other element is a list. The <code>merge</code> function composes the endomorphism <code>f</code> and <code>g</code> by standard function composition (the <code>&gt;&gt;</code> operator), and concatenates the lists with the standard <code>@</code> list concatenation operator. </p> <p> Because I'm emulating how the <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">original forum post</a>'s code behaves, I'm concatenating the two lists with the rightmost going before the leftmost. It doesn't make any other difference than determining the order of the error list. </p> <p> With the <code>merge</code> function in place, the computation expression is a simple matter: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidationBuilder&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;_.BindReturn&nbsp;(x,&nbsp;f)&nbsp;=&nbsp;Result.map&nbsp;f&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;_.MergeSources&nbsp;(x,&nbsp;y)&nbsp;=&nbsp;Result.merge&nbsp;x&nbsp;y</pre> </p> <p> The last piece is a <code>ValidationBuilder</code> value: </p> <p> <pre>[&lt;AutoOpen&gt;] <span style="color:blue;">module</span>&nbsp;ComputationExpressions&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;validation&nbsp;=&nbsp;ValidationBuilder&nbsp;()</pre> </p> <p> Now, whenever you use the <code>validation</code> computation expression, you get the desired functionality. </p> <h3 id="948b93fa6a1d47beac150180769731eb"> Validators <a href="#948b93fa6a1d47beac150180769731eb" title="permalink">#</a> </h3> <p> Before we can compose some validation functions, we'll need to have some validators in place. These are straightforward translations of the Haskell validation functions, starting with the name validator: </p> <p> <pre><span style="color:green;">//&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;string,((Input&nbsp;-&gt;&nbsp;Input)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateName&nbsp;({&nbsp;Name&nbsp;=&nbsp;name&nbsp;}&nbsp;:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;name&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;n&nbsp;<span style="color:blue;">when</span>&nbsp;n.Length&nbsp;&gt;&nbsp;3&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;n &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">fun</span>&nbsp;(args&nbsp;:&nbsp;Input)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;args&nbsp;<span style="color:blue;">with</span>&nbsp;Name&nbsp;=&nbsp;None&nbsp;}), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;no&nbsp;bob&nbsp;and&nbsp;toms&nbsp;allowed&quot;</span>]) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;name&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> When the name is too short, the endomorphism resets the <code>Name</code> field to <code>None</code>. </p> <p> The date-of-birth validation function works the same way: </p> <p> <pre><span style="color:green;">//&nbsp;DateTime&nbsp;-&gt;&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;DateTime,((Input&nbsp;-&gt;&nbsp;Input)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateDoB&nbsp;(now&nbsp;:&nbsp;DateTime)&nbsp;({&nbsp;DoB&nbsp;=&nbsp;dob&nbsp;}&nbsp;:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;dob&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;d&nbsp;<span style="color:blue;">when</span>&nbsp;d&nbsp;&gt;&nbsp;now.AddYears&nbsp;-12&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;d &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">fun</span>&nbsp;(args&nbsp;:&nbsp;Input)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;args&nbsp;<span style="color:blue;">with</span>&nbsp;DoB&nbsp;=&nbsp;None&nbsp;}), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;get&nbsp;off&nbsp;my&nbsp;lawn&quot;</span>]) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;dob&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> Again, like in the Haskell proof of concept, instead of calling <code>DateTime.Now</code> from within the function, I'm passing <code>now</code> as an argument to keep the function <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a>. </p> <p> The address validation concludes the set of validators: </p> <p> <pre><span style="color:green;">//&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;string,((&#39;a&nbsp;-&gt;&nbsp;&#39;a)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateAddress&nbsp;({&nbsp;Address&nbsp;=&nbsp;address&nbsp;}:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;address&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;a &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> The inferred endomorphism type here is the more general <code>'a -&gt; 'a</code>, but it's compatible with <code>Input -&gt; Input</code>. </p> <h3 id="836690a83da24da492f70c8eb756a52d"> Composition <a href="#836690a83da24da492f70c8eb756a52d" title="permalink">#</a> </h3> <p> All three functions have compatible <code>Error</code> types, so they ought to compose with the applicative computation expression to produce the desired behaviour: </p> <p> <pre><span style="color:green;">//&nbsp;DateTime&nbsp;-&gt;&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;ValidInput,(Input&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateInput&nbsp;now&nbsp;args&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;validation&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;name&nbsp;=&nbsp;validateName&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">and!</span>&nbsp;dob&nbsp;=&nbsp;validateDoB&nbsp;now&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">and!</span>&nbsp;address&nbsp;=&nbsp;validateAddress&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;name;&nbsp;DoB&nbsp;=&nbsp;dob;&nbsp;Address&nbsp;=&nbsp;address&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Result.mapError&nbsp;(<span style="color:blue;">fun</span>&nbsp;(f,&nbsp;msgs)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;f&nbsp;args,&nbsp;msgs)</pre> </p> <p> The <code>validation</code> expression alone produces a <code>Result&lt;ValidInput,((Input -&gt; Input) * string list)&gt;</code> value. To get an <code>Input</code> value in the <code>Error</code> tuple, we need to 'run' the <code>Input -&gt; Input</code> endomorphism. The <code>validateInput</code> function does that by applying the endomorphism <code>f</code> to <code>args</code> when mapping the error with <code>Result.mapError</code>. </p> <h3 id="df2c48e822ad48ce916e29da8638563a"> Tests <a href="#df2c48e822ad48ce916e29da8638563a" title="permalink">#</a> </h3> <p> To test that the <code>validateInput</code> works as intended, I first copied all the code from the original forum post. I then wrote eight <a href="https://en.wikipedia.org/wiki/Characterization_test">characterisation tests</a> against that code to make sure that I could reproduce the desired functionality. </p> <p> I then wrote a parametrised test against the new function: </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;ValidationTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Validation&nbsp;works``&nbsp;input&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;now&nbsp;=&nbsp;DateTime.Now &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;validateInput&nbsp;now&nbsp;input &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> The <code>ValidationTestCases</code> class is defined like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Input,&nbsp;Result&lt;ValidInput,&nbsp;Input&nbsp;*&nbsp;string&nbsp;list&gt;&gt;&nbsp;()</pre> </p> <p> This class produces a set of test cases, where each test case contains an <code>input</code> value and the <code>expected</code> output. To define the test cases, I copied the eight characterisation tests I'd already produced and adjusted them so that they fit the simpler API of the <code>validateInput</code> function. Here's a few examples: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;eightYearsAgo&nbsp;=&nbsp;DateTime.Now.AddYears&nbsp;-8 <span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;None&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;None&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>])) <span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Ok&nbsp;({&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}))</pre> </p> <p> The first case expects an <code>Error</code> value because the <code>Input</code> value has no address. The other test case expects an <code>Ok</code> value because all input is fine. </p> <p> I copied all eight characterisation tests over, so now I have those eight tests, as well as the modified eight tests for the applicative-based API shown here. All sixteen tests pass. </p> <h3 id="5a20682ce9494011b78d5a5f9c2c9bad"> Conclusion <a href="#5a20682ce9494011b78d5a5f9c2c9bad" title="permalink">#</a> </h3> <p> I find this solution to the problem elegant. It's always satisfying when you can implement what at first glance looks like custom behaviour using <a href="/2017/10/04/from-design-patterns-to-category-theory">universal abstractions</a>. </p> <p> Besides the aesthetic value, I also believe that this keeps a team more productive. These concepts of monoids, semigroups, applicative functors, and so on, are concepts that you only have to learn once. Once you know them, you'll recognise them when you run into them. This means that there's less code to understand. </p> <p> An ad-hoc implementation as the original forum post suggested (even though it looked quite decent) always puts the onus on a maintenance developer to read and understand even more one-off infrastructure code. </p> <p> With an architecture based on universal abstractions and well-documented language features, a functional programmer that knows these things will be able to pick up what's going on without much trouble. Specifically, (s)he will recognise that this is 'just' applicative validation with a twist. </p> <p> This article is the December 28 entry in the <a href="https://sergeytihon.com/2020/10/22/f-advent-calendar-in-english-2020">F# Advent Calendar in English 2020</a>. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. A Haskell proof of concept of validation with partial data round trip https://blog.ploeh.dk/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip 2020-12-21T06:54:00+00:00 Mark Seemann <div id="post"> <p> <em>Which Semigroup best addresses the twist in the previous article?</em> </p> <p> This article is part of <a href="/2020/12/14/validation-a-solved-problem">a short article series</a> on applicative validation with a twist. The twist is that validation, when it fails, should return not only a list of error messages; it should also retain that part of the input that <em>was</em> valid. </p> <p> In this article, I'll show how I did a quick proof of concept in <a href="https://www.haskell.org">Haskell</a>. </p> <h3 id="9417153ff45d4188a65470fa2d67ea2e"> Data definitions <a href="#9417153ff45d4188a65470fa2d67ea2e" title="permalink">#</a> </h3> <p> You can't use the regular <code>Either</code> instance of <code>Applicative</code> for validation because it short-circuits on the first error. In other words, you can't collect multiple error messages, even if the input has multiple issues. Instead, you need a custom <code>Applicative</code> instance. You can <a href="/2018/11/05/applicative-validation">easily write such an instance</a> yourself, but there are a couple of libraries that already do this. For this prototype, I chose the <a href="https://hackage.haskell.org/package/validation">validation</a> package. </p> <p> <pre><span style="color:blue;">import</span>&nbsp;Data.Bifunctor <span style="color:blue;">import</span>&nbsp;Data.Time <span style="color:blue;">import</span>&nbsp;Data.Semigroup <span style="color:blue;">import</span>&nbsp;Data.Validation </pre> </p> <p> Apart from importing <code>Data.Validation</code>, I also need a few other imports for the proof of concept. All of them are well-known. I used no language extensions. </p> <p> For the proof of concept, the input is a triple of a name, a date of birth, and an address: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;Input&nbsp;=&nbsp;Input&nbsp;{ &nbsp;&nbsp;<span style="color:#2b91af;">inputName</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:#2b91af;">String</span>, &nbsp;&nbsp;<span style="color:#2b91af;">inputDoB</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:blue;">Day</span>, &nbsp;&nbsp;<span style="color:#2b91af;">inputAddress</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:#2b91af;">String</span>&nbsp;} &nbsp;&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) </pre> </p> <p> The goal is actually to <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">parse (not validate)</a> <code>Input</code> into a safer data type: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;ValidInput&nbsp;=&nbsp;ValidInput&nbsp;{ &nbsp;&nbsp;<span style="color:#2b91af;">validName</span>&nbsp;::&nbsp;<span style="color:#2b91af;">String</span>, &nbsp;&nbsp;<span style="color:#2b91af;">validDoB</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>, &nbsp;&nbsp;<span style="color:#2b91af;">validAddress</span>&nbsp;::&nbsp;<span style="color:#2b91af;">String</span>&nbsp;} &nbsp;&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) </pre> </p> <p> If parsing/validation fails, the output should report a collection of error messages <em>and</em> return the <code>Input</code> value with any valid data retained. </p> <h3 id="4ce26b2031084212a5fbab83dd848d4b"> Looking for a Semigroup <a href="#4ce26b2031084212a5fbab83dd848d4b" title="permalink">#</a> </h3> <p> My hypothesis was that validation, even with that twist, can be implemented elegantly with an <code>Applicative</code> instance. The <em>validation</em> package defines its <code>Validation</code> data type such that it's an <code>Applicative</code> instance as long as its error type is a <code>Semigroup</code> instance: </p> <p> <pre><span style="color:blue;">Semigroup</span> err =&gt; <span style="color:blue;">Applicative</span> (<span style="color:blue;">Validation</span> err)</pre> </p> <p> The question is: which <code>Semigroup</code> can we use? </p> <p> Since we need to return <em>both</em> a list of error messages <em>and</em> a modified <code>Input</code> value, it sounds like we'll need a product type of some sorts. A tuple will do; something like <code>(Input, [String])</code>. Is that a <code>Semigroup</code> instance, though? </p> <p> Tuples only form semigroups if both elements give rise to a semigroup: </p> <p> <pre>(<span style="color:blue;">Semigroup</span> a, <span style="color:blue;">Semigroup</span> b) =&gt; <span style="color:blue;">Semigroup</span> (a, b)</pre> </p> <p> The second element of my candidate is <code>[String]</code>, which is fine. Lists are <code>Semigroup</code> instances. But what about <code>Input</code>? Can we somehow combine two <code>Input</code> values into one? It's not entirely clear how we should do that, so that doesn't seem too promising. </p> <p> What we need to do, however, is to take the original <code>Input</code> and modify it by (optionally) resetting one or more fields. In other words, a series of functions of the type <code>Input -&gt; Input</code>. Aha! There's the semigroup we need: <a href="https://hackage.haskell.org/package/base/docs/Data-Semigroup.html#t:Endo"><code>Endo Input</code></a>. </p> <p> So the <code>Semigroup</code> instance we need is <code>(<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code>, and the validation output should be of the type <code><span style="color:blue;">Validation</span> (<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>]) a</code>. </p> <h3 id="5da9d89ac8414ad0bc9ebe322b831390"> Validators <a href="#5da9d89ac8414ad0bc9ebe322b831390" title="permalink">#</a> </h3> <p> Cool, we can now implement the validation logic; a function for each field, starting with the name: </p> <p> <pre><span style="color:#2b91af;">validateName</span>&nbsp;::&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span> validateName&nbsp;(Input&nbsp;(Just&nbsp;name)&nbsp;_&nbsp;_)&nbsp;|&nbsp;<span style="color:blue;">length</span>&nbsp;name&nbsp;&gt;&nbsp;3&nbsp;=&nbsp;Success&nbsp;name validateName&nbsp;(Input&nbsp;(Just&nbsp;_)&nbsp;_&nbsp;_)&nbsp;= &nbsp;&nbsp;Failure&nbsp;(Endo&nbsp;$&nbsp;\x&nbsp;-&gt;&nbsp;x&nbsp;{&nbsp;inputName&nbsp;=&nbsp;Nothing&nbsp;},&nbsp;[<span style="color:#a31515;">&quot;no&nbsp;bob&nbsp;and&nbsp;toms&nbsp;allowed&quot;</span>]) validateName&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;name&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> This function reproduces the validation logic implied by <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">the forum question that started it all</a>. Notice, particularly, that when the name is too short, the endomorphism resets <code>inputName</code> to <code>Nothing</code>. </p> <p> The date-of-birth validation function works the same way: </p> <p> <pre><span style="color:#2b91af;">validateDoB</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:blue;">Day</span> validateDoB&nbsp;now&nbsp;(Input&nbsp;_&nbsp;(Just&nbsp;dob)&nbsp;_)&nbsp;|&nbsp;addGregorianYearsRollOver&nbsp;(-12)&nbsp;now&nbsp;&lt;&nbsp;dob&nbsp;= &nbsp;&nbsp;Success&nbsp;dob validateDoB&nbsp;_&nbsp;(Input&nbsp;_&nbsp;(Just&nbsp;_)&nbsp;_)&nbsp;= &nbsp;&nbsp;Failure&nbsp;(Endo&nbsp;$&nbsp;\x&nbsp;-&gt;&nbsp;x&nbsp;{&nbsp;inputDoB&nbsp;=&nbsp;Nothing&nbsp;},&nbsp;[<span style="color:#a31515;">&quot;get&nbsp;off&nbsp;my&nbsp;lawn&quot;</span>]) validateDoB&nbsp;_&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;dob&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> Again, the validation logic is inferred from the forum question, although I found it better keep the function pure by requiring a <code>now</code> argument. </p> <p> The address validation is the simplest of the three validators: </p> <p> <pre><span style="color:#2b91af;">validateAddress</span>&nbsp;::&nbsp;<span style="color:blue;">Monoid</span>&nbsp;a&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(a,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span> validateAddress&nbsp;(Input&nbsp;_&nbsp;_&nbsp;(Just&nbsp;a))&nbsp;=&nbsp;Success&nbsp;a validateAddress&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> This one's return type is actually more general than required, since I used <code>mempty</code> instead of <code>Endo id</code>. This means that it actually works for any <code>Monoid a</code>, which also includes <code>Endo Input</code>. </p> <h3 id="6d5502d178f143d58a4d3c5bef7c1f05"> Composition <a href="#6d5502d178f143d58a4d3c5bef7c1f05" title="permalink">#</a> </h3> <p> All three functions return <code><span style="color:blue;">Validation</span> (<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code>, which has an <code>Applicative</code> instance. This means that we should be able to compose them together to get the behaviour we're looking for: </p> <p> <pre><span style="color:#2b91af;">validateInput</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#2b91af;">Either</span>&nbsp;(<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:blue;">ValidInput</span> validateInput&nbsp;now&nbsp;args&nbsp;= &nbsp;&nbsp;toEither&nbsp;$ &nbsp;&nbsp;first&nbsp;(first&nbsp;(`appEndo`&nbsp;args))&nbsp;$ &nbsp;&nbsp;ValidInput&nbsp;&lt;$&gt;&nbsp;validateName&nbsp;args&nbsp;&lt;*&gt;&nbsp;validateDoB&nbsp;now&nbsp;args&nbsp;&lt;*&gt;&nbsp;validateAddress&nbsp;args </pre> </p> <p> That compiles, so it probably works. </p> <h3 id="9ed5a5fe379244a3bd1e9206a79a1ea9"> Sanity check <a href="#9ed5a5fe379244a3bd1e9206a79a1ea9" title="permalink">#</a> </h3> <p> Still, it'd be prudent to check. Since this is only a proof of concept, I'm not going to set up a test suite. Instead, I'll just start GHCi for some ad-hoc testing: </p> <p> <pre>λ&gt; now &lt;- localDay &lt;&amp;&gt; zonedTimeToLocalTime &lt;&amp;&gt; getZonedTime λ&gt; validateInput now &amp; Input Nothing Nothing Nothing Left (Input {inputName = Nothing, inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["name is required","dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Bob") Nothing Nothing Left (Input {inputName = Nothing, inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["no bob and toms allowed","dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") Nothing Nothing Left (Input {inputName = Just "Alice", inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2002 10 12) Nothing Left (Input {inputName = Just "Alice", inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["get off my lawn","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2012 4 21) Nothing Left (Input {inputName = Just "Alice", inputDoB = Just 2012-04-21, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2012 4 21) (Just "x") Right (ValidInput {validName = "Alice", validDoB = 2012-04-21, validAddress = "x"})</pre> </p> <p> In order to make the output more readable, I've manually edited the GHCi session by adding line breaks to the output. </p> <p> It looks like it's working like it's supposed to. Only the last line successfully parses the input and returns a <code>Right</code> value. </p> <h3 id="8bbb1d8ca355495b95e5e5ed85a924f4"> Conclusion <a href="#8bbb1d8ca355495b95e5e5ed85a924f4" title="permalink">#</a> </h3> <p> Before I started this proof of concept, I had an inkling of the way this would go. Instead of making the prototype in <a href="https://fsharp.org">F#</a>, I found it more productive to do it in Haskell, since Haskell enables me to compose things together. I particularly appreciate how a composition of types like <code>(<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code> is automatically a <code>Semigroup</code> instance. I don't have to do anything. That makes the language great for prototyping things like this. </p> <p> Now that I've found the appropriate semigroup, I know how to convert the code to F#. That's in the next article. </p> <p> <strong>Next:</strong> <a href="/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip">An F# demo of validation with partial data round trip</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="0ebea2f4d9c54072a5bb0c093a63fe14"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Great work and excellent post. I just had a few clarification quesitons. </p> <blockquote> <p> ...But what about <code>Input</code>? Can we somehow combine two <code>Input</code> values into one? It's not entirely clear how we should do that, so that doesn't seem too promising. </p> <p> What we need to do, however, is to take the original <code>Input</code> and modify it by (optionally) resetting one or more fields. In other words, a series of functions of the type <code>Input -&gt; Input</code>. Aha! There's the semigroup we need: <code>Endo Input</code>. </p> </blockquote> <p> How rhetorical are those questions? Whatever the case, I will take the bait. </p> <p> Any product type forms a semigroup if all of its elements do. You explicitly stated this for tuples of length 2; it also holds for records such as <code>Input</code>. Each field on that record has type <code> Maybe a</code> for some <code>a</code>, so it suffices to select a semigroup involving <code>Maybe a</code>. There are few different semigropus involving <code>Maybe</code> that have different functions. </p> <p> I think the most common semigroup for <code>Maybe a</code> has the function that returns the first <code>Just _</code> if one exists or else returns <code>Nothing</code>. Combining that with <code>Nothing</code> as the identity element gives the monoid that is typically associated with <code>Maybe a</code> (and I know by the name monoidal plus). Another monoid, and therefore a semigroup, is to return the last <code>Just _</code> instead of the first. </p> <p> Instead of the having a preference for <code>Just _</code>, the function could have a preference for <code>Nothing</code>. As before, when both inputs are <code>Just _</code>, the output could be either of the inputs. </p> <p> I think either of those last two semigroups will achieved the desired behavior in the problem at hand. Your code never replaces an instace of <code>Just a</code> with a different instance, so we don't need a preference for some input when they are both <code>Just _</code>. </p> <p> In the end though, I think the semigroup you derived from <code>Endo</code> leads to simpler code. </p> <p> At the end of the type signature for <code>validateName</code> / <code>validateDoB</code> / <code>validateAddress</code>, what does <code>String</code> / <code>Day</code> / <code>String</code> mean? </p> <p> Why did you pass all three arguments into every parsing/validation function? I think it is a bit simpler to only pass in the needed argument. Maybe you thought this was good enough for prototype code. </p> <p> Why did you use <code>add1</code> in your error message instead of <code>address</code>? Was it only for prototype code to make the message a bit shorter? </p> </div> <div class="comment-date">2020-12-21 14:21 UTC</div> </div> <div class="comment" id="510b9be50c1b43c18973008b89d2da38"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. The semigroup you suggest, I take it, would look something like this: </p> <p> <pre><span style="color:blue;">newtype</span>&nbsp;Perhaps&nbsp;a&nbsp;=&nbsp;Perhaps&nbsp;{&nbsp;runPerhaps&nbsp;::&nbsp;Maybe&nbsp;&nbsp;a&nbsp;}&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) <span style="color:blue;">instance</span>&nbsp;<span style="color:blue;">Semigroup</span>&nbsp;(<span style="color:blue;">Perhaps</span>&nbsp;a)&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;Perhaps&nbsp;Nothing&nbsp;&lt;&gt;&nbsp;_&nbsp;=&nbsp;Perhaps&nbsp;Nothing &nbsp;&nbsp;_&nbsp;&lt;&gt;&nbsp;Perhaps&nbsp;Nothing&nbsp;=&nbsp;Perhaps&nbsp;Nothing &nbsp;&nbsp;Perhaps&nbsp;(Just&nbsp;x)&nbsp;&lt;&gt;&nbsp;_&nbsp;=&nbsp;Perhaps&nbsp;(Just&nbsp;x)</pre> </p> <p> That might work, but it's an atypical semigroup. I <em>think</em> that it's lawful - at least, I can't come up with a counterexample against associativity. It seems reminiscent of Boolean <em>and</em> (the <em>All</em> monoid), but it isn't a monoid, as far as I can tell. </p> <p> Granted, a <code>Monoid</code> constraint isn't required to make the validation code work, but following the <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">principle of least surprise</a>, I still think that picking a well-known semigroup such as <code>Endo</code> is preferable. </p> <p> Regarding your second question, the type signature of e.g. <code>validateName</code> is: </p> <p> <pre><span style="color:#2b91af;">validateName</span>&nbsp;::&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span></pre> </p> <p> Like <code>Either</code>, <code>Validation</code> has two type arguments: <code>err</code> and <code>a</code>; it's defined as <code>data Validation err a</code>. In the above function type, the return value is a <code>Validation</code> value where the <code>err</code> type is <code>(Endo Input, [String])</code> and <code>a</code> is <code>String</code>. </p> <p> All three validation functions share a common <code>err</code> type: <code>(Endo Input, [String])</code>. On the other hand, they return various <code>a</code> types: <code>String</code>, <code>Day</code>, and <code>String</code>, respectively. </p> <p> Regarding your third question, I could also have defined the functions so that they would only have taken the values they'd need to validate. That would better fit <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>, so I should probably have done that... </p> <p> As for the last question, I was just following the 'spec' implied by <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">the original forum question</a>. </p> </div> <div class="comment-date">2020-12-22 15:05 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Validation, a solved problem? https://blog.ploeh.dk/2020/12/14/validation-a-solved-problem 2020-12-14T08:28:00+00:00 Mark Seemann <div id="post"> <p> <em>A validation problem with a twist.</em> </p> <p> Until recently, I thought that data validation was a solved problem: <a href="/2018/11/05/applicative-validation">Use an applicative functor</a>. I then encountered <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">a forum question</a> that for a few minutes shook my faith. </p> <p> After brief consideration, though, I realised that all is good. Validation, even with a twist, is successfully modelled with an <a href="/2018/10/01/applicative-functors">applicative functor</a>. Faith in computer science restored. </p> <h3 id="891801e3802f49d8a51b56bebaacecb9"> The twist <a href="#891801e3802f49d8a51b56bebaacecb9" title="permalink">#</a> </h3> <p> Usually, when you see a demo of applicative validation, the result of validating is one of two: either <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">a parsed result</a>, or a collection of error messages. </p> <p> <pre>λ&gt; validateReservation $ ReservationJson "2017-06-30 19:00:00+02:00" 4 "Jane Doe" "j@example.com" Validation (Right (Reservation { &nbsp;&nbsp;&nbsp;&nbsp;reservationDate = 2017-06-30 19:00:00 +0200, &nbsp;&nbsp;&nbsp;&nbsp;reservationQuantity = 4, &nbsp;&nbsp;&nbsp;&nbsp;reservationName = "Jane Doe", &nbsp;&nbsp;&nbsp;&nbsp;reservationEmail = "j@example.com"})) λ&gt; validateReservation $ ReservationJson "2017/14/12 6pm" 4.1 "Jane Doe" "jane.example.com" Validation (Left ["Not a date.","Not a positive integer.","Not an email address."]) λ&gt; validateReservation $ ReservationJson "2017-06-30 19:00:00+02:00" (-3) "Jane Doe" "j@example.com" Validation (Left ["Not a positive integer."])</pre> </p> <p> (Example from <a href="/2018/11/05/applicative-validation">Applicative validation</a>.) </p> <p> What if, instead, you're displaying an input form? When users enter data, you want to validate it. Imagine, for the rest of this short series of articles that the input form has three fields: <em>name</em>, <em>date of birth</em>, and <em>address</em>. Each piece of data has associated validation rules. </p> <p> If you enter a valid name, but an invalid date of birth, you want to clear the input form's date of birth, but not the name. It's such a bother for a user having to retype valid data just because a single field turned out to be invalid. </p> <p> Imagine, for example, that you want to bind the form to a data model like this <a href="https://fsharp.org">F#</a> record type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Input&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string&nbsp;option;&nbsp;DoB&nbsp;:&nbsp;DateTime&nbsp;option;&nbsp;Address&nbsp;:&nbsp;string&nbsp;option}</pre> </p> <p> Each of these three fields is optional. We'd like validation to work in the following way: If validation fails, the function should return <em>both</em> a list of error messages, and <em>also</em> the <code>Input</code> object, with valid data retained, but invalid data cleared. </p> <p> One of the rules implied in the forum question is that names must be more than three characters long. Thus, input like this is invalid: </p> <p> <pre>{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Tom&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}</pre> </p> <p> Both the <code>DoB</code> and <code>Address</code> fields, however, are valid, so, along with error messages, we'd like our validation function to return a partially wiped <code>Input</code> value: </p> <p> <pre>{&nbsp;Name&nbsp;=&nbsp;None;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}</pre> </p> <p> Notice that both <code>DoB</code> and <code>Address</code> field values are retained, while <code>Name</code> has been reset. </p> <p> A final requirement: If validation succeeds, the return value should be a <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">parsed value that captures that validation took place</a>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidInput&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string;&nbsp;DoB&nbsp;:&nbsp;DateTime;&nbsp;Address&nbsp;:&nbsp;string&nbsp;}</pre> </p> <p> That requirement is straightforward. That's how you'd usually implement application validation. It's the partial data round-trip that seems to throw a spanner in the works. </p> <p> How should we model such validation? </p> <h3 id="534fcc2d66f242a0ba10a9aca7827276"> Theory, applied <a href="#534fcc2d66f242a0ba10a9aca7827276" title="permalink">#</a> </h3> <p> There's a subculture of functional programming that draws heavily on <a href="https://en.wikipedia.org/wiki/Category_theory">category theory</a>. This is most prevalent in <a href="https://www.haskell.org">Haskell</a>. I've been studying category theory in an attempt to understand what it's all about. I even wrote <a href="/2017/10/04/from-design-patterns-to-category-theory">a substantial article series</a> about some design patterns and how they relate to theory. </p> <p> One thing I learned <em>after</em> I'd named that article series is that most of the useful theoretical concepts come from <a href="https://en.wikipedia.org/wiki/Abstract_algebra">abstract algebra</a>, with the possible exception of monads. </p> <p> People often ask me: does all that theory have any practical use? </p> <p> Yes, it does, as it turns out. It did, for example, enable me to identify a solution to the above twist in five to ten minutes. </p> <p> It's a discussion that I often have, particularly with the always friendly F# community. <em>Do you have to understand <a href="/2018/03/22/functors">functors</a>, monads, etcetera to be a productive F# developer?</em> </p> <p> To anyone who wants to learn F# I'd respond: Don't worry about that at the gate. Find a good learning resource and dive right in. It's a friendly language that you can learn gradually. </p> <p> Sooner or later, though, you'll run into knotty problems that you may struggle to address. I've seen this enough times that it looks like a pattern. The present forum question is just one example. A beginner or intermediate F# programmer will typically attempt to solve the problem in an ad-hoc manner that may or may not be easy to maintain. (The solution proposed by the author of that forum question doesn't, by the way, look half bad.) </p> <p> To be clear: there's nothing wrong with being a beginner. I was once a beginner programmer, and I'm <em>still</em> a beginner in multiple ways. What I'm trying to argue here is that there <em>is</em> value in knowing theory. With my knowledge of abstract algebra and how it applies to functional programming, it didn't take me long to identify a solution. I'll get to that later. </p> <p> Before I outline a solution, I'd like to round off the discussion of applied theory. That question about monads comes up a lot. <em>Do I have to understand functors, monads, etcetera to be a good F# developer?</em> </p> <p> I think it's like asking <em>Do I have to understand polymorphism, design patterns, the <a href="https://en.wikipedia.org/wiki/SOLID">SOLID principles</a>, etcetera to be a good object-oriented programmer?</em> </p> <p> Those are typically not the first topics people are taught about OOD. I would assert, however, that understanding such topics do help. They may not be required to get started with OOP, but knowing them makes you a better programmer. </p> <p> I think the same is true for functional programming. It's just a different skill set that makes you better in that paradigm. </p> <h3 id="78fdc446770a4ec48b556e0826a59ce9"> Solution outline <a href="#78fdc446770a4ec48b556e0826a59ce9" title="permalink">#</a> </h3> <p> When you know a bit of theory, you may know that validation can be implemented with an applicative sum type like <a href="/2019/01/14/an-either-functor">Either</a> (AKA <em>Option</em>), with one extra requirement. </p> <p> Either <a href="/2019/01/07/either-bifunctor">has two dimensions</a>, <em>left</em> or <em>right</em> (<em>success</em> or <em>failure</em>, <em>ok</em> or <em>error</em>, etcetera). The applicative nature of it already supplies a way to compose the successes, but what if there's more than one validation error? </p> <p> In my <a href="/2018/11/05/applicative-validation">article about applicative validation</a> I showed how to collect multiple error messages in a list. Lists, however, <a href="/2017/10/10/strings-lists-and-sequences-as-a-monoid">form a monoid</a>, so I typed the validation API to be that flexible. </p> <p> In fact, all you need is a <a href="/2017/11/27/semigroups">semigroup</a>. When I wrote the article on applicative validation, Haskell's <code>Semigroup</code> type class wasn't yet a supertype of <code>Monoid</code>, and I (perhaps without sufficient contemplation) just went with <code>Monoid</code>. </p> <p> What remains is that applicative validation can collect errors for <em>any</em> semigroup of errors. All we need to solve the above validation problem with a twist, then, is to identify a suitable semigroup. </p> <p> I don't want to give away everything in this article, so I'm going to leave you with this cliffhanger. Which semigroup solves the problem? Read on. <ul> <li><a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">A Haskell proof of concept of validation with partial data round trip</a></li> <li><a href="/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip">An F# demo of validation with partial data round trip</a></li> </ul> As is often my modus operandi, I first did a proof of concept in Haskell. With its type classes and higher-kinded polymorphism, it's much faster to prototype solutions than even in F#. In the next article, I'll describe how that turned out. </p> <p> After the Haskell article, I'll show how it translates to F#. You can skip the Haskell article if you like. </p> <h3 id="048492d078164a648dddf6c57dbaf490"> Conclusion <a href="#048492d078164a648dddf6c57dbaf490" title="permalink">#</a> </h3> <p> I still think that validation is a solved problem. It's always interesting when such a belief for a moment is challenged, and satisfying to discover that it still holds. </p> <p> This is, after all, not proof of anything. Perhaps tomorrow, someone will throw another curve ball that I can't catch. If that happens, I'll have to update my beliefs. Until then, I'll consider validation a solved problem. </p> <p> <strong>Next:</strong> <a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">A Haskell proof of concept of validation with partial data round trip</a>. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Branching tests https://blog.ploeh.dk/2020/12/07/branching-tests 2020-12-07T06:25:00+00:00 Mark Seemann <div id="post"> <p> <em>Is it ever okay to branch and loop in a unit test?</em> </p> <p> When I coach development organisations about unit testing and test-driven development, there's often a sizeable group of developers who don't see the value of unit testing. Some of the arguments they typically use are worth considering. </p> <p> A common complaint is that it's difficult to see the wisdom in writing code to prevent defects in code. That's not an unreasonable objection. </p> <p> <a href="/2020/05/25/wheres-the-science">We have scant scientific knowledge about software engineering</a>, but the little we know suggests that the number of defects is proportional to lines of code. The more lines of code, the more defects. </p> <p> If that's true, adding more code - even when it's test code - seems like a bad idea. </p> <h3 id="88c3fa7503454d9f9e329db299f70881"> Reasons to trust test code <a href="#88c3fa7503454d9f9e329db299f70881" title="permalink">#</a> </h3> <p> First, we should consider the possibility that the correlation between lines of code and defects doesn't mean that defects are <em>evenly</em> distributed. As <a href="https://www.adamtornhill.com">Adam Tornhill</a> argues in <a href="https://amzn.to/36Pd5EE">Your Code as a Crime Scene</a>, defects tend to cluster in hotspots. </p> <p> You can have a large proportion of your code base which is, for all intents and purpose, bug-free, and hotspots where defects keep spawning. </p> <p> If this is true, adding test code isn't a problem if you can keep it bug-free. </p> <p> That, however, sounds like a chicken-and-the-egg kind of problem. How can you know that test code is bug-free without tests? </p> <p> I've <a href="/2013/04/02/why-trust-tests">previously answered that question</a>. In short, you can trust a test for two reasons: <ul> <li>You've seen it fail (haven't you?)</li> <li>It's simple</li> </ul> I usually think of the simplicity criterion as a limit on <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a>: it should be <em>1</em>. This means no branching and no loops in your tests. </p> <p> That's what this article is actually about. </p> <h3 id="0494e1e3524d46b186ad7fc039dc7c17"> What's in a name? <a href="#0494e1e3524d46b186ad7fc039dc7c17" title="permalink">#</a> </h3> <p> I was working with an online restaurant reservation system (example code), and had written this test: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> This is a <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based test</a> that verifies that a valid reservation makes it to the database. The test has a cyclomatic complexity of <em>1</em>, and I've seen it fail, so all is good. (It may, in fact, contain a future maintenance problem, but that's a topic for <a href="/2021/01/11/waiting-to-happen">another article</a>.) </p> <p> What constitutes a valid reservation? At the very least, we should demand that <code>At</code> is a valid date and time, and that <code>Quantity</code> is a positive number. The restaurant would like to be able to email a confirmation to the user, so an email address is also required. Email addresses are notoriously difficult to validate, so we'll just require that the the string isn't null. </p> <p> What about the <code>Name</code>? I thought about this a bit and decided that, according to <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>, the system should accept null names. The name is only a convenience; the system doesn't need it, it's just there so that when you arrive at the restaurant, you can say <em>"I have a reservation for Julia"</em> instead of giving an email address to the <a href="https://en.wikipedia.org/wiki/Ma%C3%AEtre_d%27h%C3%B4tel">maître d'hôtel</a>. But then, if you didn't supply a name when you made the reservation, you can always state your email address when you arrive. To summarise, the name is just a convenience, not a requirement. </p> <p> This decision meant that I ought to write a test case with a null name. </p> <p> That turned out to present a problem. I'd defined the <code>Reservation</code> class so that it didn't accept <code>null</code> arguments, and I think that's the appropriate design. Null is just evil and has no place in my domain models. </p> <p> That's not a problem in itself. In this case, I think it's acceptable to convert a null name to the empty string. </p> <h3 id="f0caaae1c01d47378b0f23a8d6a95d72"> Copy and paste <a href="#f0caaae1c01d47378b0f23a8d6a95d72" title="permalink">#</a> </h3> <p> Allow me to summarise. If you consider the above unit test, I needed a third test case with a null <code>name</code>. In that case, <code>expected</code> should be a <code>Reservation</code> value with the name <code>""</code>. Not <code>null</code>, but <code>""</code>. </p> <p> As far as I can tell, you can't easily express that in <code>PostValidReservationWhenDatabaseIsEmpty</code> without increasing its cyclomatic complexity. Based on the above introduction, that seems like a no-no. </p> <p> What's the alternative? Should I copy the test and adjust the <em>single</em> line of code that differs? If I did, it would look like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithNullNameWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Apart from the values in the <code>[InlineData]</code> attribute and the method name, the <em>only</em> difference from <code>PostValidReservationWhenDatabaseIsEmpty</code> is that <code>expected</code> has a hard-coded name of <code>""</code>. </p> <p> This is not acceptable. </p> <p> There's a common misconception that the <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> principle doesn't apply to unit tests. I don't see why this should be true. The DRY principle exists because copy-and-paste code is difficult to maintain. Unit test code is also code that you have to maintain. All the rules about writing maintainable code also apply to unit test code. </p> <h3 id="1d10bd69a6364e6984cb808ab5d9a8f8"> Branching in test <a href="#1d10bd69a6364e6984cb808ab5d9a8f8" title="permalink">#</a> </h3> <p> What's the alternative? One option (that shouldn't be easily dismissed) is to introduce a <a href="http://xunitpatterns.com/Test%20Helper.html">Test Helper</a> to perform the conversion from a nullable name to a non-nullable name. Such a helper would have a cyclomatic complexity of <em>2</em>, but could be unit tested in isolation. It might even turn out that it'd be useful in the production code. </p> <p> Still, that seems like overkill, so I instead made the taboo move and added branching logic to the existing test to see how it'd look: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Notice that the <code>expected</code> name is now computed as <code>dto.Name ?? ""</code>. Perhaps you think about branching instructions as relating exclusively to keywords such as <code>if</code> or <code>switch</code>, but the <code>??</code> operator is also a branching instruction. The test now has a cyclomatic complexity of <code>2</code>. </p> <p> Is that okay? </p> <h3 id="baf81334614241e7b36bbe7f8f1af821"> To branch or not to branch <a href="#baf81334614241e7b36bbe7f8f1af821" title="permalink">#</a> </h3> <p> I think that in this case, it's okay to slightly increase the cyclomatic complexity of the test. It's not something I just pull out of my hat, though. I think it's possible to adjust the above heuristics to embrace this sort of variation. </p> <p> To be clear, I consider this an <em>advanced</em> practice. If you're just getting started with unit testing, try to keep tests simple. Keep the cyclomatic complexity at <em>1</em>. </p> <p> Had I been in the above situation a couple of years ago, I might not have considered this option. About a year ago, though, I watched <a href="https://en.wikipedia.org/wiki/John_Hughes_(computer_scientist)">John Hughes'</a> presentation <a href="https://youtu.be/NcJOiQlzlXQ">Building on developers' intuitions to create effective property-based tests</a>. When he, about 15 minutes in, wrote a test with a branching instruction, I remember becoming quite uncomfortable. This lasted for a while until I understood where he was going with it. It's truly an inspiring and illuminating talk; I highly recommend it. </p> <p> How it relates to the problem presented here is through <em>coverage</em>. While the <code>PostValidReservationWhenDatabaseIsEmpty</code> test now has a cyclomatic complexity of <em>2</em>, it's a parametrised test with three test cases. Two of these cover one branch, and the third covers the other. </p> <p> What's more important is the process that produced the test. I added one test case at a time, and for each case, <em>I saw the test fail</em>. </p> <p> Specifically, when I added the third test case with the null name, I first added the branching expression <code>dto.Name ?? ""</code> and ran the two existing tests. They still both passed, which bolstered my belief that they both exercised the left branch of that expression. I then added the third case and saw that it (and only it) failed. This supported my belief that the third case exercised the right branch of <code>??</code>. </p> <p> Branching in unit tests isn't something I do lightly. I still believe that it could make the test more vulnerable to future changes. I'm particularly worried about making a future change that might shift one or more of these test cases into false negatives in the form of <a href="/2019/10/14/tautological-assertion">tautological assertions</a>. </p> <h3 id="bfa3a3d3f82044f289568efcc675c70f"> Conclusion <a href="#bfa3a3d3f82044f289568efcc675c70f" title="permalink">#</a> </h3> <p> As you can tell, when I feel that I'm moving onto thin ice, I move deliberately. If there's one thing I've learned from decades of professional programming it's that my brain loves jumping to conclusions. Moving slowly and deliberately is my attempt at countering this tendency. I believe that it enables me to go faster in the long run. </p> <p> I don't think that branching in unit tests should be common, but I believe that it may be occasionally valid. The key, I think, is to guarantee that each branch in the test is covered by a test case. The implication is that there must be <em>at least</em> as many test cases as the cyclomatic complexity. In other words, the test <em>must</em> be a parametrised test. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="cbbf719d6e1744c9879edb6242bbdf21"> <div class="comment-author"><a href="http://www.morcs.com">James Morcom</a></div> <div class="comment-content"> <p>Hi Mark, I guess there is implicit cyclomatic complexity in the testing framework itself (For example, it loops through the <code>InlineData</code> records). That feels fine though, does this somehow have less cost than cyclomatic complexity in the test code itself? I guess, as you mentioned, it's acceptable because the alternative is violation of DRY. </p> <p> With this in mind, I wonder how you feel about adding an <code>expectedName</code> parameter to the <code>InlineData</code> attributes, instead of the conditional in the test code? Maybe it's harder to read though when the test data includes input and output. </p> </div> <div class="comment-date">2020-12-07 08:36 UTC</div> </div> <div class="comment" id="1f5efe14d22441ffac85f5f5afc9b3b1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> James, thank you for writing. I consider <a href="/2019/12/09/put-cyclomatic-complexity-to-good-use#de927bfcc95d410bbfcd0adf7a63926b">the cyclomatic complexity of a method call to be <em>1</em></a>, and Visual Studio code metrics agree with me. Whatever happens in a framework should, in my opinion, likewise be considered as encapsulated abstraction that's none of our business. </p> <p> Adding an <code>expectedName</code> parameter to the method is definitely an option. I sometimes do that, and I could have done that here, too. In this situation, I think it's a toss-up. It'd make it harder for a later reader of the code to parse the test cases, but would simplify the test code itself, so that alternative comes with both advantages and disadvantages. </p> </div> <div class="comment-date">2020-12-08 11:02 UTC</div> </div> <div class="comment" id="e5a0f70c39f411ebadc10242ac120002"> <div class="comment-author">Romain Deneau <a href="https://twitter.com/DeneauRomain">@DeneauRomain</a></div> <div class="comment-content"> <p> Hi Mark. To build up on the additional <code>expectedName</code> parameter, instead of keeping a single test with the 3 cases but the last being a edge case, I prefer introduce a specific test for the last case. </p> <p> Then, to remove the duplication, we can extract a common method which will take this additional <code>expectedName</code> parameter: </p> <p> <pre> [<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;9)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithNameWhenDatabaseIsEmpty &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">string</span>&nbsp;at,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity)&nbsp;=> PostValidReservationWhenDatabaseIsEmpty(at, email, name, expectedName: name, quantity); [<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithoutNameWhenDatabaseIsEmpty()&nbsp;=> PostValidReservationWhenDatabaseIsEmpty( at          : <span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>, email       : <span style="color:#a31515;">&quot;juliad@example.net&quot;</span>, name        : <span style="color:blue;">null</span>, expectedName: <span style="color:#a31515;">&quot;&quot;</span>, quantity    : 5); <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;expectedName, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>   &nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity, &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;expectedName,&nbsp;<span style="color:green;">// /!\ Not `dto.Name`</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); } </pre> </p> </div> <div class="comment-date">2020-12-09 8:44 UTC</div> </div> <div class="comment" id="2779280fced748b0879a4d7267f3d634"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Romain, thank you for writing. There are, indeed, many ways to skin that cat. If you're comfortable with distributing a test over more than one method, I instead prefer to use another data source for the <code>[Theory]</code> attribute: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TheoryData</span>&lt;<span style="color:#2b91af;">ReservationDto</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2023,&nbsp;11,&nbsp;24,&nbsp;19,&nbsp;0,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2024,&nbsp;2,&nbsp;13,&nbsp;18,&nbsp;15,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithoutName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2023,&nbsp;8,&nbsp;23,&nbsp;16,&nbsp;55,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;2); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithName(<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(at,&nbsp;email,&nbsp;name,&nbsp;quantity)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithoutName(<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;{&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>),&nbsp;Email&nbsp;=&nbsp;email,&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(at,&nbsp;email,&nbsp;<span style="color:#a31515;">&quot;&quot;</span>,&nbsp;quantity)); &nbsp;&nbsp;&nbsp;&nbsp;} } [<span style="color:#2b91af;">Theory</span>,&nbsp;<span style="color:#2b91af;">ClassData</span>(<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;expected) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Whether you prefer one over the other is, I think, subjective. I like my alternative, using a <code>[ClassData]</code> source, better, because I find it a bit more principled and 'pattern-based', if you will. I also like how small the actual test method becomes. </p> <p> Your solution, on the other hand, is more portable, in the sense that you could also apply it in a testing framework that doesn't have the sort of capability that xUnit.net has. That's a definite benefit with your suggestion. </p> </div> <div class="comment-date">2020-12-10 20:05 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Name by role https://blog.ploeh.dk/2020/11/30/name-by-role 2020-11-30T06:31:00+00:00 Mark Seemann <div id="post"> <p> <em>Consider naming variables according to their role, instead of their type.</em> </p> <p> My <a href="/2020/11/23/good-names-are-skin-deep">recent article on good names</a> might leave you with the impression that I consider good names unimportant. Not at all. That article was an attempt at delineating the limits of naming. Good names aren't the panacea some people seem to imply, but they're still important. </p> <p> As the cliché goes, naming is one of the hardest problems in software development. Perhaps it's hard because you have to do it so frequently. Every time you create a variable, you have to name it. It's also an opportunity to add clarity to a code base. </p> <p> A common naming strategy is to name objects after their type: </p> <p> <pre><span style="color:#2b91af;">Reservation</span>?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(id);</pre> </p> <p> or: </p> <p> <pre><span style="color:#2b91af;">Restaurant</span>?&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase.GetRestaurant(restaurantId);</pre> </p> <p> There's nothing inherently wrong with a naming scheme like this. It often makes sense. The <code>reservation</code> variable is a <code>Reservation</code> object, and there's not that much more to say about it. The same goes for the <code>restaurant</code> object. </p> <p> In some contexts, however, objects play specific <em>roles</em>. This is particularly prevalent with primitive types, but can happen to any type of object. It may help the reader if you name the variables according to such roles. </p> <p> In this article, I'll show you several examples. I hope these examples are so plentiful and varied that they can inspire you to come up with good names. </p> <h3 id="895798815b414b368058ed8640236ff9"> A variable introduced only to be named <a href="#895798815b414b368058ed8640236ff9" title="permalink">#</a> </h3> <p> In a <a href="/2020/11/09/checking-signed-urls-with-aspnet">recent article</a> I showed this code snippet: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;signaturesMatch&nbsp;=&nbsp;computedSignature.SequenceEqual(receivedSignature); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;signaturesMatch; }</pre> </p> <p> Did you wonder about the <code>signaturesMatch</code> variable? Why didn't I just return the result of <code>SequenceEqual</code>, like the following? </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;computedSignature.SequenceEqual(receivedSignature); }</pre> </p> <p> Visual Studio even offers this as a possible refactoring that it'll do for you. </p> <p> The inclusion of the <code>signaturesMatch</code> variable was a conscious decision of mine. I felt that directly returning the result of <code>SequenceEqual</code> was a bit too implicit. It forces readers to make the inference themselves: <em>Ah, the two arrays contain the same sequence of bytes; that must mean that the signatures match!</em> </p> <p> Instead of asking readers to do that work themselves, I decided to do it for them. I hope that it improves readability. It doesn't change the behaviour of the code one bit. </p> <h3 id="e5f18d0e31264b29bf11cd817b8c7bfa"> Test roles <a href="#e5f18d0e31264b29bf11cd817b8c7bfa" title="permalink">#</a> </h3> <p> When it comes to unit testing, there's plenty of inconsistent terminology. One man's <em>mock object</em> is another woman's <em>test double</em>. Most of the jargon isn't even internally consistent. Do yourself a favour and adopt a consistent pattern language. I use the one presented in <a href="http://bit.ly/xunitpatterns">xUnit Test Patterns</a>. </p> <p> For instance, the thing that you're testing is the System Under Test (SUT). This can be a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a> or a static method, but when it's an object, you're going to create a variable. Consider <a href="https://docs.microsoft.com/en-us/archive/blogs/ploeh/naming-sut-test-variables">naming it <em>sut</em></a>. A typical test also defines other variables. Naming one of them <code>sut</code> clearly identifies which of them is the SUT. It also protects the tests against the class in question being renamed. </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ScheduleSingleReservationCommunalTable() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;table&nbsp;=&nbsp;<span style="color:#2b91af;">Table</span>.Communal(12); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(6), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;table); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Schedule(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;r&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSlot</span>(r.At,&nbsp;table.Reserve(r))&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(expected,&nbsp;actual); }</pre> </p> <p> The above test follows <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">my AAA.formatting heuristic</a>. In all, it defines five variables, but there can be little doubt about which one is the <code>sut</code>. </p> <p> The <code>table</code> and <code>r</code> variables follow the mainstream practice of naming variables after their type. They play no special role, so that's okay. You may balk at such a short variable name as <code>r</code>, and that's okay. In my defence, I follow <a href="http://amzn.to/XCJi9X">Clean Code</a>'s <em>N5</em> heuristic for long and short scopes. A variable name like <code>r</code> is fine when it only spans three lines of code (four, if you also count the blank line). </p> <p> Consider also using the variable names <code>expected</code> and <code>actual</code>, as in the above example. In many unit testing frameworks, those are the argument names for the assertion. For instance, in <a href="https://xunit.net">xUnit.net</a> (which the above test uses) the <code>Assert.Equals</code> overloads are defined as <code>Equal&lt;T&gt;(T expected, T actual)</code>. Using these names for variables makes the roles clearer, I think. </p> <h3 id="bb01b7e1d37f469e8a81f288b513248a"> The other <a href="#bb01b7e1d37f469e8a81f288b513248a" title="permalink">#</a> </h3> <p> The above assertion relies on structural equality. The <code>TimeSlot</code> class is immutable, so it can safely override <code>Equals</code> (and <code>GetHashCode</code>) to implement structural equality: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>?&nbsp;obj) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">TimeSlot</span>&nbsp;other&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;==&nbsp;other.At&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tables.SequenceEqual(other.Tables); }</pre> </p> <p> I usually call the downcast variable <code>other</code> because, from the perspective of the instance, it's the other object. I usually use that convention whenever an instance interacts with another object of the same type. Among other examples, this happens when you model objects as <a href="/2017/11/27/semigroups">semigroups</a> and <a href="/2017/10/06/monoids">monoids</a>. The <a href="/2018/07/16/angular-addition-monoid">Angle struct, for example, defines this binary operation</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Angle</span>&nbsp;Add(<span style="color:#2b91af;">Angle</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Angle</span>(<span style="color:blue;">this</span>.degrees&nbsp;+&nbsp;other.degrees); }</pre> </p> <p> Again, the method argument is in the role as the other object, so naming it <code>other</code> seems natural. </p> <p> Here's another example from a restaurant reservation code base: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Seating</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Start&nbsp;&lt;&nbsp;other.End&nbsp;&amp;&amp;&nbsp;other.Start&nbsp;&lt;&nbsp;End; }</pre> </p> <p> The <code>Overlaps</code> method is an instance method on the <code>Seating</code> class. Again, <code>other</code> seems natural. </p> <h3 id="8cf3583ad0ac446fa4b4dee2272ba054"> Candidates <a href="#8cf3583ad0ac446fa4b4dee2272ba054" title="permalink">#</a> </h3> <p> The <code>Overlaps</code> method looks like a <em>predicate</em>, i.e. a function that returns a Boolean value. In the case of that method, <code>other</code> indicates the role of being the other object, but it also plays another role. It makes sense to me to call predicate input <em>candidates</em>. Typically, you have some input that you want to evaluate as either true or false. I think it makes sense to think of such a parameter as a 'truth candidate'. You can see one example of that in the above <code>SignatureIsValid</code> method. </p> <p> There, the <code>string</code> parameter is a <code>candidate</code> for having a valid signature. </p> <p> Here's another restaurant-related example: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;WillAccept( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;now, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;existingReservations, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;candidate) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(existingReservations&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(existingReservations)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(candidate&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(candidate.At&nbsp;&lt;&nbsp;now) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsOutsideOfOpeningHours(candidate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;seating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;candidate.At); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;relevantReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;existingReservations.Where(seating.Overlaps); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;availableTables&nbsp;=&nbsp;Allocate(relevantReservations); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;availableTables.Any(t&nbsp;=&gt;&nbsp;t.Fits(candidate.Quantity)); }</pre> </p> <p> Here, the reservation in question is actually not yet a reservation. It might be rejected, so it's a <code>candidate</code> reservation. </p> <p> You can also use that name in <code>TryParse</code> methods, as shown in <a href="/2019/12/09/put-cyclomatic-complexity-to-good-use">this article</a>. </p> <h3 id="4301289931dc4f94abf46578ddcfd693"> Data Transfer Objects <a href="#4301289931dc4f94abf46578ddcfd693" title="permalink">#</a> </h3> <p> Another name that I like to use is <code>dto</code> for <a href="https://en.wikipedia.org/wiki/Data_transfer_object">Data Transfer Objects</a> (DTOs). The benefit here is that as long as <code>dto</code> is unambiguous in context, it makes it easier to distinguish between a DTO and the domain model you might want to turn it into: </p> <p> <pre>[<span style="color:#2b91af;">HttpPost</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/reservations&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;<span style="color:#2b91af;">Guid</span>.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase.GetRestaurant(restaurantId); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(restaurant&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;TryCreate(restaurant,&nbsp;reservation); }</pre> </p> <p> By naming the input parameter <code>dto</code>, I keep the name <code>reservation</code> free for the domain object, which ought to be the more important object of the two. <blockquote> <p> "A Data Transfer Object is one of those objects our mothers told us never to write." </p> <footer><cite><a href="http://bit.ly/patternsofeaa">Martin Fowler</a></cite></footer> </blockquote> I could have named the input parameter <code>reservationDto</code> instead of <code>dto</code>, but that would diminish the 'mental distance' between <code>reservationDto</code> and <code>reservation</code>. I like to keep that distance, so that the roles are more explicit. </p> <h3 id="b8ece382f7554141aa741fc9ded9e02a"> Time <a href="#b8ece382f7554141aa741fc9ded9e02a" title="permalink">#</a> </h3> <p> You often need to make decisions based on the current time or date. In .NET the return value from <a href="https://docs.microsoft.com/dotnet/api/system.datetime.now">DateTime.Now</a> is a <code>DateTime</code> value. Typical variable names are <code>dateTime</code>, <code>date</code>, <code>time</code>, or <code>dt</code>, but why not call it <code>now</code>? </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;TryCreate(<span style="color:#2b91af;">Restaurant</span>&nbsp;restaurant,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;scope&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TransactionScope</span>(<span style="color:#2b91af;">TransactionScopeAsyncFlowOption</span>.Enabled); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.ReadReservations(restaurant.Id,&nbsp;reservation.At); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(restaurant.Id,&nbsp;reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;scope.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(restaurant.Id,&nbsp;reservation); }</pre> </p> <p> This is the <code>TryCreate</code> method called by the above <code>Post</code> method. Here, <code>DateTime.Now</code> is hidden behind <code>Clock.GetCurrentDateTime()</code> in order to make <a href="/2020/03/23/repeatable-execution">execution repeatable</a>, but the idea remains: the variable represents the current time or date, or, with a bit of good will, <code>now</code>. </p> <p> Notice that the <code>WillAccept</code> method (shown above) also uses <code>now</code> as a parameter name. That value's role is to represent <code>now</code> as a concept. </p> <p> When working with time, I also sometimes use the variable names <code>before</code> and <code>after</code>. This is mostly useful in integration tests: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;GetCurrentYear() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LegacyApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;before&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;response&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.GetCurrentYear(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;after&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;response.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;response.ParseJsonContent&lt;<span style="color:#2b91af;">CalendarDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;AssertOneOf(before.Year,&nbsp;after.Year,&nbsp;actual.Year); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Null(actual.Month); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Null(actual.Day); &nbsp;&nbsp;&nbsp;&nbsp;AssertLinks(actual); }</pre> </p> <p> While you can inject something like a <code>Clock</code> dependency in order to make your SUT deterministic, in integration tests you might want to see behaviour when using the system clock. You can often verify such behaviour by surrounding the test's <em>Act</em> phase with two calls to <code>DateTime.Now</code>. This gives you the time <code>before</code> and <code>after</code> the test exercised the SUT. </p> <p> When you do that, however, be careful with the assertions. If such a test runs at midnight, <code>before</code> and <code>after</code> might be two different dates. If it runs on midnight December 31, it might actually be two different years! That's the reason that the test passes as long as the <code>actual.Year</code> is either of <code>before.Year</code> and <code>after.Year</code>. </p> <h3 id="8183f4e53fdd4ceca701ef17a6509614"> Invalid values <a href="#8183f4e53fdd4ceca701ef17a6509614" title="permalink">#</a> </h3> <p> While integration tests often test happy paths, unit tests should also exercise error paths. What happens when you supply invalid input to a method? When you write such tests, you can identify the invalid values by naming the variables or parameters accordingly: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:blue;">null</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;bas&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PutInvalidId(<span style="color:blue;">string</span>&nbsp;invalidId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Some</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dummyDto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2024-06-25&nbsp;18:19&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;colera@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Cole&nbsp;Aera&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Put(invalidId,&nbsp;dummyDto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">NotFoundResult</span>&gt;(actual); }</pre> </p> <p> Here, the invalid input represent an ID. To indicate that, I called the parameter <code>invalidId</code>. </p> <p> The system under test is the <code>Put</code> method, which takes two arguments: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Put(<span style="color:blue;">string</span>&nbsp;id,&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto)</pre> </p> <p> When testing an error path, it's important to keep other arguments well-behaved. In this example, I want to make sure that it's the <code>invalidId</code> that causes the <code>NotFoundResult</code> result. Thus, the <code>dto</code> argument should be as well-behaved as possible, so that it isn't going to be the source of divergence. </p> <p> Apart from being well-behaved, that object plays no role in the test. It just needs to be there to make the code compile. <em>xUnit Test Patterns</em> calls such an object a <em>Dummy Object</em>, so I named the variable <code>dummyDto</code> as information to any reader familiar with that pattern language. </p> <h3 id="441f47fab8b74787840c0237b3958316"> Derived class names <a href="#441f47fab8b74787840c0237b3958316" title="permalink">#</a> </h3> <p> The thrust of all of these examples is that you don't <em>have</em> to name variables after their types. You can extend this line of reasoning to class inheritance. Just because a base class is called <code>Foo</code> it doesn't mean that you <em>have</em> to call a derived class <code>SomethingFoo</code>. </p> <p> This is something of which I have to remind myself. For example, to support integration testing with ASP.NET you'll need a <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.testing.webapplicationfactory-1">WebApplicationFactory&lt;TEntryPoint&gt;</a>. To override the default DI Container configuration, you'll have to derive from this class and override its <code>ConfigureWebHost</code> method. In <a href="/2020/04/20/unit-bias-against-collections">an example I've previously published</a> I didn't spend much time thinking about the class name, so <code>RestaurantApiFactory</code> it was. </p> <p> At first, I named the variables of this type <code>factory</code>, or something equally devoid of information. That bothered me, so instead tried <code>service</code>, which I felt was an improvement, but still too vapid. I then adopted <code>api</code> as a variable name, but then realised that that also suggested a better class name. So currently, this defines my self-hosting API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>&nbsp;:&nbsp;<span style="color:#2b91af;">WebApplicationFactory</span>&lt;<span style="color:#2b91af;">Startup</span>&gt;</pre> </p> <p> Here's how I use it: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;ReserveTableAtNono() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;client&nbsp;=&nbsp;api.CreateClient(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation.ToDto(); &nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity&nbsp;=&nbsp;6; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;response&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;client.PostReservation(<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation.At; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;AssertRemainingCapacity(client,&nbsp;at,&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;4); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;AssertRemainingCapacity(client,&nbsp;at,&nbsp;<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>,&nbsp;10); }</pre> </p> <p> The variable is just called <code>api</code>, but the reader can tell from the initialisation that this is an instance of the <code>SelfHostedApi</code> class. I like how that communicates that this is an integration test that uses a self-hosted API. It literally says that. </p> <p> This test also uses the <code>dto</code> naming convention. Additionally, you may take note of the variable and property called <code>at</code>. That's another name for a date and time. I struggled with naming this value, until <a href="http://blog.strobaek.org">Karsten Strøbæk</a> suggested that I used the simple word <em>at:</em> <code>reservation.At</code> indicates the date and time of the reservation without being encumbered by awkward details about date and time. Should we call it <code>date</code>? <code>time</code>? <code>dateTime</code>? No, just call it <code>at</code>. I find it elegant. </p> <h3 id="6692436d37dd491fa9920a5f4ac63118"> Conclusion <a href="#6692436d37dd491fa9920a5f4ac63118" title="permalink">#</a> </h3> <p> Sometimes, a <code>Reservation</code> object is just a <code>reservation</code>, and that's okay. At other times, it's the <code>actual</code> value, or the <code>expected</code> value. If it represents an invalid reservation in a test case, it makes sense to call the variable <code>invalidResevation</code>. </p> <p> Giving variables descriptive names improves <a href="/2019/03/04/code-quality-is-not-software-quality">code quality</a>. You don't have to write <a href="http://butunclebob.com/ArticleS.TimOttinger.ApologizeIncode">comments as apologies for poor readability</a> if a better name communicates what the comment would have said. </p> <p> Consider naming variables (and classes) for the <em>roles</em> they play, rather than their types. </p> <p> On the other hand, <a href="/2016/10/25/when-variable-names-are-in-the-way">when variable names are in the way</a>, consider <a href="https://en.wikipedia.org/wiki/Tacit_programming">point-free code</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="8d7dc043829244509b1a13c184f3cbbf"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Excellent name suggestions. Thanks for sharing them :) </p> <blockquote> I usually call the downcast variable <code>other</code> because, from the perspective of the instance, it's the other object. I usually use that convention whenever an instance interacts with another object of the same type. </blockquote> <p> The name <code>other</code> is good, but I prefer <code>that</code> because I think it is a better antonym of <code>this</code> (the keyword for the current instance) and because it has the same number of letters as <code>this</code>. </p> <blockquote> <p> It makes sense to me to call predicate input <em>candidates</em>. Typically, you have some input that you want to evaluate as either true or false. I think it makes sense to think of such a parameter as a 'truth candidate'. You can see one example of that in the above <code>SignatureIsValid</code> method. </p> <p>...</p> <p> Here, the reservation in question is actually not yet a reservation. It might be rejected, so it's a <code>candidate</code> reservation. </p> </blockquote> <p> I typically try to avoid turning some input into either true or false. In particular, I find it confusing for the syntax to say that some instance is a <code>Reservation</code> while the semantics says that it "is actually not yet a reservation". I think of this as an example of <a href="https://blog.ploeh.dk/2015/01/19/from-primitive-obsession-to-domain-modelling/">primitive obsession</a>. Strictly speaking, I think <a href="https://medium.com/the-sixt-india-blog/primitive-obsession-code-smell-that-hurt-people-the-most-5cbdd70496e9#5009:~:text=Primitive%20Obsession%20is%20when%20the%20code%20relies%20too%20much%20on%20primitives.">"Primitive Obsession is when the code relies too much on primitives."</a> (aka, on primitive types). In my mind though, I have generalized this to cover any code that relies too much on weaker types. Separate types <code>Reservation</code> and <code>bool</code> are weaker than separate types <code>Reservation</code> and <code>CandidateReservation</code>. I think Alexis King summarized this well with a blog post titled <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">Parse, don’t validate</a>. </p> <p> And yet, my coworkers and I have engaged in friendly but serious debates for years about which of those two approaches is better. My argument, essentially as given above, is for separate types <code>Reservation</code> and <code>CandidateReservation</code>. The main counterargument is that these types are the same except for a database-generated ID, so just represent both using one type with an optional ID. </p> <p> Have you thought about this before? </p> <blockquote> <p> By naming the input parameter <code>dto</code>, I keep the name <code>reservation</code> free for the domain object, which ought to be the more important object of the two. </p> <p>...</p> <p> I could have named the input parameter <code>reservationDto</code> instead of <code>dto</code>, but that would diminish the 'mental distance' between <code>reservationDto</code> and <code>reservation</code>. I like to keep that distance, so that the roles are more explicit. </p> </blockquote> <p> I prefer to emphasize the roles even more by using the names <code>dto</code> and <code>model</code>. We are in the implementation of the (Post) route for <code>"restaurants/{restaurantId}/reservations"</code>, so I think it is clear from context that the <code>dto</code> and <code>model</code> are really a reservation DTO and a reservation model. </p> </div> <div class="comment-date">2020-11-30 20:46 UTC</div> </div> <div class="comment" id="931a54d51d3d445c9eaf6a41d8923406"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. Certainly, I didn't intent my article to dictate names. As you imply, there's room for both creativity and subjectivity, and that's fine. My suggestions were meant only for inspiration. <blockquote> <p> The main counterargument is that these types are the same except for a database-generated ID, so just represent both using one type with an optional ID. </p> <p> Have you thought about this before? </p> </blockquote> Yes; I would <a href="/2014/08/11/cqs-versus-server-generated-ids">think twice before deciding to model a domain type with a database-generated ID</a>. A server-generated ID is an implementation detail that shouldn't escape the data access layer. If it does, you have a leaky abstraction at hand. Sooner or later, it's going to bite you. </p> <p> The <code>Reservation</code> class in the above examples has this sole constructor: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(<span style="color:#2b91af;">Guid</span>&nbsp;id,&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:#2b91af;">Email</span>&nbsp;email,&nbsp;<span style="color:#2b91af;">Name</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity)</pre> </p> <p> You can't create an instance without supplying an ID. On the other hand, any code can conjure up a GUID, so no server is required. At the type-level, there's no compelling reason to distinguish between a reservation and a candidate reservation. </p> <p> Granted, you <em>could</em> define two types, <code>Reservation</code> and <code>CandidateReservation</code>, but they'd be isomorphic. In Haskell, you'd probably use a <code>newtype</code> for one of these types, and then you're <a href="https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-type-safety">back at Alexis King's blog</a>. </p> </div> <div class="comment-date">2020-12-02 7:43 UTC</div> </div> <div class="comment" id="fe3ac3a60c754340801b877666a07b65"> <div class="comment-author"><a href="https://ttulka.com">Tomas Tulka</a></div> <div class="comment-content"> <blockquote>...naming is one of the hardest problems in software development. Perhaps it's hard because you have to do it so frequently.</blockquote> <p>Usually, doing things frequently means mastering them pretty quickly. Not so for naming. I guess, there are multiple issues:</p> <ol> <li>Words are ambiguous. The key is, not to do naming in isolation, the context matters. For example, it's difficult to come up with a good name for a method when we don't have a good name for its class, the whole component, etc. Similar with Clean Code's N5: the meaning of a short variable is clear in a small scope, closed context.</li> <li>Good naming requires deep understanding of the domain. Developers are usualy not good at the business they model. Sadly, it often means "necessary evil" for them.</li> </ol> <p>Naming variables by their roles is a great idea!</p> <p>Many thanks for another awesome post, I enjoyed reading it.</p> </div> <div class="comment-date">2020-12-11 09:11 UTC</div> </div> <div class="comment" id="e9299b251c8e4b7f9a168ff571f70950"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tomas, thank you for writing. <blockquote> <p> doing things frequently means mastering them pretty quickly. Not so for naming. I guess </p> </blockquote> Good point; I hadn't thought about that. I think that the reasons you list are valid. </p> <p> As an additional observation, it may be that there's a connection to the notion of <em>deliberate practice</em>. As the catch-phrase about professional experience puts it, there's a difference between 20 years of experience and one year of experience repeated 20 times. </p> <p> Doing a thing again and again generates little improvement if one does it by rote. One has to deliberately practice. In this case, it implies that a programmer should explicitly reflect on variable names, and consider more than one option. </p> <p> I haven't met many software developers who do that. </p> </div> <div class="comment-date">2020-12-15 9:19 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Good names are skin-deep https://blog.ploeh.dk/2020/11/23/good-names-are-skin-deep 2020-11-23T06:33:00+00:00 Mark Seemann <div id="post"> <p> <em>Good names are important, but insufficient, for code maintainability.</em> </p> <p> You should give the building blocks of your code bases descriptive names. It's easier to understand the purpose of a library, module, class, method, function, etcetera if the name contains a clue about the artefact's purpose. This is hardly controversial, and while naming is hard, most teams I visit agree that names are important. </p> <p> Still, despite good intentions and efforts to name things well, code bases deteriorate into unmaintainable clutter. </p> <p> Clearly, good names aren't enough. </p> <h3 id="5a20f600298844e98c89c423f5c66c5f"> Tenuousness of names <a href="#5a20f600298844e98c89c423f5c66c5f" title="permalink">#</a> </h3> <p> A good name is tenuous. First, naming is hard, so while you may have spent some effort coming up with a good name, other people may misinterpret it. Because they originate from natural language, names are as ambiguous as language. (<a href="/2018/07/02/terse-operators-make-business-code-more-readable">Terse operators, on the other hand...</a>) </p> <p> Another maintainability problem with names is that implementation may change over time, but the names remain constant. Granted, modern IDEs make it easy to rename methods, but developers rarely adjust names when they adjust behaviour. Even the best names may become misleading over time. </p> <p> These weakness aren't the worst, though. In my experience, a more fundamental problem is that all it takes is one badly named 'wrapper object' before the information in a good name is lost. </p> <p> <img src="/content/binary/vague-names-hiding-clear-names.png" alt="Object with clear names enclosed in object with vague names."> </p> <p> In the figure, the inner object is well-named. It has a clear name and descriptive method names. All it takes before this information is lost, however, is another object with vague names to 'encapsulate' it. </p> <h3 id="c0cfcf2d16a94e4c96c33c9d0359846f"> An attempt at a descriptive method name <a href="#c0cfcf2d16a94e4c96c33c9d0359846f" title="permalink">#</a> </h3> <p> Here's an example. Imagine an online restaurant reservation system. One of the features of this system is to take reservations and save them in the database. </p> <p> A restaurant, however, is a finite resource. It can only accommodate a certain number of guests at the same time. Whenever the system receives a reservation request, it'll have to retrieve the existing reservations for that time and make a decision. <a href="/2020/01/27/the-maitre-d-kata">Can it accept the reservation?</a> Only if it can should it save the reservation. </p> <p> How do you model such an interaction? How about a descriptive name? How about <code>TrySave</code>? Here's a possible implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">bool</span>&gt;&nbsp;TrySave(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(reservation)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservation.At, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservation.At&nbsp;+&nbsp;SeatingDuration) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;availableTables&nbsp;=&nbsp;Allocate(reservations); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!availableTables.Any(t&nbsp;=&gt;&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;t.Seats)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">true</span>; }</pre> </p> <p> There's an implicit naming convention in .NET that methods with the <code>Try</code> prefix indicate an operation that may or may not succeed. The return value of such methods is either <code>true</code> or <code>false</code>, and they may also have <code>out</code> parameters if they optionally produce a value. That's not the case here, but I think one could make the case that <code>TrySave</code> succinctly describes what's going on. </p> <p> All is good, then? </p> <h3 id="bbf9fd5509ba4a2c823483acd40fbe22"> A vague wrapper <a href="#bbf9fd5509ba4a2c823483acd40fbe22" title="permalink">#</a> </h3> <p> After our conscientious programmer meticulously designed and named the above <code>TrySave</code> method, it turns out that it doesn't meet all requirements. Users of the system file a bug: the system accepts reservations outside the restaurant's opening hours. </p> <p> The original programmer has moved on to greener pastures, so fixing the bug falls on a poor maintenance developer with too much to do. Having recently learned about the <a href="https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle">open-closed principle</a>, our new protagonist decides to wrap the existing <code>TrySave</code> in a new method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">bool</span>&gt;&nbsp;Check(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(reservation)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation.At&nbsp;&lt;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation.At.TimeOfDay&nbsp;&lt;&nbsp;OpensAt) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(LastSeating&nbsp;&lt;&nbsp;reservation.At.TimeOfDay) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;Manager.TrySave(reservation).ConfigureAwait(<span style="color:blue;">false</span>); }</pre> </p> <p> This new method first checks whether the <code>reservation</code> is within opening hours and in the future. If that's not the case, it returns <code>false</code>. Only if these preconditions are fulfilled does it delegate the decision to that <code>TrySave</code> method. </p> <p> Notice, however, the name. The bug was urgent, and our poor programmer didn't have time to think of a good name, so <code>Check</code> it is. </p> <h3 id="6056d14ac9ab4046b4aa417d3902fbf1"> Caller's perspective <a href="#6056d14ac9ab4046b4aa417d3902fbf1" title="permalink">#</a> </h3> <p> How does this look from the perspective of calling code? Here's the Controller action that handles the pertinent HTTP request: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;r&nbsp;=&nbsp;dto.Validate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(r&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;isOk&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Manager.Check(r).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!isOk) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">StatusCodeResult</span>(<span style="color:#2b91af;">StatusCodes</span>.Status500InternalServerError); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NoContentResult</span>(); }</pre> </p> <p> Try to forget the code you've just seen and imagine that you're looking at this code first. You'd be excused if you miss what's going on. It looks as though the method just does a bit of validation and then <em>checks</em> 'something' concerning the reservation. </p> <p> There's no hint that the <code>Check</code> method might perform the significant side effect of saving the reservation in the database. </p> <p> You'll only learn that if you <em>read</em> the implementation details of <code>Check</code>. As I argue in my <a href="https://cleancoders.com/episode/humane-code-real-episode-1">Humane Code video</a>, <em>if you have to read the source code of an object, encapsulation is broken.</em> </p> <p> Such code doesn't fit in your brain. You'll struggle as you try keep track of all the things that are going on in the code, all the way from the outer boundary of the application to implementation details that relate to databases, third-party services, etcetera. </p> <h3 id="59249ae122b540ca907549ade3eca649"> Straw man? <a href="#59249ae122b540ca907549ade3eca649" title="permalink">#</a> </h3> <p> You may think that this is a straw man argument. After all, wouldn't it be better to edit the original <code>TrySave</code> method? </p> <p> Perhaps, but it would make that class more complex. The <code>TrySave</code> method has a <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a> of only <em>3</em>, while the <code>Check</code> method has a complexity of <em>5</em>. Combining them might easily take them over some <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a>. </p> <p> Additionally, each of these two classes have different dependencies. As the <code>TrySave</code> method implies, it relies on both <code>Repository</code> and <code>SeatingDuration</code>, and the <code>Allocate</code> helper method (not shown) uses a third dependency: the restaurant's table configuration. </p> <p> Likewise, the <code>Check</code> method relies on <code>OpensAt</code> and <code>LastSeating</code>. If you find it better to edit the original <code>TrySave</code> method, you'd have to combine these dependencies as well. Each time you do that, the class grows until it becomes a <a href="https://en.wikipedia.org/wiki/God_object">God object</a>. </p> <p> It's rational to attempt to separate things in multiple classes. It also, on the surface, seems to make unit testing easier. For example, here's a test that verifies that the <code>Check</code> method rejects reservations before the restaurant's opening time: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;RejectReservationBeforeOpeningTime() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.AddDays(10).Date.AddHours(17), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;colaera@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Cole&nbsp;Aera&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;mgrTD&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">IReservationsManager</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;mgrTD.Setup(mgr&nbsp;=&gt;&nbsp;mgr.TrySave(r)).ReturnsAsync(<span style="color:blue;">true</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">RestaurantManager</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mgrTD.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Check(r); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.False(actual); }</pre> </p> <p> By replacing the <code>TrySave</code> method by a test double, you've ostensibly decoupled the <code>Check</code> method from all the complexity of the <code>TrySave</code> method. </p> <p> To be clear, this style of programming, with lots of nested interfaces and tests with <a href="/2013/10/23/mocks-for-commands-stubs-for-queries">mocks and stubs</a> is far from ideal, but I still find it better than a <a href="https://en.wikipedia.org/wiki/Big_ball_of_mud">big ball of mud</a>. </p> <h3 id="33cb10390fa4417f96b24e4b9102d4ed"> Alternative <a href="#33cb10390fa4417f96b24e4b9102d4ed" title="permalink">#</a> </h3> <p> A better alternative is <a href="https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell">Functional Core, Imperative Shell</a>, AKA <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>. Move all impure actions to the edge of the system, leaving only <a href="https://en.wikipedia.org/wiki/Referential_transparency">referentially transparent</a> functions as the main implementers of logic. It could look like this: </p> <p> <pre>[<span style="color:#2b91af;">HttpPost</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;<span style="color:#2b91af;">Guid</span>.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;r&nbsp;=&nbsp;dto.Validate(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(r&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.ReadReservations(r.At).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!MaitreD.WillAccept(<span style="color:#2b91af;">DateTime</span>.Now,&nbsp;reservations,&nbsp;r)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(r).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(r); }</pre> </p> <p> Nothing is swept under the rug here. <code>WillAccept</code> is a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a>, and while it encapsulates significant complexity, the only thing you need to understand when you're trying to understand the above <code>Post</code> code is that it returns either <code>true</code> or <code>false</code>. </p> <p> Another advantage of pure functions is that they are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. That makes unit testing and test-driven development easier. </p> <p> Even with a functional core, you'll also have an imperative shell. You can still test that, too, such as the <code>Post</code> method. It isn't referentially transparent, so you might be inclined to use mocks and stubs, but I instead recommend <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based testing with a Fake database</a>. </p> <h3 id="6758361ac712400aae148a7dcc2a4a70"> Conclusion <a href="#6758361ac712400aae148a7dcc2a4a70" title="permalink">#</a> </h3> <p> Good names are important, but don't let good names, alone, lull you into a false sense of security. All it takes is one vaguely named wrapper object, and all the information in your meticulously named methods is lost. </p> <p> This is one of many reasons I try to design with static types instead of names. Not that I dismiss the value of good names. After all, you'll have to give your types good names as well. </p> <p> Types are more robust in the face of inadvertent changes; or, rather, they tend to resist when we try to do something stupid. I suppose that's what lovers of dynamically typed languages feel as 'friction'. In my mind, it's entirely opposite. Types keep me honest. </p> <p> Unfortunately, most type systems don't offer an adequate degree of safety. Even in <a href="https://fsharp.org">F#</a>, which has a great type system, you can introduce impure actions into what you thought was a pure function, and <a href="/2020/02/24/discerning-and-maintaining-purity">you'd be none the wiser</a>. That's one of the reasons I find <a href="https://www.haskell.org">Haskell</a> so interesting. Because of <a href="/2020/06/08/the-io-container">the way IO works</a>, you can't inadvertently sweep surprises under the rug. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="9ecb24dc5f78413687547c7f74f2d8b9"> <div class="comment-author">Johannes Schmitt</div> <div class="comment-content"> <p> I find the idea of the impure/pure/impure sandwich rather interesting and I agree with the benefits that it yields. However, I was wondering about where to move synchronization logic, i.e. the reservation system should avoid double bookings. With the initial TrySave approach it would be clear for me where to put this logic: the synchonrization mechanism should be part of the TrySave method. With the impure/pure/impure sandwich, it will move out to the most outer layern (HTTP Controller) - at least this is how I'd see it. My feelings tells me that this is a bit smelly, but I can't really pin point why I think so. Can you give some advice on this? How would you solve that? </p> </div> <div class="comment-date">2020-12-12 19:08 UTC</div> </div> <div class="comment" id="ebce3b718bc84329b6979bcacf6c2573"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Johannes, thank you for writing. There are several ways to address that issue, depending on what sort of trade-off you're looking for. There's always a trade-off. </p> <p> You can address the issue with a lock-free architecture. This typically involves expressing the desired action as a Command and putting it on a durable queue. If you combine that with a single-threaded, single-instance Actor that pulls Commands off the queue, you need no further transaction processing, because the architecture itself serialises writes. You can find plenty of examples of such an architecture on the internet, including (IIRC) my Pluralsight course <a href="/functional-architecture-with-fsharp">Functional Architecture with F#</a>. </p> <p> Another option is to simply surround the <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a> with a <a href="https://docs.microsoft.com/dotnet/api/system.transactions.transactionscope">TransactionScope</a> (if you're on .NET, that is). </p> </div> <div class="comment-date">2020-12-16 16:59 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Redirect legacy URLs https://blog.ploeh.dk/2020/11/16/redirect-legacy-urls 2020-11-16T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Evolving REST API URLs when cool URIs don't change.</em> </p> <p> More than one reader reacted to my article on <a href="/2020/10/26/fit-urls">fit URLs</a> by asking about bookmarks and original URLs. <a href="/2020/10/26/fit-urls#b0fb43ff9aba4c14a075e7effc9fae25">Daniel Sklenitzka's question</a> is a good example: <blockquote> <p> "I see how signing the URLs prevents clients from retro-engineering the URL templates, but how does it help preventing breaking changes? If the client stores the whole URL instead of just the ID and later the URL changes because a restaurant ID is added, the original URL is still broken, isn't it?" </p> </blockquote> While I answered the question on the same page, I think that it's worthwhile to expand it. </p> <h3 id="3e1e7cd6b79e4e7b964a96a527312ab0"> The rules of HTTP <a href="#3e1e7cd6b79e4e7b964a96a527312ab0" title="permalink">#</a> </h3> <p> I agree with the implicit assumption that clients are allowed to bookmark links. It seems, then, like a breaking change if you later change your internal URL scheme. That seems to imply that the bookmarked URL is gone, breaking a tenet of the HTTP protocol: <a href="https://www.w3.org/Provider/Style/URI">Cool URIs don't change</a>. </p> <p> REST APIs are supposed to play by the rules of HTTP, so it'd seem that once you've published a URL, you can never retire it. You can, on the other hand, change its behaviour. </p> <p> Let's call such URLs <em>legacy URLs</em>. Keep them around, but change them to return <code>301 Moved Permanently</code> responses. </p> <p> The rules of REST go both ways. The API is expected to play by the rules of HTTP, and so are the clients. Clients are not only expected to follow links, but also redirects. If a legacy URL starts returning a <code>301 Moved Permanently</code> response, a well-behaved client doesn't break. </p> <h3 id="96913fb84fb344ccae75e1e2542ed51c"> Reverse proxy <a href="#96913fb84fb344ccae75e1e2542ed51c" title="permalink">#</a> </h3> <p> As I've <a href="/2020/06/01/retiring-old-service-versions#bc73620aa71141b6a74f4a4aaf395d75">previously described</a>, one of the many benefits of HTTP-based services is that you can put a <a href="https://en.wikipedia.org/wiki/Reverse_proxy">reverse proxy</a> in front of your application servers. I've no idea how to configure or operate <a href="https://en.wikipedia.org/wiki/Nginx">NGINX</a> or <a href="https://en.wikipedia.org/wiki/Varnish_(software)">Varnish</a>, but from talking to people who do know, I get the impression that they're quite scriptable. </p> <p> Since the above ideas are independent of actual service implementation or behaviour, it's a generic problem that you should seek to address with general-purpose software. </p> <p> <img src="/content/binary/reverse-proxy-based-redirect.png" alt="Sequence diagram showing a reverse proxy returning a redirect response to a request for a legacy URL."> </p> <p> Imagine that a reverse proxy is configured with a set of rules that detects legacy URLs and knows how to forward them. Clearly, the reverse proxy must know of the REST API's current URL scheme to be able to do that. You might think that this would entail leaking an implementation detail, but just as I consider any database used by the API as part of the overall system, I'd consider the reverse proxy as just another part. </p> <h3 id="410c6db0f3a54e969c1f54aa9f1ce59e"> Redirecting with ASP.NET <a href="#410c6db0f3a54e969c1f54aa9f1ce59e" title="permalink">#</a> </h3> <p> If you don't have a reverse proxy, you can also implement redirects in code. It'd be better to use something like a reverse proxy, because that would mean that you get to delete code from your code base, but sometimes that's not possible. </p> <p> In ASP.NET, you can return <code>301 Moved Permanently</code> responses just like any other kind of HTTP response: </p> <p> <pre>[<span style="color:#2b91af;">Obsolete</span>(<span style="color:#a31515;">&quot;Use&nbsp;Get&nbsp;method&nbsp;with&nbsp;restaurant&nbsp;ID.&quot;</span>)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;calendar/{year}/{month}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;LegacyGet(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">RedirectToActionResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(Get), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;restaurantId&nbsp;=&nbsp;<span style="color:#2b91af;">Grandfather</span>.Id,&nbsp;year,&nbsp;month&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;permanent:&nbsp;<span style="color:blue;">true</span>); }</pre> </p> <p> This <code>LegacyGet</code> method redirects to the current Controller action called <code>Get</code> by supplying the arguments that the new method requires. The <code>Get</code> method has this signature: </p> <p> <pre>[<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/calendar/{year}/{month}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month)</pre> </p> <p> When I expanded the API from a single restaurant to a multi-tenant system, I had to <a href="https://en.wikipedia.org/wiki/Grandfather_clause">grandfather in</a> the original restaurant. I gave it a <code>restaurantId</code>, but in order to not put <a href="https://en.wikipedia.org/wiki/Magic_number_(programming)">magic constants</a> in the code, I defined it as the named constant <code>Grandfather.Id</code>. </p> <p> Notice that I also adorned the <code>LegacyGet</code> method with an <code>[Obsolete]</code> attribute to make it clear to maintenance programmers that this is legacy code. You might argue that the <em>Legacy</em> prefix already does that, but the <code>[Obsolete]</code> attribute will make the compiler emit a warning, which is <a href="/2011/04/29/Feedbackmechanismsandtradeoffs">even better feedback</a>. </p> <h3 id="69aad25084a5452b96e14bc81a28a53c"> Regression test <a href="#69aad25084a5452b96e14bc81a28a53c" title="permalink">#</a> </h3> <p> While legacy URLs may be just that: legacy, that doesn't mean that it doesn't matter whether or not they work. You may want to add regression tests. </p> <p> If you implement redirects in code (as opposed to a reverse proxy), you should also add automated tests that verify that the redirects work: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;http://localhost/calendar/2020?sig=ePBoUg5gDw2RKMVWz8KIVzF%2Fgq74RL6ynECiPpDwVks%3D&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;http://localhost/calendar/2020/9?sig=ZgxaZqg5ubDp0Z7IUx4dkqTzS%2Fyjv6veDUc2swdysDU%3D&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;BookmarksStillWork(<span style="color:blue;">string</span>&nbsp;bookmarkedAddress) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LegacyApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateDefaultClient().GetAsync(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(bookmarkedAddress)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(<span style="color:#2b91af;">HttpStatusCode</span>.MovedPermanently,&nbsp;actual.StatusCode); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;follow&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(actual.Headers.Location); &nbsp;&nbsp;&nbsp;&nbsp;follow.EnsureSuccessStatusCode(); }</pre> </p> <p> This test interacts with a self-hosted service at the HTTP level. <code>LegacyApi</code> is a test-specific helper class that derives from <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.testing.webapplicationfactory-1">WebApplicationFactory&lt;Startup&gt;</a>. </p> <p> The test uses URLs that I 'bookmarked' before I evolved the URLs to a multi-tenant system. As you can tell from the host name (<code>localhost</code>), these are bookmarks against the self-hosted service. The test first verifies that the response is <code>301 Moved Permanently</code>. It then requests the new address and <a href="/2020/09/28/ensuresuccessstatuscode-as-an-assertion">uses EnsureSuccessStatusCode as an assertion</a>. </p> <h3 id="f16f3cd819804e54add83ad32aaff188"> Conclusion <a href="#f16f3cd819804e54add83ad32aaff188" title="permalink">#</a> </h3> <p> When you evolve fit URLs, it could break clients that may have bookmarked legacy URLs. Consider leaving <code>301 Moved Permanently</code> responses at those addresses. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Checking signed URLs with ASP.NET https://blog.ploeh.dk/2020/11/09/checking-signed-urls-with-aspnet 2020-11-09T12:19:00+00:00 Mark Seemann <div id="post"> <p> <em>Use a filter to check all requested URL signatures.</em> </p> <p> This article is part of <a href="/2020/10/26/fit-urls">a short series on fit URLs</a>. In the overview article, I argued that you should be signing URLs in order to prevent your REST APIs from becoming victims of <a href="https://www.hyrumslaw.com">Hyrum's law</a>. <a href="/2020/11/02/signing-urls-with-aspnet">In the previous article</a> you saw how to sign URLs with ASP.NET. </p> <p> In this article you'll see how to check the URLs of all HTTP requests to the API and reject those that aren't up to snuff. </p> <h3 id="beb942e872e6470e94777e228967b58a"> Filter <a href="#beb942e872e6470e94777e228967b58a" title="permalink">#</a> </h3> <p> If you want to intercept all incoming HTTP requests in ASP.NET Core, an <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.filters.iasyncactionfilter">IAsyncActionFilter</a> is a good option. This one should look at the URL of all incoming HTTP requests and detect if the client tried to tamper with it. </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IAsyncActionFilter</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>(<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;comes&nbsp;here...</span></pre> </p> <p> The interface only defines a single method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;OnActionExecutionAsync( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutionDelegate</span>&nbsp;next) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsGetHomeRequest(context)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;strippedUrl&nbsp;=&nbsp;GetUrlWithoutSignature(context); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(SignatureIsValid(strippedUrl,&nbsp;context)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;context.Result&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); }</pre> </p> <p> While the rule is to reject requests with invalid signatures, there's one exception. The 'home' resource requires no signature, as this is the only publicly documented URL for the API. Thus, if <code>IsGetHomeRequest</code> returns <code>true</code>, the filter invokes the <code>next</code> delegate and returns. </p> <p> Otherwise, it strips the signature off the URL and checks if the signature is valid. If it is, it again invokes the <code>next</code> delegate and returns. </p> <p> If the signature is invalid, on the other hand, the filter stops further execution by <em>not</em> invoking <code>next</code>. Instead, it sets the response to a <code>404 Not Found</code> result. </p> <p> It may seem odd to return <code>404 Not Found</code> if the signature is invalid. Wouldn't <code>401 Unauthorized</code> or <code>403 Forbidden</code> be more appropriate? </p> <p> Not really. Keep in mind that while this behaviour may use encryption technology, it's not a security feature. The purpose is to make it impossible for clients to 'retro-engineer' an implied interface. This protects them from breaking changes in the future. Clients are supposed to follow links, and the URLs given by the API itself are proper, existing URLs. If you try to edit a URL, then that URL doesn't work. It represents a resource that doesn't exist. While it may seem surprising at first, I find that a <code>404 Not Found</code> result is the most appropriate status code to return. </p> <h3 id="3584198282b64c85ab5872493da8149d"> Detecting a home request <a href="#3584198282b64c85ab5872493da8149d" title="permalink">#</a> </h3> <p> The <code>IsGetHomeRequest</code> helper method is straightforward: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsGetHomeRequest(<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;context.HttpContext.Request.Path&nbsp;==&nbsp;<span style="color:#a31515;">&quot;/&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp;&nbsp;context.HttpContext.Request.Method&nbsp;==&nbsp;<span style="color:#a31515;">&quot;GET&quot;</span>; }</pre> </p> <p> This predicate only looks at the <code>Path</code> and <code>Method</code> of the incoming request. Perhaps it also ought to check that the URL has no query string parameters, but I'm not sure if that actually matters. </p> <h3 id="c5f674b301404a2f95177d4c1dc73cc0"> Stripping off the signature <a href="#c5f674b301404a2f95177d4c1dc73cc0" title="permalink">#</a> </h3> <p> The <code>GetUrlWithoutSignature</code> method strips off the signature query string variable from the URL, but leaves everything else intact: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;GetUrlWithoutSignature(<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restOfQuery&nbsp;=&nbsp;<span style="color:#2b91af;">QueryString</span>.Create( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;context.HttpContext.Request.Query.Where(x&nbsp;=&gt;&nbsp;x.Key&nbsp;!=&nbsp;<span style="color:#a31515;">&quot;sig&quot;</span>)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;context.HttpContext.Request.GetEncodedUrl(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ub&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>(url); &nbsp;&nbsp;&nbsp;&nbsp;ub.Query&nbsp;=&nbsp;restOfQuery.ToString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ub.Uri.AbsoluteUri; }</pre> </p> <p> The purpose of removing only the <code>sig</code> query string parameter is that it restores the rest of the URL to the value that it had when it was signed. This enables the <code>SignatureIsValid</code> method to recalculate the <a href="https://en.wikipedia.org/wiki/HMAC">HMAC</a>. </p> <h3 id="4fa6c27d6b504f2baf4b27b2f431851a"> Validating the signature <a href="#4fa6c27d6b504f2baf4b27b2f431851a" title="permalink">#</a> </h3> <p> The <code>SignatureIsValid</code> method validates the signature: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;signaturesMatch&nbsp;=&nbsp;computedSignature.SequenceEqual(receivedSignature); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;signaturesMatch; }</pre> </p> <p> If the <code>receivedSignature</code> equals the <code>computedSignature</code> the signature is valid. </p> <p> This prevents clients from creating URLs based on implied templates. Since clients don't have the signing key, they can't compute a valid HMAC, and therefore the URLs they'll produce will fail the integrity test. </p> <h3 id="8c8d2409a9a647b6a4d9c2970a02a9c7"> Configuration <a href="#8c8d2409a9a647b6a4d9c2970a02a9c7" title="permalink">#</a> </h3> <p> As is the case for the URL-signing feature, you'll first need to read the signing key from the configuration system. This is the same key used to sign URLs: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;urlSigningKey&nbsp;=&nbsp;<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes( &nbsp;&nbsp;&nbsp;&nbsp;Configuration.GetValue&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">&quot;UrlSigningKey&quot;</span>));</pre> </p> <p> Next, you'll need to register the filter with the ASP.NET framework: </p> <p> <pre>services.AddControllers(opts&nbsp;=&gt;&nbsp;opts.Filters.Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>(urlSigningKey)));</pre> </p> <p> This is typically done in the <code>ConfigureServices</code> method of the <code>Startup</code> class. </p> <h3 id="e032ab8e735b4e54a733ded5d086bee8"> Conclusion <a href="#e032ab8e735b4e54a733ded5d086bee8" title="permalink">#</a> </h3> <p> With a filter like <code>UrlIntegrityFilter</code> you can check the integrity of URLs on all incoming requests to your REST API. This prevents clients from making up URLs based on an implied interface. This may seem restrictive, but is actually for their own benefit. When they can't assemble URLs from scratch, the only remaining option is to follow the links that the API provides. </p> <p> This enables you to evolve the API without breaking existing clients. While client developers may not initially appreciate having to follow links instead of building URLs out of templates, they may value that their clients don't break as you evolve the API. </p> <p> <strong>Next:</strong> <a href="/2020/11/16/redirect-legacy-urls">Redirect legacy URLs</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Signing URLs with ASP.NET https://blog.ploeh.dk/2020/11/02/signing-urls-with-aspnet 2020-11-02T08:20:00+00:00 Mark Seemann <div id="post"> <p> <em>A few Decorators is all it takes.</em> </p> <p> This article is part of <a href="/2020/10/26/fit-urls">a short series on fit URLs</a>. In the overview article, I argued that you should be signing URLs in order to prevent your REST APIs from becoming victims of <a href="https://www.hyrumslaw.com">Hyrum's law</a>. </p> <p> In this article, you'll see how to do this with ASP.NET Core 3.1, and in the next article you'll see how to check URL integrity. </p> <h3 id="8d83a9fd4dec48efa79455d6f3356726"> SigningUrlHelper <a href="#8d83a9fd4dec48efa79455d6f3356726" title="permalink">#</a> </h3> <p> I wanted the URL-signing functionality to slot into the ASP.NET framework, which supplies the <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.iurlhelper">IUrlHelper</a> interface for the purpose of creating URLs. (For example, the <a href="/2020/08/10/an-aspnet-core-url-builder">UrlBuilder I recently described</a> relies on that interface.) </p> <p> Since it's an interface, you can define a <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorator</a> around it: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IUrlHelper</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>(<span style="color:#2b91af;">IUrlHelper</span>&nbsp;inner,&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.inner&nbsp;=&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;comes&nbsp;here...</span></pre> </p> <p> As you can tell, this Decorator requires an <code>inner</code> <code>IUrlHelper</code> and a <code>urlSigningKey</code>. Most of the members just delegate to the <code>inner</code> implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsLocalUrl(<span style="color:blue;">string</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;inner.IsLocalUrl(url); }</pre> </p> <p> The <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.iurlhelper.action">Action</a> method creates URLs, so this is the method to modify: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Action(<span style="color:#2b91af;">UrlActionContext</span>&nbsp;actionContext) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;inner.Action(actionContext); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsLocalUrl(url)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;b&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ActionContext.HttpContext.Request.Host.ToUriComponent()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(b.Uri,&nbsp;url).AbsoluteUri; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ub&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>(url); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.ToBase64String( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(url))); &nbsp;&nbsp;&nbsp;&nbsp;ub.Query&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">QueryString</span>(ub.Query).Add(<span style="color:#a31515;">&quot;sig&quot;</span>,&nbsp;sig).ToString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ub.ToString(); }</pre> </p> <p> The <code>actionContext</code> may sometimes indicate a local (relative) URL, in which case I wanted to convert it to an absolute URL. Once that's taken care of, the method calculates an <a href="https://en.wikipedia.org/wiki/HMAC">HMAC</a> and adds it as a query string variable. </p> <h3 id="37abf2a288444c9cab0541f6acafda8b"> SigningUrlHelperFactory <a href="#37abf2a288444c9cab0541f6acafda8b" title="permalink">#</a> </h3> <p> While it's possible take ASP.NET's default <code>IUrlHelper</code> instance (e.g. from <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.url">ControllerBase.Url</a>) and manually decorate it with <code>SigningUrlHelper</code>, that doesn't slot seamlessly into the framework. </p> <p> For example, to add the <code>Location</code> header that you saw in <a href="/2020/10/26/fit-urls">the previous article</a>, the code is this: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;Reservation201Created( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;r) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CreatedAtActionResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(Get), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;restaurantId,&nbsp;id&nbsp;=&nbsp;r.Id.ToString(<span style="color:#a31515;">&quot;N&quot;</span>)&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;r.ToDto()); }</pre> </p> <p> The method just returns a new <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.createdatactionresult">CreatedAtActionResult</a> object, and the framework takes care of the rest. No explicit <code>IUrlHelper</code> object is used, so there's nothing to manually decorate. By default, then, the URLs created from such <code>CreatedAtActionResult</code> objects aren't signed. </p> <p> It turns out that the ASP.NET framework uses an interface called <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.routing.iurlhelperfactory">IUrlHelperFactory</a> to create <code>IUrlHelper</code> objects. Decorate that as well: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>(<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;inner,&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.inner&nbsp;=&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;GetUrlHelper(<span style="color:#2b91af;">ActionContext</span>&nbsp;context) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;inner.GetUrlHelper(context); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>(url,&nbsp;urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Straightforward: use the <code>inner</code> object to get an <code>IUrlHelper</code> object, and decorate it with <code>SigningUrlHelper</code>. </p> <h3 id="92c13952cd48418c81c6925d9af15635"> Configuration <a href="#92c13952cd48418c81c6925d9af15635" title="permalink">#</a> </h3> <p> The final piece of the puzzle is to tell the framework about the <code>SigningUrlHelperFactory</code>. You can do this in the <code>Startup</code> class' <code>ConfigureServices</code> method. </p> <p> First, read the signing key from the configuration system (e.g. a configuration file): </p> <p> <pre><span style="color:blue;">var</span>&nbsp;urlSigningKey&nbsp;=&nbsp;<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes( &nbsp;&nbsp;&nbsp;&nbsp;Configuration.GetValue&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">&quot;UrlSigningKey&quot;</span>));</pre> </p> <p> Then use the signing key to configure the <code>SigningUrlHelperFactory</code> service. Here, I wrapped that in a little helper method: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ConfigureUrSigning( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IServiceCollection</span>&nbsp;services, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) { &nbsp;&nbsp;&nbsp;&nbsp;services.RemoveAll&lt;<span style="color:#2b91af;">IUrlHelperFactory</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;services.AddSingleton&lt;<span style="color:#2b91af;">IUrlHelperFactory</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlHelperFactory</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;urlSigningKey)); }</pre> </p> <p> This method first removes the default <code>IUrlHelperFactory</code> service and then adds the <code>SigningUrlHelperFactory</code> instead. It decorates <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.routing.urlhelperfactory">UrlHelperFactory</a>, which is the default built-in implementation of the interface. </p> <h3 id="2efe02caa55742679a953a398c235bc5"> Conclusion <a href="#2efe02caa55742679a953a398c235bc5" title="permalink">#</a> </h3> <p> You can extend the ASP.NET framework to add a signature to all the URLs it generates. All it takes is two simple Decorators. </p> <p> <strong>Next:</strong> <a href="/2020/11/09/checking-signed-urls-with-aspnet">Checking signed URLs with ASP.NET</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Fit URLs https://blog.ploeh.dk/2020/10/26/fit-urls 2020-10-26T06:19:00+00:00 Mark Seemann <div id="post"> <p> <em>Keep REST API URLs evolvable. A way to address Hyrum's law.</em> </p> <p> Publishing and maintaining a true (<a href="https://martinfowler.com/articles/richardsonMaturityModel.html">level 3</a>) RESTful API is difficult. This is the style of REST design where clients are expected to <em>follow links</em> to perform the work they want to accomplish; not assemble URLs from templates. </p> <p> Have you ever designed a URL scheme and published it, only to later discover that you wished you'd come up with a different structure? If you've published a set of URL templates, changing your mind constitutes a breaking change. If clients follow links, however, URLs are opaque and you can redesign the URLs without breaking existing clients. </p> <p> You can try to document this design principle all you want, to no avail. You can <em>tell</em> client developers that they're supposed to follow links, not try to <a href="/2013/05/01/rest-lesson-learned-avoid-hackable-urls">retro-engineer the URLs</a>, and still they'll do it. </p> <p> I know; I've experienced it. When we later changed the URL structure, it didn't take long for the client developers to complain that we broke their code. </p> <h3 id="f9591f3f57534243afc883e166727635"> Hyrum's law <a href="#f9591f3f57534243afc883e166727635" title="permalink">#</a> </h3> <p> This is an example of <a href="https://www.hyrumslaw.com">Hyrum's law</a> in action, albeit on the scale of web service interactions, rather than low-level APIs. The presence of a discernible system to URLs suggests an <em>implicit</em> interface. </p> <p> Consider this 'home' resource for an online restaurant reservation system: </p> <p> <pre>GET / HTTP/1.1 HTTP/1.1 200 OK Content-Type: application/json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/reservations&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/10&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/10/23&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> It doesn't take much interaction with the API before you realise that there's a system to the URLs provided in the links. If you want to see the calendar for a specific date, you can easily retro-engineer the URL template <code>/calendar/{yyyy}/{MM}/{dd}</code>, and <code>/calendar/{yyyy}/{MM}</code> for a month, and so on. </p> <p> The same is likely to happen with the <em>reservations</em> link. You can <code>POST</code> to this link to make a new reservation: </p> <p> <pre>POST /reservations HTTP/1.1 Content-Type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-12-09 19:15&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;rainboughs@example.com&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Raine&nbsp;Burroughs&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;5 } HTTP/1.1 201 Created Content-Type: application/json Location: http://localhost:53568/reservations/fabc5bf63a1a4db38b95deaa89c01178 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;fabc5bf63a1a4db38b95deaa89c01178&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-12-09T19:15:00.0000000&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;rainboughs@example.com&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Raine&nbsp;Burroughs&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;5 }</pre> </p> <p> Notice that when the API responds, its <code>Location</code> header gives you the URL for that particular reservation. It doesn't take long to figure out that there's a template there, as well: <code>/reservations/{id}</code>. </p> <p> So client developers may just store the ID (<code>fabc5bf63a1a4db38b95deaa89c01178</code>) and use the implied template to construct URLs on the fly. And who can blame them? </p> <p> That, however, misses the point of REST. The ID of that reservation isn't <code>fabc5bf63a1a4db38b95deaa89c01178</code>, but rather <code>http://localhost:53568/reservations/fabc5bf63a1a4db38b95deaa89c01178</code>. Yes, the URL, all of it, is the ID. </p> <h3 id="fff55f373ff24a4da6cd4e50dd8b97bd"> Evolving URLs <a href="#fff55f373ff24a4da6cd4e50dd8b97bd" title="permalink">#</a> </h3> <p> Why does that matter? </p> <p> It matters because you're human, and you make mistakes. Or, rather, it's intrinsic to software development that you learn as you work. You'll make decisions at the beginning that you'll want to change as you gain more insight. </p> <p> Also, requirements change. Consider the URL template scheme implied by the above examples. Can you spot any problems? Would you want to change anything? </p> <p> Imagine, for example, that you've already deployed the first version of the API. It's a big success. Now the product owner wants to expand the market to more restaurants. She wants to make the service a multi-tenant API. How does that affect URLs? </p> <p> In that new context, perhaps URLs like <code>/restaurants/1/reservations</code> or <code>/restaurants/90125/calendar/2020/10</code> would be better. </p> <p> That, however, would be a breaking change if clients construct URLs based on implied templates. </p> <p> Couldn't you just pass the restaurant ID as an HTTP header instead of in the URL? Yes, technically you could do that, but that doesn't work well with HTTP caching. It's not a RESTful thing to do, for that, and other, reasons. </p> <h3 id="41cdd42bac1b4bd5a573070ee27e902d"> Fitness <a href="#41cdd42bac1b4bd5a573070ee27e902d" title="permalink">#</a> </h3> <p> Do we just give up in the face of Hyrum's law? Or can we keep URLs evolvable? In evolution, organisms evolve according to a 'fitness function', so to name such URLs, we could call them <em>fit URL</em>. </p> <p> To keep URLs fit, we must prevent client developers from retro-engineering the implied interface. My original thought was to give each URL an opaque ID, such as a GUID, but <a href="/2013/05/01/rest-lesson-learned-avoid-hackable-urls#424bd3f6199e422c8294e300d312ffb4">in 2015 Dan Kubb instead suggested to sign the URLs</a>. What a great idea! </p> <p> If you do that, then the above home resource might look like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020?sig=eIFuUkb6WprPrp%2B4HPSPaavcUdwVjeG%2BKVrIRqDs9OI%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020/10?sig=mGqkAjY7vMbC5Fr7UiRXWWnjn3pFn21MYrMagpdWaU0%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020/10/23?sig=Ua5F%2FucP6zmAy219cHa4WG7zIcCa0hgVD5ModXcNQuo%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> Even if you can still figure out what the URL templates are, it doesn't avail you. Creating a new reservation may return a URL like <code>https://localhost:53568/restaurants/1/reservations/fabc5bf63a1a4db38b95deaa89c01178?sig=8e80PmVi8aSS1UH6iSJ73nHmOsCLrUMs7yggEOkvEqo%3D</code>, but you can't just replace the ID with another ID and expect it to work: </p> <p> <pre>GET /restaurants/1/reservations/79520877ef4f4acdb69838e22ad04510?sig=8e80PmVi8aSS1UH6iSJ73nHmOsCLrUMs7yggEOkvEqo%3D HTTP/1.1 HTTP/1.1 404 Not Found</pre> </p> <p> You're requesting a URL that doesn't exist, so the result is <code>404 Not Found</code>. To be clear: yes, there <em>is</em> a reservation with the ID <code>79520877ef4f4acdb69838e22ad04510</code>, but its URL isn't the above URL. </p> <h3 id="104b052da10e463297020c109dec705d"> ASP.NET implementation <a href="#104b052da10e463297020c109dec705d" title="permalink">#</a> </h3> <p> In two articles, I'll show you how to implement <em>fit URLs</em> in ASP.NET Core. <ul> <li><a href="/2020/11/02/signing-urls-with-aspnet">Signing URLs with ASP.NET</a></li> <li><a href="/2020/11/09/checking-signed-urls-with-aspnet">Checking signed URLs with ASP.NET</a></li> <li><a href="/2020/11/16/redirect-legacy-urls">Redirect legacy URLs</a></li> </ul> The ASP.NET framework comes with enough extensibility points to make this a non-intrusive operation. I implemented this using a filter and a few <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorators</a> in a way so that you can easily turn the feature on or off. </p> <h3 id="ff0ced83b9fb49e8835fc92e08dd2859"> Conclusion <a href="#ff0ced83b9fb49e8835fc92e08dd2859" title="permalink">#</a> </h3> <p> One of the major benefits of true RESTful API design is that it's evolvable. It enables you to learn and improve as you go, without breaking existing clients. </p> <p> You have to take care, however, that clients don't retro-engineer the URL templates that you may be using for implementation purposes. You want to be able to change URLs in the future. </p> <p> Hyrum's law suggests that clients will rely on undocumented features if they can. By signing the URLs you keep them fit to evolve. </p> <p> <strong>Next:</strong> <a href="/2020/11/02/signing-urls-with-aspnet">Signing URLs with ASP.NET</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="b0fb43ff9aba4c14a075e7effc9fae25"> <div class="comment-author"><a href="https://github.com/Skleni">Daniel Sklenitzka</a></div> <div class="comment-content"> <p> I see how signing the URLs prevents clients from retro-engineering the URL templates, but how does it help preventing breaking changes? If the client stores the whole URL instead of just the ID and later the URL changes because a restaurant ID is added, the original URL is still broken, isn't it? </p> </div> <div class="comment-date">2020-10-27 07:20 UTC</div> </div> <div class="comment" id="848275a366db40e2a35810554633447f"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Daniel, thank you for writing. You're correct when assuming that clients are allowed to 'bookmark' URLs. An implicit assumption that I didn't explicitly state is that clients are supposed to follow not only links, but also redirects. Thus, to avoid breaking changes, it's the API's responsibility to leave a <code>301 Moved Permanently</code> response behind at the old address. </p> <p> As a service owner, though, you have some flexibility in how to achieve this. You can code this into the service code itself, but another option might be to use a reverse proxy for such purposes. One of the many advantages of REST is that you can offload a lot HTTP-level behaviour on standard networking software; <a href="/2020/06/01/retiring-old-service-versions#bc73620aa71141b6a74f4a4aaf395d75">here's another example</a>. </p> </div> <div class="comment-date">2020-10-28 7:15 UTC</div> </div> <div class="comment" id="880c4cda8ab14b1d8de05438becac2f1"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> I think I have the same confusion as Daniel. </p> <blockquote> <p> If you've published a set of URL templates, changing your mind constitutes a breaking change. If clients follow links, however, URLs are opaque and you can redesign the URLs without breaking existing clients. </p> <p> ... </p> <p> You're correct when assuming that clients are allowed to 'bookmark' URLs. An implicit assumption that I didn't explicitly state is that clients are supposed to follow not only links, but also redirects. Thus, to avoid breaking changes, it's the API's responsibility to leave a <code>301 Moved Permanently</code> response behind at the old address. </p> </blockquote> <p> Is there a certain kind of breaking change that exists for a level 2 API that doesn't exist for a level 3 API? </p> </div> <div class="comment-date">2020-10-28 22:55 UTC</div> </div> <div class="comment" id="aae1427ebab14c428291984efb06fd94"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I haven't thought much about whether there are categories of errors that differ between the levels, but I think that in practice, there's a difference in mindset. </p> <p> With a level 2 API, I think most people are still caught in a mindset that's largely a projection of RPC. URL templates map easily to programming procedures (e.g. object-oriented methods). Thus, you have a mental model that includes <code>PostReservation</code>, <code>GetCalendar</code>, etc. People think of <em>that</em> as being the API. With that mindset, I don't think that many client developers configure their HTTP clients to follow redirects. Thus, one could argue that changing URLs are breaking changes for a level 2 API, even if you leave <code>301 Moved Permanently</code> responses behind. </p> <p> With level 3 APIs, you encourage client developers to think in terms of 'the web'. That includes following links and redirects. I believe that there's a difference in perception, even if there may not be any technical difference. </p> <p> I do believe, however, that the real advantage is that you impose a smaller maintenance burden on client developers with a level 3 API. Granted, a client developer may have to spend a little more effort up front to follow links, but once a compliant client is in place, it <em>needs no more maintenance</em>. It'll just keep working. </p> <p> Not so with published URL templates. Here you have to entice client developers to <em>actively</em> update their code. This may be impossible if you don't know who the clients are, or if the client software is in maintenance mode. This may make it harder to <a href="/2020/06/01/retiring-old-service-versions">retire old versions</a>. You may be stuck with these old versions forever. </p> </div> <div class="comment-date">2020-10-30 17:32 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Monomorphic functors https://blog.ploeh.dk/2020/10/19/monomorphic-functors 2020-10-19T07:36:00+00:00 Mark Seemann <div id="post"> <p> <em>Non-generic mappable containers, with a realistic example.</em> </p> <p> This article is an instalment in <a href="/2018/03/22/functors">an article series about functors</a>. Previous articles have covered <a href="/2018/03/26/the-maybe-functor">Maybe</a>, <a href="/2018/09/10/the-lazy-functor">Lazy</a>, and other functors. This article looks at what happens when you weaken one of the conditions that the article series so far has implied. </p> <p> In the <a href="/2018/03/22/functors">introductory article</a>, I wrote: <blockquote> As a rule of thumb, if you have a type with a generic type argument, it's a candidate to be a functor. </blockquote> That still holds, but then <a href="https://about.me/tysonwilliams">Tyson Williams</a> <a href="/2018/03/22/functors#dee7ec216a464248b6103ea0979948ab">asks if that's a required condition</a>. It turns out that it isn't. In this article, you'll learn about the implications of weakening this condition. </p> <p> As is my habit with many of the articles in this article series, I'll start by uncovering the structure of the concept, and only later show a more realistic example. </p> <h3 id="bea56fadde1e4e9da8389a6c31937216"> Mapping strings <a href="#bea56fadde1e4e9da8389a6c31937216" title="permalink">#</a> </h3> <p> So far in this article series, you've seen examples of <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">containers</a> with a generic type argument: <code>Tree&lt;T&gt;</code>, <code>Task&lt;T&gt;</code>, and so on. Furthermore, you've seen how a functor is a container with a <em>structure-preserving map</em>. This function has various names in different languages: <code>Select</code>, <code>fmap</code>, <code>map</code>, etcetera. </p> <p> Until now, you've only seen examples where this mapping enables you to translate from one type argument to another. You can, for example, translate the characters in a string to Boolean values, like this: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()) Enumerable.WhereSelectEnumerableIterator&lt;char, bool&gt; &nbsp;&nbsp;{ false, true, false, true, false, false, false, true, false, false, false, true, false, false }</pre> </p> <p> This works because in C#, <a href="https://docs.microsoft.com/dotnet/api/system.string">the String class</a> implements various interfaces, among these <code>IEnumerable&lt;char&gt;</code>. By treating a <code>string</code> as an <code>IEnumerable&lt;char&gt;</code>, you can map each element. That's the standard IEnumerable functor (AKA <em>the list functor</em>). </p> <p> What if you'd like to map the characters in a string to other characters? Perhaps you'd like to map vowels to upper case, and all other characters to lower case. You could try this: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)) Enumerable.WhereSelectEnumerableIterator&lt;char, char&gt; &nbsp;&nbsp;{ 's', 'A', 'f', 'E', ' ', 'f', 'r', 'O', 'm', ' ', 'h', 'A', 'r', 'm' }</pre> </p> <p> That sort of works, but as you can tell, the result isn't a <code>string</code>, it's an <code>IEnumerable&lt;char&gt;</code>. </p> <p> This isn't a big problem, because one of the <code>string</code> constructor overloads take a <code>char</code> array as input, so you can do this: </p> <p> <pre>&gt; <span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>&nbsp;(<span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)).<span style="color:#2b91af;">ToArray</span>()) "sAfE frOm hArm"</pre> </p> <p> It isn't the prettiest, but it gets the job done. </p> <h3 id="fd3da7dcd560415d81ab82bfb772d94c"> Monomorphic functor in C# <a href="#fd3da7dcd560415d81ab82bfb772d94c" title="permalink">#</a> </h3> <p> If you contemplate the last example, you may arrive at the conclusion that you could package some of that boilerplate code in a reusable function. Since we're already talking about functors, why not call it <code>Select</code>? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Select(<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">string</span>&nbsp;source,&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:blue;">char</span>,&nbsp;<span style="color:blue;">char</span>&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>(source.AsEnumerable().Select(selector).ToArray()); }</pre> </p> <p> It somewhat simplifies things: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.Select(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)) "sAfE frOm hArm"</pre> </p> <p> Since I deliberately wrote the <code>Select</code> method in the style of other <code>Select</code> methods (apart from the generics), you may wonder if C# query syntax also works? </p> <p> <pre>&gt; <span style="color:blue;">from</span>&nbsp;c&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#a31515;">&quot;Army&nbsp;of&nbsp;Me&quot;</span> . <span style="color:blue;">select</span>&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c) "ArmY Of mE"</pre> </p> <p> It compiles and works! The C# compiler understands monomorphic containers! </p> <p> I admit that I was quite surprised when I first tried this out. </p> <h3 id="8ef3808c11a34d9aaa1c25b94539f843"> Monomorphic functor in Haskell <a href="#8ef3808c11a34d9aaa1c25b94539f843" title="permalink">#</a> </h3> <p> Surprisingly, in this particular instance, C# comes out looking more flexible than <a href="https://www.haskell.org">Haskell</a>. This is mainly because in C#, functors are implemented as a special compiler feature, whereas in Haskell, <code>Functor</code> is defined using the general-purpose <em>type class</em> language feature. </p> <p> There's <a href="https://hackage.haskell.org/package/mono-traversable">a package</a> that defines a <code>MonoFunctor</code> type class, as well as some instances. With it, you can write code like this: </p> <p> <pre><span style="color:#2b91af;">ftg</span>&nbsp;::&nbsp;<span style="color:blue;">Text</span> ftg&nbsp;=&nbsp;omap&nbsp;(\c&nbsp;-&gt;&nbsp;<span style="color:blue;">if</span>&nbsp;isVowel&nbsp;c&nbsp;<span style="color:blue;">then</span>&nbsp;toUpper&nbsp;c&nbsp;<span style="color:blue;">else</span>&nbsp;toLower&nbsp;c)&nbsp;<span style="color:#a31515;">&quot;Fade&nbsp;to&nbsp;Grey&quot;</span></pre> </p> <p> Even though <code>Text</code> isn't a <code>Functor</code> instance, it <em>is</em> a <code>MonoFunctor</code> instance. The value of <code>ftg</code> is <code>"fAdE tO grEY"</code>. </p> <p> All the normal functors (<code>[]</code>, <code>Maybe</code>, etc.) are also <code>MonoFunctor</code> instances, since the normal <code>Functor</code> instance is more capable than a <code>MonoFunctor</code>. </p> <h3 id="30dd605a3bb34d1985e1d277e1c9b0a5"> Restaurant example <a href="#30dd605a3bb34d1985e1d277e1c9b0a5" title="permalink">#</a> </h3> <p> While I've introduced the concept of a monomorphic functor with a trivial string example, I actually discovered the C# feature when I was working on some more realistic code. As I often do, I was working on a variation of an online restaurant reservation system. </p> <p> The code base contained a rather complex variation of an implementation of the <a href="/2020/01/27/the-maitre-d-kata">Maître d' kata</a>. The <code>MaitreD</code> constructor looked like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;opensAt, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;lastSeating, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>&nbsp;seatingDuration, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;tables)</pre> </p> <p> This had worked well when the system only dealt with a single restaurant, but I was now expanding it to a multi-tenant system and I needed to keep track of some more information about each restaurant, such as its name and ID. While I could have added such information to the <code>MaitreD</code> class, I didn't want to pollute that class with data it didn't need. While the restaurant's opening time and seating duration are necessary for the decision algorithm, the name isn't. </p> <p> So I introduced a wrapper class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Restaurant</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(<span style="color:blue;">int</span>&nbsp;id,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:#2b91af;">MaitreD</span>&nbsp;maitreD) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MaitreD&nbsp;=&nbsp;maitreD; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Name&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>&nbsp;MaitreD&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;follows...</span></pre> </p> <p> I also added copy-and-update methods (AKA 'withers'): </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithId(<span style="color:blue;">int</span>&nbsp;newId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(newId,&nbsp;Name,&nbsp;MaitreD); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithName(<span style="color:blue;">string</span>&nbsp;newName) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(Id,&nbsp;newName,&nbsp;MaitreD); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithMaitreD(<span style="color:#2b91af;">MaitreD</span>&nbsp;newMaitreD) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(Id,&nbsp;Name,&nbsp;newMaitreD); }</pre> </p> <p> Still, if you need to modify the <code>MaitreD</code> within a given <code>Restaurant</code> object, you first have to have a reference to the <code>Restaurant</code> so that you can read its <code>MaitreD</code> property, then you can edit the <code>MaitreD</code>, and finally call <code>WithMaitreD</code>. Doable, but awkward: </p> <p> <pre>restaurant.WithMaitreD(restaurant.MaitreD.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5)))</pre> </p> <p> So I got the idea that I might try to add a structure-preserving map to <code>Restaurant</code>, which I did: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;Select(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">MaitreD</span>,&nbsp;<span style="color:#2b91af;">MaitreD</span>&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(selector&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(selector)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;WithMaitreD(selector(MaitreD)); }</pre> </p> <p> The first time around, it enabled me to rewrite the above expression as: </p> <p> <pre>restaurant.Select(m&nbsp;=&gt;&nbsp;m.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5)))</pre> </p> <p> That's already a little nicer. It also handles the situation where you may not have a named <code>Restaurant</code> variable you can query; e.g. if you have a method that returns a <code>Restaurant</code> object, and you just want to continue calling into a <a href="https://martinfowler.com/bliki/FluentInterface.html">Fluent API</a>. </p> <p> Then I thought, <em>I wonder if query syntax works, too...</em> </p> <p> <pre><span style="color:blue;">from</span>&nbsp;m&nbsp;<span style="color:blue;">in</span>&nbsp;restaurant <span style="color:blue;">select</span>&nbsp;m.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5))</pre> </p> <p> And it <em>does</em> work! </p> <p> I know that a lot of people don't like query syntax, but I think it has certain advantages. In this case, it actually isn't shorter, but it's a nice alternative. I particularly find that if I try to <a href="/2019/11/04/the-80-24-rule">fit my code into a tight box</a>, query syntax sometimes gives me an opportunity to format code over multiple lines in a way that's more natural than with method-call syntax. </p> <h3 id="c386660f74424f33921f35265e61ecf0"> Conclusion <a href="#c386660f74424f33921f35265e61ecf0" title="permalink">#</a> </h3> <p> A monomorphic functor is still a functor, only it constrains the type of mapping you can perform. It can be useful to map monomorphic containers like strings, or immutable nested data structures like the above <code>Restaurant</code> class. </p> <p> <strong>Next:</strong> <a href="/2018/12/03/set-is-not-a-functor">Set is not a functor</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bb0bb9d1bf094c1897f9dd42fb49712c"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Excellent article! I was planning to eventually write a blog post on this topic, but now there is no need. </p> <blockquote> <p> I know that a lot of people don't like query syntax, but I think it has certain advantages. In this case, it actually isn't shorter, but it's a nice alternative. I particularly find that if I try to <a href="/2019/11/04/the-80-24-rule">fit my code into a tight box</a>, query syntax sometimes gives me an opportunity to format code over multiple lines in a way that's more natural than with method-call syntax. </p> </blockquote> <p> Said another way, one objective measure that is better in this case when using query syntax is that the maximum line width is smaller. It is objective primarily in the sense that maximum line width is objective and secondarily in the sense that we generally agree that code with a lower maximum line width is typically easier to understand. </p> <p> Another such objective measure of quality that I value is the maximum number of matching pairs of parentheses that are nested. This measure improves to two in your query syntax code from three in your previous code. The reason it was three before is because there are four levels of detail and your code decreases the level of detail three times: <code>Restaurant</code> to <code>MaitreD</code>, <code>MaitreD</code> to <code>TimeSpan</code>, and <code>TimeSpan</code> to <code>double</code>. Query syntax is one way to decrease the level of detail without having to introduce a pair of matching parentheses. </p> <p> I find more success minimizing this measure of quality when taking a bottom-up approach. Using the <a href="https://github.com/louthy/language-ext/blob/3df8c837961a502d7a03268b6cac636a57c49056/LanguageExt.Core/DataTypes/Cond/Cond.cs#L56-L70"><code>Apply</code> extension method from <code>language-ext</code></a>, which is like the <a href="https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/symbol-and-operator-reference/#table-of-symbols-and-operators:~:text=Passes%20the%20result%20of%20the%20left%20side%20to%20the%20function%20on%20the%20right%20side%20(forward%20pipe%20operator).">forward pipe operator in F#</a>, we can rewrite this code without any nested pairs of matching parentheses as </p> <p> <pre>.5.Apply(<span style="color:#2b91af;">TimeSpan</span>.FromHours).Apply(m&nbsp;=&gt;&nbsp;m.WithSeatingDuration).Apply(restaurant.Select)</pre> </p> <p> (I did not check if this code compiles. If it does not, it would be because the C# compiler is unsure how to implicitly convert some method group to its intended <code>Func&lt;,&gt;</code> delegate.) This code also has natural line-breaking points before each dot operator, which leads to a comfortable value for the maximum line width. </p> <p> Another advantage of this code that I value is that the execution happens in the same order in which I (as an English speaker) read it: left to right. Your code before using the query syntax executed from right to left as dictated by the matching parentheses. In fact, since the dot operator is left associative, the degree to which execution occurs from left to right is inversely correlated to the number of nested pairs of parentheses. (One confusing thing here is the myriad of different semantic meanings in C# to the syntactic use of a pair of parentheses.) </p> </div> <div class="comment-date">2020-10-19 14:05 UTC</div> </div> <div class="comment" id="8e6b870ce4aa44aa98a16a7969def6e9"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I've never thought about measuring complexity by nested parentheses, but I like it! It makes intuitive sense. </p> <p> I'm not sure it applies to LISP-like languages, but perhaps some reader comes by some day who can enlighten us. </p> </div> <div class="comment-date">2020-10-21 13:55 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Subjectivity https://blog.ploeh.dk/2020/10/12/subjectivity 2020-10-12T07:40:00+00:00 Mark Seemann <div id="post"> <p> <em>A subjective experience can be sublime, but it isn't an argument.</em> </p> <p> I once wrote <a href="https://amzn.to/36xLycs">a book</a>. One of the most common adjectives people used about it was <em>opinionated</em>. That's okay. It is, I think, a ramification of <a href="/2020/04/13/curb-code-rot-with-thresholds#6c5e4b0817f742bf9ecc0a9050ecb7b5">the way I write</a>. I prioritise writing so that you understand what I mean, and why I mean it. You can call that opinionated if you like. </p> <p> This means that I frequently run into online arguments, mostly on Twitter. This particularly happens when I write something that readers don't like. </p> <h3 id="6a04cf26fdfc483db096de13451a7273"> The sublime <a href="#6a04cf26fdfc483db096de13451a7273" title="permalink">#</a> </h3> <p> Whether or not you <em>like</em> something is a subjective experience. I have nothing against subjective experiences. </p> <p> Subjective experiences fuel art. Before I became a programmer, I wanted to be an artist. I've loved <a href="https://en.wikipedia.org/wiki/Bande_dessin%C3%A9e">French-Belgian</a> comics for as long as I remember (my favourites have always been <a href="https://en.wikipedia.org/wiki/Hermann_Huppen">Hermann</a> and <a href="https://en.wikipedia.org/wiki/Jean_Giraud">Jean Giraud</a>), and until I gave up in my mid-twenties, I wanted to become a graphic novelist. </p> <p> <img src="/content/binary/graphic-novel-pages.jpg" alt="Spread of original pages of my graphic novel."> </p> <p> This interest in drawing also fuelled a similar interest in paintings and art in general. I've had many sublime experiences in front of a work of art in various museums around the world. These were all subjective experiences that were precious to me. </p> <p> For as long as I've been interested in graphic arts, I've also loved music. I've worn <a href="https://en.wikipedia.org/wiki/LP_record">LPs</a> thin in the 1970s and '80s. I've played guitar and wanted to be a rock star. I regularly get goosebumps from listing to outstanding music. These are all subjective experiences. </p> <p> Having an experience is part of the human condition. </p> <p> How you <em>feel</em> about something, however, isn't an argument. </p> <h3 id="c60fc277c32d4e67be30f81d7434da51"> Arguing about subjective experience <a href="#c60fc277c32d4e67be30f81d7434da51" title="permalink">#</a> </h3> <p> I rarely read music reviews. How you experience music is such a personal experience that I find it futile to argue about it. Maybe you share my feelings that <a href="https://en.wikipedia.org/wiki/Army_of_Me">Army of Me</a> is a sublime track, or maybe it leaves you cold. If you don't like that kind of music, I can't argue that you should. </p> <p> Such a discussion generally takes place at the kindergarten level: <em>"Army of Me is a great song!</em>, <em>"No it isn't,"</em> <em>"Yes it is,"</em> and so on. </p> <p> This doesn't mean that we can't talk about music (or, by extension, other subjective experiences). Talking about music can be exhilarating, particularly when you meet someone who shares your taste. Finding common ground, exploring and inspiring each other, learning about artists you didn't know about, is wonderful. It forges bonds between people. I suppose it's the same mechanism in which sports fans find common ground. </p> <p> Sharing subjective experience may not be entirely futile. You can't use your <em>feelings</em> as an argument, but explaining <em>why</em> you feel a certain way can be illuminating. This is the realm of much art criticism. A good, popular and easily digestible example is the <a href="https://strongsongspodcast.com">Strong Songs podcast</a>. The host, Kirk Hamilton, likes plenty of music that I don't. I must admit, though, that his <a href="https://strongsongspodcast.com/satisfied-from-hamilton">exigesis of <em>Satisfied</em> from Hamilton</a> left me with more appreciation for that song than I normally have for that genre. </p> <p> This is why, in a technical argument, when people say, <em>"I don't like it"</em>, I ask <em>why?</em> </p> <p> Your <em>preference</em> doesn't convince me to change my mind, but if you can explain your perspective, it may make me consider a point of which I wasn't aware. </p> <h3 id="8759a0fb5e2f496ebc7d264072906648"> It Depends™ <a href="#8759a0fb5e2f496ebc7d264072906648" title="permalink">#</a> </h3> <p> Among other roles, I consult. Consultants are infamous for saying <em>it depends</em>, and obviously, if that's all we say, it'd be useless. </p> <p> Most advice is given in a context. When a consultant starts a sentence with <em>it depends</em>, he or she must continue by explaining on what it depends. Done right, this can be tremendously valuable, because it gives the customer a framework they can use to make decisions. </p> <p> In the same family as <em>it depends</em>, I often run into arguments like <em>nothing is black or white</em>, <em>it's a trade-off</em>, or other platitudes. Okay, it probably <em>is</em> a trade-off, but between <em>what?</em> </p> <p> If it's a trade-off between my reasoned argument on the one hand, and someone's preference on the other hand, my reasoned argument wins. </p> <p> Why? Because I can always trump with <em>my</em> preference as well. </p> <h3 id="6dd1b6841a5d42c8b295432859c5187d"> Conclusion <a href="#6dd1b6841a5d42c8b295432859c5187d" title="permalink">#</a> </h3> <p> Our subjective experiences are important pieces of our lives. I hope that my detour around my love of art highlighted that I consider subjective experiences vital. </p> <p> When we discuss how to do things, however, a <em>preference</em> isn't an argument. You may prefer to do things in one way, and I another. Such assertions, alone, don't move the discussion anywhere. </p> <p> What we <em>can</em> do is to attempt to uncover the underlying causes for our preferences. There's no guarantee that we'll reach enlightenment, but it's certain that we'll get nowhere if we don't. </p> <p> That's the reason that, when someone states <em>"I prefer"</em>, I often ask <em>why?</em> </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="f826ff3d591f4d6ebb1e7d93c00cea3a"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> <p> Consultants are infamous for saying <em>it depends</em>, and obviously, if that's all we say, it'd be useless. </p> <p> Most advice is given in a context. When a consultant starts a sentence with <em>it depends</em>, he or she must continue by explaining on what it depends. Done right, this can be tremendously valuable, because it gives the customer a framework they can use to make decisions. </p> </blockquote> <p> I have a vague memory of learning this through some smoothly worded saying. As I recall, it went something like this. </p> <blockquote> Anyone can say "It depends", but an expert can say on what it depends. </blockquote> <p> I don't think that is the exact phrase, because I don't think it sounds very smooth. Does anyone know of a succinct and smooth mnemonic phrase that captures this idea? </p> </div> <div class="comment-date">2020-10-12 13:54 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Fortunately, I don't squash my commits https://blog.ploeh.dk/2020/10/05/fortunately-i-dont-squash-my-commits 2020-10-05T06:00:00+00:00 Mark Seemann <div id="post"> <p> <em>The story of a bug, and how I addressed it.</em> </p> <p> Okay, I admit it: I could have given this article all sorts of alternative titles, each of which would have made as much sense as the one I chose. I didn't want to go with some of the other titles I had in mind, because they would give it all away up front. I didn't want to spoil the surprise. </p> <p> I recently ran into this bug, it took me hours to troubleshoot it, and I was appalled when I realised what the problem was. </p> <p> This is the story of that bug. </p> <p> There are several insights from this story, and I admit that I picked the most click-baity one for the title. </p> <h3 id="94bc204499df483897121bb0343f73f7"> Yak shaving <a href="#94bc204499df483897121bb0343f73f7" title="permalink">#</a> </h3> <p> I was working on the umpteenth variation of an online restaurant reservations system, and one of the features I'd added was a schedule only visible to the <a href="https://en.wikipedia.org/wiki/Ma%C3%AEtre_d%27h%C3%B4tel">maître d'</a>. The schedule includes a list of all reservations for a day, including guests' email addresses and so on. For that reason, I'd protected that resource by requiring a valid <a href="https://en.wikipedia.org/wiki/JSON_Web_Token">JSON Web Token</a> (JWT) with an appropriate role. </p> <p> I'd deployed a new version of the API and went for an ad-hoc test. To my surprise, that particular resource didn't work. When I tried to request it, I got a <code>403 Forbidden</code> response. </p> <p> "That's odd," I though, "it worked the last time I tried this." </p> <p> The system is set up with continuous deployment. I push <em>master</em> to a remote repository, and a build pipeline takes over from there. It only deploys the new version if all tests pass, so my first reaction was that I might have made a mistake with the JWT. </p> <p> I wasted significant time decoding the JWT and comparing its contents to what it was supposed to be. I couldn't find any problems. </p> <p> I also meticulously compared the encryption key I'd used to sign the JWT with the key on the server. They were identical. </p> <p> Incredulous, and running out of ideas, I tried running all tests on my development machine. Indeed, all 170 tests passed. </p> <p> Finally, I gave up and ran the API on my development machine. It takes all of a 30 seconds to configure the code to run in that environment, so you're excused if you wonder why I didn't already do that. What can I say? I've almost two decades of experience with automated test suites. Usually, if all tests pass, the problem is environmental: a network topology issue, a bad or missing connection string, a misconfigured encryption key, an invalid JWT, things like that. </p> <p> To my surprise, the problem also manifested on my machine. </p> <h3 id="b726532ae06844eb8aeaa2c27ca0d913"> Not my code <a href="#b726532ae06844eb8aeaa2c27ca0d913" title="permalink">#</a> </h3> <p> Okay, even with hundreds of tests, perhaps some edge case went unnoticed. The only problem with that hypothesis was that this was hardly an edge case. I was only making a <code>GET</code> request with a <code>Bearer</code> token. I wasn't going through some convoluted sequence of steps. </p> <p> <pre>GET /restaurants/1/schedule/2020/9/30 HTTP/1.1 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5c[...]</pre> </p> <p> I expected a successful response containing some JSON, but the result was <code>403 Forbidden</code>. That was the same behaviour I saw in the production environment. </p> <p> Now, to be clear, this is indeed a protected resource. If you present an invalid JWT, <code>403 Forbidden</code> is the expected response. That's why I wasted a few hours looking for problems with the the JWT. </p> <p> I finally, hesitatingly, concluded that the problem might be somewhere else. The JWT looked okay. So, hours into my troubleshooting I reluctantly set a breakpoint in my code and started the debugger. It isn't rational, but I tend to see it as a small personal defeat if I have to use the debugger. Even so, if used judiciously, it can be an effective way to identify problems. </p> <p> The debugger never hit my breakpoint. </p> <p> To be clear, the beginning of my Controller method looked like this: </p> <p> <pre>[<span style="color:#2b91af;">Authorize</span>(Roles&nbsp;=&nbsp;<span style="color:#a31515;">&quot;MaitreD&quot;</span>)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/schedule/{year}/{month}/{day}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;day) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!AccessControlList.Authorize(restaurantId)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ForbidResult</span>();</pre> </p> <p> My breakpoint was on the first line (the <code>if</code> conditional), but the debugger didn't break into it. When I issued my <code>GET</code> request, I immediately got the <code>403 Forbidden</code> response. The breakpoint just sat there in the debugger, mocking me. </p> <p> When that happens, it's natural to conclude that the problem occurs somewhere in the framework; in this case, ASP.NET. To test that hypothesis, I commented out the <code>[Authorize]</code> attribute and reissued the <code>GET</code> request. My hypothesis was that I'd get a <code>200 OK</code> response, since the attribute is what tells ASP.NET to check authorisation. </p> <p> The hypothesis held. The response was <code>200 OK</code>. </p> <h3 id="fdff404f12644a898b6a3e695fb6533f"> Test interdependency <a href="#fdff404f12644a898b6a3e695fb6533f" title="permalink">#</a> </h3> <p> I hate when that happens. It's up there with fixing other people's printers. The problem is in the framework, not in my code. I didn't have any authorisation callbacks registered, so I was fairly certain that the problem wasn't in my code. </p> <p> I rarely jump to the conclusion that there's a bug in the framework. In my experience, <a href="https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault">select is rarely broken</a>. My new hypothesis had to be that I'd somehow managed to misconfigure the framework. </p> <p> But where? There were automated tests that verified that a client could request that resource with a valid JWT. There were other automated tests that verified what happened if you presented an invalid JWT, or none at all. And all tests were passing. </p> <p> While I was fiddling with the tests, I eventually ran a parametrised test by itself, instead of the entire test suite: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>,&nbsp;2024,&nbsp;11,&nbsp;&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;2018,&nbsp;&nbsp;9,&nbsp;&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;The&nbsp;Vatican&nbsp;Cellar&quot;</span>,&nbsp;2021,&nbsp;10,&nbsp;10)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;GetRestaurantScheduleWhileAuthorized( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;day)</pre> </p> <p> This parametrised test has three test cases. When I ran just that test method, two of the test cases passed, but one failed: the <em>Nono</em> case, for some reason I haven't yet figured out. </p> <p> I didn't understand why that test case ought to fail while the others succeeded, but I had an inkling. I commented out the <em>Nono</em> test case and ran the test method again. </p> <p> One passed and one failing test. </p> <p> Now <em>The Vatican Cellar</em> test case was failing. I commented that out and ran the test method again. The remaining test case failed. </p> <p> This reeks of some sort of test interdependency. Apparently, <em>something</em> happens during the first test run that makes succeeding tests pass, but it happens too late for the first one. But what? </p> <h3 id="c95c3e707f46467abbf8ff02a9594c4c"> Bisect <a href="#c95c3e707f46467abbf8ff02a9594c4c" title="permalink">#</a> </h3> <p> Then something occurred to me that I should have thought of sooner. This feature used to work. Not only had the tests been passing, but I'd actually interacted with the deployed service, presenting a valid JWT and received a <code>200 OK</code> response. </p> <p> Once than dawned on me, I realised that it was just a manner of performing a binary search. Since, of course, I use version control, I had a version I knew worked, and a version that didn't work. The task, then, was to find the commit that introduced the defect. </p> <p> As I've already implied, the system in question is an example code base. While I have a cloud-based production environment, none but I use it. It had been four or five days since I'd actually interacted with the real service, and I'd been busy making changes, trusting exclusively in my test suite. I tend to make frequent, small commits instead of big, infrequent commits, so I had accumulated about a hundred and fifty commits since the 'last known good' deployment. </p> <p> Searching through hundreds of commits sounds overwhelming, but using binary search, it's actually not that bad. Pick the commit halfway between the 'last known good' commit and the most recent commit, and check it out. See if the defect is present there. If it is, you know that it was introduced somewhere between the commit you're looking at, and the 'last known good' commit. If it isn't present, it was introduced later. Regardless of the outcome, you know in which half to look. You now pick a new commit in the middle of that set and repeat the exercise. Even with, say, a hundred commits, the first bisection reduces the candidate set to 50, the next bisection to 25, then 13, then 7, 4, 2, and then you have it. If you do this systematically, you should find the exact commit in less than eight iterations. </p> <p> This is, as far as I understand it, the algorithm used by <em>Git bisect</em>. You don't have to use the <code>bisect</code> command - the algorithm is easy enough to do by hand - but let's see how it works. </p> <p> You start a <code>bisect</code> session with: </p> <p> <pre><span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...))</span> $ git bisect start <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...)|BISECTING)</span></pre> </p> <p> This starts an interactive session, which you can tell from the Git integration in Git Bash (it says <code>BISECTING</code>). You now mark a commit as being bad: </p> <p> <pre>$ git bisect bad <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...)|BISECTING)</span></pre> </p> <p> If you don't provide a commit ID at that point, Git is going to assume that you meant that the current commit (in this case <code>93c6c35</code>) is bad. That's what I had in mind, so that's fine. </p> <p> You now tell it about a commit ID that you know is good: </p> <p> <pre>$ git bisect good 7dfdab2 Bisecting: 75 revisions left to test after this (roughly 6 steps) [1f78c9a90c2088423ab4fc145b7b2ec3859d6a9a] Use InMemoryRestaurantDatabase in a test <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((1f78c9a...)|BISECTING)</span></pre> </p> <p> Notice that Git is already telling us how many iterations we should expect. You can also see that it checked out a new commit (<code>1f78c9a</code>) for you. That's the half-way commit. </p> <p> At this point, I manually ran the test method with the three test cases. All three passed, so I marked that commit as good: </p> <p> <pre>$ git bisect good Bisecting: 37 revisions left to test after this (roughly 5 steps) [5abf65a72628efabbf05fccd1b79340bac4490bc] Delete Either API <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((5abf65a...)|BISECTING)</span></pre> </p> <p> Again, Git estimates how many more steps are left and checks out a new commit (<code>5abf65a</code>). </p> <p> I repeated the process for each step, marking the commit as either good or bad, depending on whether or not the test passed: </p> <p> <pre>$ git bisect bad Bisecting: 18 revisions left to test after this (roughly 4 steps) [fc48292b0d654f4f20522710c14d7726e6eefa70] Delete redundant Test Data Builders <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((fc48292...)|BISECTING)</span> $ git bisect good Bisecting: 9 revisions left to test after this (roughly 3 steps) [b0cb1f5c1e9e40b1dabe035c41bfb4babfbe4585] Extract WillAcceptUpdate helper method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((b0cb1f5...)|BISECTING)</span> $ git bisect good Bisecting: 4 revisions left to test after this (roughly 2 steps) [d160c57288455377f8b0ad05985b029146228445] Extract ConfigureClock helper method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((d160c57...)|BISECTING)</span> $ git bisect good Bisecting: 2 revisions left to test after this (roughly 1 step) [4cb73c219565d8377aa67d79024d6836f9000935] Compact code <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((4cb73c2...)|BISECTING)</span> $ git bisect good Bisecting: 0 revisions left to test after this (roughly 1 step) [34238c7d2606e9007b96b54b43e678589723520c] Extract CreateTokenValidationParameters method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((34238c7...)|BISECTING)</span> $ git bisect bad Bisecting: 0 revisions left to test after this (roughly 0 steps) [7d6583a97ff45fbd85878cecb5af11d93213a25d] Move Configure method up <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((7d6583a...)|BISECTING)</span> $ git bisect good 34238c7d2606e9007b96b54b43e678589723520c is the first bad commit <span style="color: orange">commit 34238c7d2606e9007b96b54b43e678589723520c</span> Author: Mark Seemann &lt;mark@example.com&gt; Date: Wed Sep 16 07:15:12 2020 +0200 Extract CreateTokenValidationParameters method Restaurant.RestApi/Startup.cs | 32 <span style="color: green">+++++++++++++++++++</span><span style="color: red">-------------</span> 1 file changed, 19 insertions(+), 13 deletions(-) <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((7d6583a...)|BISECTING)</span></pre> </p> <p> Notice that the last step finds the culprit. It tells you which commit is the bad one, but surprisingly doesn't check it out for you. You can do that using a label Git has created for you: </p> <p> <pre>$ git checkout refs/bisect/bad Previous HEAD position was 7d6583a Move Configure method up HEAD is now at 34238c7 Extract CreateTokenValidationParameters method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((34238c7...)|BISECTING)</span></pre> </p> <p> You've now found and checked out the offending commit. Hopefully, the changes in that commit should give you a clue about the problem. </p> <h3 id="2f3c813d0cbf47eabb52467cff0c7692"> The culprit <a href="#2f3c813d0cbf47eabb52467cff0c7692" title="permalink">#</a> </h3> <p> What's in that commit? Take a gander: </p> <p> <pre>$ git show <span style="color: orange">commit 34238c7d2606e9007b96b54b43e678589723520c (</span><span style="color: mediumturquoise">HEAD</span><span style="color: orange">,</span> refs/bisect/bad<span style="color: orange">)</span> Author: Mark Seemann &lt;mark@example.com&gt; Date: Wed Sep 16 07:15:12 2020 +0200 Extract CreateTokenValidationParameters method <strong>diff --git a/Restaurant.RestApi/Startup.cs b/Restaurant.RestApi/Startup.cs index 6c161b5..bcde861 100644 --- a/Restaurant.RestApi/Startup.cs +++ b/Restaurant.RestApi/Startup.cs</strong> <span style="color: darkturquoise">@@ -79,10 +79,6 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private void ConfigureAuthorization(IServiceCollection services) { <span style="color: red">- JwtSecurityTokenHandler.DefaultMapInboundClaims = false; - - var secret = Configuration["JwtIssuerSigningKey"]; -</span> services.AddAuthentication(opts =&gt; { opts.DefaultAuthenticateScheme = <span style="color: darkturquoise">@@ -91,15 +87,8 @@</span> namespace Ploeh.Samples.Restaurants.RestApi JwtBearerDefaults.AuthenticationScheme; }).AddJwtBearer(opts =&gt; { <span style="color: red">- opts.TokenValidationParameters = new TokenValidationParameters - { - ValidateIssuerSigningKey = true, - IssuerSigningKey = new SymmetricSecurityKey( - Encoding.ASCII.GetBytes(secret)), - ValidateIssuer = false, - ValidateAudience = false, - RoleClaimType = "role" - };</span> <span style="color: green">+ opts.TokenValidationParameters = + CreateTokenValidationParameters();</span> opts.RequireHttpsMetadata = false; }); <span style="color: darkturquoise">@@ -108,6 +97,23 @@</span> namespace Ploeh.Samples.Restaurants.RestApi sp.GetService&lt;IHttpContextAccessor&gt;().HttpContext.User)); } <span style="color: green">+ private TokenValidationParameters CreateTokenValidationParameters() + { + JwtSecurityTokenHandler.DefaultMapInboundClaims = false; + + var secret = Configuration["JwtIssuerSigningKey"]; + + return new TokenValidationParameters + { + ValidateIssuerSigningKey = true, + IssuerSigningKey = new SymmetricSecurityKey( + Encoding.ASCII.GetBytes(secret)), + ValidateIssuer = false, + ValidateAudience = false, + RoleClaimType = "role" + }; + } +</span> private void ConfigureRepository(IServiceCollection services) {</pre> </p> <p> Can you spot the problem? </p> <p> As soon as I saw that diff, the problem almost jumped out at me. I'm so happy that I make small commits. What you see here is, I promise, the entire diff of that commit. </p> <p> Clearly, you're not familiar with this code base, but even so, you might be able to intuit the problem from the above diff. You don't need domain-specific knowledge for it, or knowledge of the rest of the code base. </p> <p> I'll give you a hint: I moved a line of code into a lambda expression. </p> <h3 id="1cfcd1adb0b24b6098aa250c73f5cba2"> Deferred execution <a href="#1cfcd1adb0b24b6098aa250c73f5cba2" title="permalink">#</a> </h3> <p> In the above commit, I extracted a helper method that I called <code>CreateTokenValidationParameters</code>. As the name communicates, it creates JWT validation parameters. In other words, it supplies the configuration values that determine how ASP.NET figures out how to authorise an HTTP request. </p> <p> Among other things, it sets the <code>RoleClaimType</code> property to <code>"role"</code>. By default, the <code>[Authorize]</code> attribute is looking for a role claim with a rather complicated identifier. Setting the <code>RoleClaimType</code> enables you to change the identifier. By setting it to <code>"role"</code>, I'm telling the ASP.NET framework that it should look for claims named <code>role</code> in the JWT, and these should be translated to role claims. </p> <p> Except that it doesn't work. </p> <p> Ostensibly, this is for backwards compatibility reasons. You have to set <a href="https://docs.microsoft.com/dotnet/api/system.identitymodel.tokens.jwt.jwtsecuritytokenhandler.defaultmapinboundclaims">JwtSecurityTokenHandler.DefaultMapInboundClaims</a> to <code>false</code>. Notice that this is a global variable. </p> <p> I thought that the line of code that does that conceptually belongs together with the other code that configures the JWT validation parameters, so I moved it in there. I didn't think much about it, and all my tests were still passing. </p> <p> What happened, though, was that I moved mutation of a global variable into a helper method that was called from a lambda expression. Keep in mind that lambda expressions represent code that may run later; that execution is deferred. </p> <p> By moving that mutation statement into the helper method, I inadvertently deferred its execution. When the application's <code>Startup</code> code runs, it configures the service. All the code that runs inside of <code>AddJwtBearer</code>, however, doesn't run immediately; it runs when needed. </p> <p> This explains why all tests were passing. My test suite has plenty of self-hosted integration tests in the style of <a href="/outside-in-tdd">outside-in test-driven development</a>. When I ran all the tests, the deferred code block would run in some other test context, flipping that global bit as a side effect. When the test suite reaches the test that fails when run in isolation, the bit is already flipped, and then it works. </p> <p> It took me hours to figure out what the problem was, and it turned out that the root cause was a global variable. </p> <p> Global variables are evil. Who knew? </p> <h3 id="cbf70c019a264e8fa861e13536db688f"> Can't reproduce <a href="#cbf70c019a264e8fa861e13536db688f" title="permalink">#</a> </h3> <p> Once you figure out what the problem is, you should reproduce it with an automated test. </p> <p> Yes, and how do I do that here? I already had tests that verify that you can <code>GET</code> the desired resource if you present a valid JWT. And <em>that test passes!</em> </p> <p> The only way I can think of to reproduce the issue with an automated test is to create a completely new, independent test library with only one test. I considered doing that, weighed the advantages against the disadvantages and decided, given the context, that it wasn't worth the effort. </p> <p> That means, though, that I had to accept that within the confines of my existing testing strategy, I can't reproduce the defect. This doesn't happen to me often. In fact, I can't recall that it's ever happened to me before, like this. </p> <h3 id="2ae70aa4524e48bf9b0190aeaa158192"> Resolution <a href="#2ae70aa4524e48bf9b0190aeaa158192" title="permalink">#</a> </h3> <p> It took me hours to find the bug, and ten seconds to fix it. I just moved the mutation of <code>JwtSecurityTokenHandler.DefaultMapInboundClaims</code> back to the <code>ConfigureAuthorization</code> method: </p> <p> <pre><strong>diff --git a/Restaurant.RestApi/Startup.cs b/Restaurant.RestApi/Startup.cs index d4917f5..e40032f 100644 --- a/Restaurant.RestApi/Startup.cs +++ b/Restaurant.RestApi/Startup.cs</strong> <span style="color: darkturquoise">@@ -79,6 +79,7 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private void ConfigureAuthorization(IServiceCollection services) { <span style="color: green">+ JwtSecurityTokenHandler.DefaultMapInboundClaims = false;</span> services.AddAuthentication(opts =&gt; { opts.DefaultAuthenticateScheme = <span style="color: darkturquoise">@@ -99,8 +100,6 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private TokenValidationParameters CreateTokenValidationParameters() { <span style="color: red">- JwtSecurityTokenHandler.DefaultMapInboundClaims = false; -</span> var secret = Configuration["JwtIssuerSigningKey"]; return new TokenValidationParameters</pre> </p> <p> An ad-hoc smoke test verified that this solved the problem. </p> <h3 id="55a0017f1aab47b6b9049e63d51d79ff"> Conclusion <a href="#55a0017f1aab47b6b9049e63d51d79ff" title="permalink">#</a> </h3> <p> There are multiple insights to be had from this experience: <ul> <li>Global variables are evil</li> <li>Binary search, or bisection, is useful when troubleshooting</li> <li>Not all bugs can be reproduced by an automated test</li> <li>Small commits make it easy to detect problems</li> </ul> I've always disliked Git's <em>squash</em> feature, and here's one of the many reasons to dislike it. Had this happened in a code base with a 'nice history' (as the <em>squash</em> proponents like to present it), that small commit would have been bundled with various other commits. The problem wouldn't have jumped at me if buried in dozens of other changes. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. EnsureSuccessStatusCode as an assertion https://blog.ploeh.dk/2020/09/28/ensuresuccessstatuscode-as-an-assertion 2020-09-28T05:50:00+00:00 Mark Seemann <div id="post"> <p> <em>It finally dawned on me that EnsureSuccessStatusCode is a fine test assertion.</em> </p> <p> Okay, I admit that sometimes I can be too literal and rigid in my thinking. At least as far back as 2012 I've been doing <a href="/outside-in-tdd">outside-in TDD</a> against self-hosted HTTP APIs. When you do that, you'll regularly need to verify that an HTTP response is successful. </p> <h3 id="77cb250db1ab4278bad68cf7736ae62e"> Assert equality <a href="#77cb250db1ab4278bad68cf7736ae62e" title="permalink">#</a> </h3> <p> You can, of course, write an assertion like this: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.Equal(<span style="color:#2b91af;">HttpStatusCode</span>.OK,&nbsp;response.StatusCode);</pre> </p> <p> If the assertion fails, it'll produce a comprehensible message like this: </p> <p> <pre>Assert.Equal() Failure Expected: OK Actual: BadRequest</pre> </p> <p> What's not to like? </p> <h3 id="2ba5bb17d1f8423184a50b2fa1fcb667"> Assert success <a href="#2ba5bb17d1f8423184a50b2fa1fcb667" title="permalink">#</a> </h3> <p> The problem with testing strict equality in this context is that it's often too narrow a definition of success. </p> <p> You may start by returning <code>200 OK</code> as response to a <code>POST</code> request, but then later evolve the API to return <code>201 Created</code> or <code>202 Accepted</code>. Those status codes still indicate success, and are typically accompanied by more information (e.g. a <code>Location</code> header). Thus, evolving a REST API from returning <code>200 OK</code> to <code>201 Created</code> or <code>202 Accepted</code> isn't a breaking change. </p> <p> It does, however, break the above assertion, which may now fail like this: </p> <p> <pre>Assert.Equal() Failure Expected: OK Actual: Created</pre> </p> <p> You could attempt to fix this issue by instead verifying <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage.issuccessstatuscode">IsSuccessStatusCode</a>: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.True(response.IsSuccessStatusCode);</pre> </p> <p> That permits the API to evolve, but introduces another problem. </p> <h3 id="0389a3d633c64aec8068be619ddce54c"> Legibility <a href="#0389a3d633c64aec8068be619ddce54c" title="permalink">#</a> </h3> <p> What happens when an assertion against <code>IsSuccessStatusCode</code> fails? </p> <p> <pre>Assert.True() Failure Expected: True Actual: False</pre> </p> <p> That's not helpful. At least, you'd like to know which other status code was returned instead. You can address that concern by using an overload to <code>Assert.True</code> (most unit testing frameworks come with such an overload): </p> <p> <pre><span style="color:#2b91af;">Assert</span>.True( &nbsp;&nbsp;&nbsp;&nbsp;response.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{response.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> This produces more helpful error messages: </p> <p> <pre>Actual status code: BadRequest. Expected: True Actual: False</pre> </p> <p> Granted, the <em>Expected: True, Actual: False</em> lines are redundant, but at least the information you care about is there; in this example, the status code was <code>400 Bad Request</code>. </p> <h3 id="9697ab68967948f89792b82d00028ccf"> Conciseness <a href="#9697ab68967948f89792b82d00028ccf" title="permalink">#</a> </h3> <p> That's not bad. You can use that <code>Assert.True</code> overload everywhere you want to assert success. You can even copy and paste the code, <a href="/2014/08/07/why-dry">because it's an idiom that doesn't change</a>. Still, since <a href="/2019/11/04/the-80-24-rule">I like to stay within 80 characters line width</a>, this little phrase takes up three lines of code. </p> <p> Surely, a helper method can address that problem: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AssertSuccess(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">HttpResponseMessage</span>&nbsp;response) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;response.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{response.StatusCode}<span style="color:#a31515;">.&quot;</span>); }</pre> </p> <p> I can now write a one-liner as an assertion: </p> <p> <pre>response.AssertSuccess();</pre> </p> <p> What's not to like? </p> <h3 id="6990f2fbc33f4beca15c122b5b4ccc22"> Carrying coals to Newcastle <a href="#6990f2fbc33f4beca15c122b5b4ccc22" title="permalink">#</a> </h3> <p> If you're already familiar with the <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage">HttpResponseMessage</a> class, you may have been reading this far with some frustration. It already comes with a method called <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage.ensuresuccessstatuscode">EnsureSuccessStatusCode</a>. Why not just use that instead? </p> <p> <pre>response.EnsureSuccessStatusCode();</pre> </p> <p> As long as the <code>response</code> status code is in the 200 range, this method call succeeds, but if not, it throws an exception: </p> <p> <pre>Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> That's exactly the information you need. </p> <p> I admit that I can sometimes be too rigid in my thinking. I've known about this method for years, but I never thought of it as a <em>unit testing assertion</em>. It's as if there's a category in my programming brain that's labelled <em>unit testing assertions</em>, and it only contains methods that originate from unit testing. </p> <p> I even knew about <a href="https://docs.microsoft.com/dotnet/api/system.diagnostics.debug.assert">Debug.Assert</a>, so I knew that assertions also exist outside of unit testing. That assertion isn't suitable for unit testing, however, so I never included it in my category of unit testing assertions. </p> <p> The <code>EnsureSuccessStatusCode</code> method doesn't have <em>assert</em> in its name, but it behaves just like a unit testing assertion: it throws an exception with a useful error message if a criterion is not fulfilled. I just never thought of it as useful for unit testing purposes because it's part of 'production code'. </p> <h3 id="5ff49213dd304440887719d6879ff7fb"> Conclusion <a href="#5ff49213dd304440887719d6879ff7fb" title="permalink">#</a> </h3> <p> The built-in <code>EnsureSuccessStatusCode</code> method is fine for unit testing purposes. Who knows, perhaps other class libraries contain similar assertion methods, although they may be hiding behind names that don't include <em>assert</em>. Perhaps your own production code contains such methods. If so, they can double as unit testing assertions. </p> <p> I've been doing test-driven development for close to two decades now, and one of many consistent experiences is this: <ul> <li>In the beginning of a project, the unit tests I write are verbose. This is natural, because I'm still getting to know the domain.</li> <li>After some time, I begin to notice patterns in my tests, so I refactor them. I introduce helper APIs in the test code.</li> <li>Finally, I realise that some of those unit testing APIs might actually be useful in the production code too, so I move them over there.</li> </ul> Test-driven development is process that gives you feedback about your production code's APIs. Listen to your tests. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="20484094920349b7a91eebcb9b15c628"> <div class="comment-author"><a href="https://github.com/YevgeniyShunevych">Yevgeniy Shunevych</a></div> <div class="comment-content"> <p> Unfortunately, I can't agree that it's fine to use <code>EnsureSuccessStatusCode</code> or other similar methods for assertions. An exception of a kind different from assertion (<code>XunitException</code> for XUnit, <code>AssertionException</code> for NUnit) is handled in a bit different way by the testing framework. <code>EnsureSuccessStatusCode</code> method throws <code>HttpRequestException</code>. Basically, a test that fails with such exception is considered as "Failed by error". </p> <p> For example, the test result message in case of 404 status produced by the <code>EnsureSuccessStatusCode</code> method is: </p> <p> <pre>System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (Not Found).</pre> </p> <p> Where "System.Net.Http.HttpRequestException :" text looks redundant, as we do assertion. This also may make other people think that the reason for this failure is not an assertion, but an unexpected exception. In case of regular assertion exception type, that text is missing. </p> <p> Other points are related to NUnit: <ul> <li> There are 2 different failure test outcome kinds in NUnit: <ul> <li>Failure: a test assertion failed. (Status=Failed, Label=empty)</li> <li>Error: an unexpected exception occurred. (Status=Failed, Label=Error)</li> </ul> See NUnit <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html#common-outcomes">Common Outcomes</a>. Thus `EnsureSuccessStatusCode` produces "Error" status instead of desirable "Failure". </li> <li>You can't use methods like <code>EnsureSuccessStatusCode</code> as assertion inside multiple asserts.</li> <li>NUnit tracks the count of assertions for each test. <code>"assertions"</code> property gets into the test results XML file and might be useful. This property increments on assertion methods, <code>EnsureSuccessStatusCode</code> - obviously doesn't increment it.</li> </ul> </p> </div> <div class="comment-date">2020-09-30 15:25 UTC</div> </div> <div class="comment" id="16adadb969f1406c9c18605c8c85ef83"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Yevgeniy, thank you for writing. This shows, I think, that all advice is contextual. It's been more than a decade since I regularly used NUnit, but I vaguely recall that it gives special treatment to its own assertion exceptions. </p> <p> I also grant that the "System.Net.Http.HttpRequestException" part of the message is redundant. To make matters worse, even without that 'prologue' the information that we care about (the actual status code) is all the way to the right. It'll often require horizontal scrolling to see it. </p> <p> That doesn't much bother me, though. I think that one should optimise for the most frequent scenario. For example, I favour taking a bit of extra time to write readable code, because <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">the bottleneck in programming isn't typing</a>. In the same vein, I find it a good trade-off being able to avoid adding more code against an occasional need to scroll horizontally. The cost of code isn't the time it takes to write it, but the maintenance burden it imposes, as well as the cognitive load it adds. </p> <p> The point here is that while horizontal scrolling is inconvenient, the <em>most frequent scenario</em> is that tests don't fail. The inconvenience only manifests in the rare circumstance that a test fails. </p> <p> I want to thank you, however, for pointing out the issues with NUnit. It reinforces the impression I already had of it, as a framework to stay away from. That design seems to me to impose a lock-in that prevents you from mixing NUnit with improved APIs. </p> <p> In contrast, the <a href="https://www.nuget.org/packages/xunit.core">xunit.core</a> library doesn't include <a href="https://www.nuget.org/packages/xunit.assert">xunit.assert</a>. I often take advantage of that when I write unit tests in F#. While the xUnit.net assertion library is adequate, <a href="https://github.com/SwensenSoftware/unquote">Unquote</a> is much better for F#. I appreciate that xUnit.net enables me to combine the tools that work best in a given situation. </p> </div> <div class="comment-date">2020-10-02 07:35 UTC</div> </div> <div class="comment" id="3dbfa056360b4df6a052be2776a6f930"> <div class="comment-author"><a href="https://github.com/YevgeniyShunevych">Yevgeniy Shunevych</a></div> <div class="comment-content"> <p> I see your point, Mark. Thanks for your reply. </p> <p> I would like to add a few words in defense of the NUnit. </p> <p> First of all, there is no problem to use an extra library for NUnit assertions too. For example, I like <a href="https://fluentassertions.com/">FluentAssertions</a> that works great for both XUnit and NUnit. </p> <p> NUnit 3 also has a set of useful features that are missing in XUnit like: <ul> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">Multiple asserts</a> - quite useful for UI testing, for example. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/Warnings.html">Warnings</a>. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html">TestContext</a>. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html#addtestattachment-37">File attachments</a> - useful for UI tests or tests that verify files. For example, when the UI test fails, you can take a screenshot, save it and add to NUnit as an attachment. Or when you test PDF file, you can attach the file that doesn't pass the test. </li> </ul> </p> <p> I can recommend to reconsider the view of NUnit, as the library progressed well during its 3rd version. XUnit and NUnit seem interchangeable for unit testing. But NUnit, from my point of view, provides the features that are helpful in integration and UI testing. </p> </div> <div class="comment-date">2020-10-05 13:24 UTC</div> </div> <div class="comment" id="fded96f5f68a4d8ba1e782678dd854be"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Yevgeniy, I haven't done UI testing since sometime in 2005, I think, so I wasn't aware of that. It's good to know that NUnit may excel at that, and the other scenarios you point out. </p> <p> To be clear, while I still think I may prefer xUnit.net, I don't consider it perfect. </p> </div> <div class="comment-date">2020-10-05 14:10 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An XPath query for long methods https://blog.ploeh.dk/2020/09/21/an-xpath-query-for-long-methods 2020-09-21T05:28:00+00:00 Mark Seemann <div id="post"> <p> <em>A filter for Visual Studio code metrics.</em> </p> <p> I consider it a good idea to <a href="/2020/04/13/curb-code-rot-with-thresholds">limit</a> the size of code blocks. Small methods are <a href="/2019/11/18/small-methods-are-easy-to-troubleshoot">easier to troubleshoot</a> and in general fit better in your head. When it comes to vertical size, however, the editors I use don't come with visual indicators like <a href="https://marketplace.visualstudio.com/items?itemName=PaulHarrington.EditorGuidelines">they do for horizontal size</a>. </p> <p> When you're in the midst of developing a feature, you don't want to be <em>prevented</em> from doing so by a tool that refuses to compile your code if methods get too long. On the other hand, it might be a good idea to regularly run a tool over your code base to identify which methods are getting too long. </p> <p> With Visual Studio you can calculate code metrics, which include a measure of the number of lines of code for each method. One option produces an XML file, but if you have a large code base, those XML files are big. </p> <p> The XML files typically look like this: </p> <p> <pre><span style="color:blue;">&lt;?</span><span style="color:#a31515;">xml</span><span style="color:blue;">&nbsp;</span><span style="color:red;">version</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1.0</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">encoding</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">utf-8</span>&quot;<span style="color:blue;">?&gt;</span> <span style="color:blue;">&lt;</span><span style="color:#a31515;">CodeMetricsReport</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Version</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1.0</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Targets</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Target</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Restaurant.RestApi.csproj</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Assembly</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Ploeh.Samples.Restaurants.RestApi,&nbsp;...</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">87</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">537</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">208</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">3188</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">711</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Namespaces</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Namespace</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Ploeh.Samples.Restaurants.RestApi</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">87</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">499</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">204</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">3100</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">701</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Types</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">NamedType</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CalendarController</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">73</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">14</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">34</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">190</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">42</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Members</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Method</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Task</span><span style="color:red;">&amp;lt;</span><span style="color:blue;">ActionResult</span><span style="color:red;">&amp;gt;</span><span style="color:blue;">&nbsp;CalendarController.Get(...</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">64</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">2</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">12</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">28</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">7</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;!--</span><span style="color:green;">Much&nbsp;more&nbsp;data&nbsp;goes&nbsp;here</span><span style="color:blue;">--&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Members</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">NamedType</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Types</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Namespace</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Namespaces</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Assembly</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Target</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Targets</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&lt;/</span><span style="color:#a31515;">CodeMetricsReport</span><span style="color:blue;">&gt;</span></pre> </p> <p> How can you filter such a file to find only those methods that are too long? </p> <p> That sounds like a job for <a href="https://en.wikipedia.org/wiki/XPath">XPath</a>. I admit, though, that I use XPath only rarely. While the general idea of the syntax is easy to grasp, it has enough subtle pitfalls that it's not that easy to use, either. </p> <p> Partly for my own benefit, and partly for anyone else who might need it, here's an XPath query that looks for long methods: </p> <p> <pre>//Members/child::*[Metrics/Metric[@Value &gt; 24 and @Name = "SourceLines"]]</pre> </p> <p> This query looks for <a href="/2019/11/04/the-80-24-rule">methods longer that 24 lines of code</a>. If you don't agree with that <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a> you can always change it to another value. You can also change <code>@Name</code> to look for <code>CyclomaticComplexity</code> or one of the other metrics. </p> <p> Given the above XML metrics report, the XPath filter would select (among other members) the <code>CalendarController.Get</code> method, because it has 28 lines of source code. It turns out, though, that the filter produces some false positives. The method in question is actually fine: </p> <p> <pre><span style="color:green;">/*&nbsp;This&nbsp;method&nbsp;loads&nbsp;a&nbsp;year&#39;s&nbsp;worth&nbsp;of&nbsp;reservations&nbsp;in&nbsp;order&nbsp;to&nbsp;segment &nbsp;*&nbsp;them&nbsp;all.&nbsp;In&nbsp;a&nbsp;realistic&nbsp;system,&nbsp;this&nbsp;could&nbsp;be&nbsp;quite&nbsp;stressful&nbsp;for &nbsp;*&nbsp;both&nbsp;the&nbsp;database&nbsp;and&nbsp;the&nbsp;web&nbsp;server.&nbsp;Some&nbsp;of&nbsp;that&nbsp;concern&nbsp;can&nbsp;be &nbsp;*&nbsp;addressed&nbsp;with&nbsp;an&nbsp;appropriate&nbsp;HTTP&nbsp;cache&nbsp;header&nbsp;and&nbsp;a&nbsp;reverse&nbsp;proxy, &nbsp;*&nbsp;but&nbsp;a&nbsp;better&nbsp;solution&nbsp;would&nbsp;be&nbsp;a&nbsp;CQRS-style&nbsp;architecture&nbsp;where&nbsp;the &nbsp;*&nbsp;calendars&nbsp;get&nbsp;re-rendered&nbsp;as&nbsp;materialised&nbsp;views&nbsp;in&nbsp;a&nbsp;background &nbsp;*&nbsp;process.&nbsp;That&#39;s&nbsp;beyond&nbsp;the&nbsp;scope&nbsp;of&nbsp;this&nbsp;example&nbsp;code&nbsp;base,&nbsp;though. &nbsp;*/</span> [<span style="color:#2b91af;">ResponseCache</span>(Duration&nbsp;=&nbsp;60)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/calendar/{year}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:blue;">int</span>&nbsp;year) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.GetRestaurant(restaurantId).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(restaurant&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;<span style="color:#2b91af;">Period</span>.Year(year); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;MakeDays(restaurant,&nbsp;period) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;restaurant.Name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Year&nbsp;=&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Days&nbsp;=&nbsp;days &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> That method only has 18 lines of actual source code, from the beginning of the method declaration to the closing bracket. Visual Studio's metrics calculator, however, also counts the attributes and the comments. </p> <p> In general, I only add comments when I want to communicate something that I can't express as a type or a method name, so in this particular code base, it's not much of an issue. If you consistently adorn every method with doc comments, on the other hand, you may need to perform some pre-processing on the source code before you calculate the metrics. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. We need young programmers; we need old programmers https://blog.ploeh.dk/2020/09/14/we-need-young-programmers-we-need-old-programmers 2020-09-14T05:46:00+00:00 Mark Seemann <div id="post"> <p> <em>The software industry loves young people, but old-timers serve an important purpose, too.</em> </p> <p> Our culture idolises youth. There's several reasons for this, I believe. Youth seems synonymous with vigour, strength, beauty, and many other desirable qualities. The cynical perspective is that young people, while rebellious, also tend to be easy to manipulate, if you know which buttons to push. A middle-aged man like me isn't susceptible to the argument that I should buy a particular pair of Nike shoes because they're named after Michael Jordan, but for a while, one pair wasn't enough for my teenage daughter. </p> <p> In intellectual pursuits (like software development), youth is often extolled as the source of innovation. You're often confronted with examples like that of <a href="https://en.wikipedia.org/wiki/%C3%89variste_Galois">Évariste Galois</a>, who made all his discoveries before turning 21. <a href="https://en.wikipedia.org/wiki/Ada_Lovelace">Ada Lovelace</a> was around 28 years when she produced what is considered the 'first computer program'. <a href="https://en.wikipedia.org/wiki/Alan_Turing">Alan Turing</a> was 24 when he wrote <a href="https://en.wikipedia.org/wiki/Turing%27s_proof">On Computable Numbers, with an Application to the Entscheidungsproblem</a>. </p> <p> Clearly, young age is no detriment to making ground-breaking contributions. It has even become folklore that everyone past the age of 35 is a has-been whose only chance at academic influence is to write a textbook. </p> <h3 id="800321a74c054ea0b75815c86f4ce18d"> The story of the five monkeys <a href="#800321a74c054ea0b75815c86f4ce18d" title="permalink">#</a> </h3> <p> You may have seen a story called <em>the five monkeys experiment</em>. It's most likely a fabrication, but it goes like this: </p> <p> A group of scientists placed five monkeys in a cage, and in the middle, a ladder with bananas on the top. Every time a monkey went up the ladder, the scientists soaked the rest of the monkeys with cold water. After a while, every time a monkey went up the ladder, the others would beat it up. </p> <p> After some time, none of the monkeys dared go up the ladder regardless of the temptation. The scientists then substituted one of the monkeys with a new one, who'd immediately go for the bananas, only to be beaten up by the others. After several beatings, the new member learned not to climb the ladder even though it never knew why. </p> <p> A second monkey was substituted and the same occurred. The first monkey participated in beating the second. A third monkey was exchanged and the story repeated. The fourth was substituted and the beating was repeated. Finally the fifth monkey was replaced. </p> <p> Left was a group of five monkeys who, even though they never received a cold shower, continued to beat up any monkey who attempted to climb the ladder. If it was possible to ask the monkeys why they would beat up all who attempted to go up the ladder, the answer would probably be: </p> <p> "That's how we do things here." </p> <p> While the story is probably just that: a story, it tells us something about the drag induced by age and experience. If you've been in the business for decades, you've seen numerous failed attempts at something you yourself tried when you were young. You know that it can't be done. </p> <p> Young people don't know that a thing can't be done. If they can avoid the monkey-beating, they'll attempt the impossible. </p> <h3 id="4add8a9af0424d7e889d3125837ed611"> Changing circumstances <a href="#4add8a9af0424d7e889d3125837ed611" title="permalink">#</a> </h3> <p> Is attempting the impossible a good idea? </p> <p> In general, no, because it's... impossible. There's a reason older people tell young people that a thing can't be done. It's not just because they're stodgy conservatives who abhor change. It's because they see the effort as wasteful. Perhaps they're even trying to be kind, guiding young people off a path where only toil and disappointment is to be found. </p> <p> What old people don't realise is that sometimes, circumstances change. </p> <p> What was impossible twenty years ago may not be impossible today. We see this happening in many fields. Producing a commercially viable electric car was impossible for decades, until, with the advances made in battery technology, it became possible. </p> <p> Technology changes rapidly in software development. People trying something previously impossible may find that it's possible today. Once, if you had lots of data, you had to store it in fully normalised form, because storage was expensive. For a decade, relational databases were the only game in town. Then circumstances changed. Storage became cheaper, and a new movement of NoSQL storage emerged. What was before impossible became possible. </p> <p> Older people often don't see the new opportunities, because they 'know' that some things are impossible. Young people push the envelope driven by a combination of zest and ignorance. Most fail, but a few succeed. </p> <h3 id="4272a069588e47f796646bd282b9de02"> Lottery of the impossible <a href="#4272a069588e47f796646bd282b9de02" title="permalink">#</a> </h3> <p> I think of this process as a lottery. Imagine that every impossible thing is a red ball in an urn. Every young person who tries the impossible draws a random ball from the urn. </p> <p> The urn contains millions of red balls, but every now and then, one of them turns green. You don't know which one, but if you draw it, it represents something that was previously impossible which has now become possible. </p> <p> This process produces growth, because once discovered, the new and better way of doing things can improve society in general. Occasionally, the young discoverer may even gain some fame and fortune. </p> <p> It seems wasteful, though. Most people who attempt the impossible will reach the predictable conclusion. What was deemed impossible was, indeed, impossible. </p> <p> When I'm in a cynical mood, I don't think that it's youth in itself that is the source of progress. It's just the <a href="https://en.wikipedia.org/wiki/Law_of_large_numbers">law of large numbers</a> applied. If there's a one in million chance that something will succeed, but ten million people attempt it, it's only a matter of time before one succeeds. </p> <p> Society at large can benefit from the success of the few, but ten million people still wasted their efforts. </p> <h3 id="016744f0ea77495c958a7914f08187db"> We need the old, too <a href="#016744f0ea77495c958a7914f08187db" title="permalink">#</a> </h3> <p> If you accept the argument that young people are more likely to try the impossible, we need the young people. Do we need the old people? </p> <p> I'm turning fifty in 2020. You may consider that old, but I expect to work for many more years. I don't know if the software industry needs fifty-year-olds, but that's not the kind of old I have in mind. I'm thinking of people who have retired, or are close to retirement. </p> <p> In our youth-glorifying culture, we tend to dismiss the opinion and experiences of old people. <em>Oh, well, it's just a codgy old man</em> (or woman), we'll say. </p> <p> We ignore the experience of the old, because we believe that they haven't been keeping up with times. Their experiences don't apply to us, because we live under new circumstance. Well, see above. </p> <p> I'm not advocating that we turn into a gerontocracy that venerates our elders solely because of their age. Again, according to the law of large numbers, some people live to old age. There need not be any correlation between survivors and wisdom. </p> <p> We need the old to tell us the truth, because they have little to lose. </p> <h3 id="8b5c613ba6c44bb4b4e6dbba7ae7d19a"> Nothing to lose <a href="#8b5c613ba6c44bb4b4e6dbba7ae7d19a" title="permalink">#</a> </h3> <p> In the last couple of years, I've noticed a trend. A book comes out, exposing the sad state of affairs in some organisation. This has happened regularly in Denmark, where I live. One book may expose the deplorable conditions of the Danish tax authorities, one may describe the situation in the ministry of defence, one criticises the groupthink associated with the climate crisis, and so on. </p> <p> Invariably, it turns out that the book is written by a professor emeritus or a retired department head. </p> <p> I don't think that these people, all of a sudden, had an epiphany after they retired. They knew all about the rot in the system they were part of, while they were part of it, but they've had too much to lose. You could argue that they should have said something before they retired, but that requires a moral backbone we can't expect most people to have. </p> <p> When people retire, the threat of getting fired disappears. Old people can speak freely to a degree most other people can't. </p> <p> Granted, many may simply use that freedom to spew bile or shout <em>Get off my lawn!</em>, but many are in the unique position to reveal truths no-one else dare speak. Many are, perhaps, just bitter, but some may possess knowledge that they are in a unique position to reveal. </p> <p> When that grumpy old guy on Twitter writes something that makes you uncomfortable, consider this: he may still be right. </p> <h3 id="2d64bd2c7ccb4b7ca2418802ed82689e"> Being unreasonable <a href="#2d64bd2c7ccb4b7ca2418802ed82689e" title="permalink">#</a> </h3> <p> In a way, you could say that we need young and old people for the same fundamental reason. Not all of them, but enough of them, are in a position to be unreasonable. <blockquote> <p> "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." </p> <footer><cite>George Bernard Shaw</cite></footer> </blockquote> Young people and old people are unreasonable in each their own way, and we need both. </p> <h3 id="df88f595ec814e2bafcbd018ff5f5ad2"> Conclusion <a href="#df88f595ec814e2bafcbd018ff5f5ad2" title="permalink">#</a> </h3> <p> We need young people in the software development industry. Because of their vigour and inexperience, they'll push the envelope. Most will fail to do the impossible, but a few succeed. </p> <p> This may seem like a cynical view, but we've all been young, and most of us have been through such a phase. It's like a rite of passage, and even if you fail to make your mark on the world, you're still likely to have learned a lot. </p> <p> We need old people because they're in a position to speak truth to the world. Notice that I didn't make my argument about the <em>experience</em> of old-timers. Actually, I find that valuable as well, but that's the ordinary argument: <em>Listen to old people, because they have experience and wisdom.</em> </p> <p> Some of them do, at least. </p> <p> I didn't make much out of that argument, because you already know it. There'd be no reason to write this essay if that was all I had to say. Old people have less on the line, so they can speak more freely. If someone you used to admire retires and all of a sudden starts saying or writing unpleasant and surprising things, there might be a good explanation, and it might be a good idea to pay attention. </p> <p> Or maybe he or she is just bitter or going senile... </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Add null check without brackets in Visual Studio https://blog.ploeh.dk/2020/09/07/add-null-check-without-brackets-in-visual-studio 2020-09-07T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>A Visual Studio tweak.</em> </p> <p> The most recent versions of Visual Studio have included many new <em>Quick Actions</em>, accessible with <kbd>Ctrl</kbd> + <kbd>.</kbd> or <kbd>Alt</kbd> + <kbd>Enter</kbd>. The overall feature has been around for some years, but the product group has been adding features at a good pace recently, it seems to me. </p> <p> One feature has been around for at least a year: <em>Add null check</em>. In a default installation of Visual Studio, it looks like this: </p> <p> <img src="/content/binary/add-null-check-with-brackets.png" alt="Screen shot of the 'Add null check' Quick Action. By default it'll add brackets around the throw statement."> </p> <p> As the screen shot shows, it'll auto-generate a Guard Clause like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Reservation</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;otherSeating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;other.Date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Overlaps(otherSeating); }</pre> </p> <p> Part of my personal coding style is that I don't use brackets for one-liners. This is partially motivated by the desire to save vertical space, since I try to <a href="/2019/11/04/the-80-24-rule">keep methods as small as possible</a>. Some people worry that not using brackets for one-liners makes the code more vulnerable to defects, but I typically have automated regressions tests to keep an eye on correctness. </p> <p> The above default behaviour of the <em>Add null check</em> Quick Action annoyed me because I had to manually remove the brackets. It's one of those things that annoy you a little, but not enough that you throw aside what you're doing to figure out if you can change the situation. </p> <p> Until it annoyed me enough to investigate whether there was something I <em>could</em> do about it. It turns out to be easy to tweak the behaviour. </p> <p> In Visual Studio 2019, go to <em>Tools</em>, <em>Options</em>, <em>Text Editor</em>, <em>C#</em>, <em>Code Style</em>, <em>General</em> and change the <em>Prefer braces</em> option to <em>No</em>: </p> <p> <img src="/content/binary/visual-studio-prefer-braces-option.png" alt="Screen shot of Visual Studio Options dialog box."> </p> <p> This changes the behaviour of the <em>Add null check</em> Quick Action: </p> <p> <img src="/content/binary/add-null-check-without-brackets.png" alt="Screen shot of the 'Add null check' Quick Action after the behaviour change. It no longer adds brackets around the throw statement."> </p> <p> After applying the Quick Action, my code now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Reservation</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;otherSeating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;other.Date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Overlaps(otherSeating); }</pre> </p> <p> This better fits my preference. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bbe35bc429dc4d759fab30cfc90c9bdd"> <div class="comment-author">Daniel</div> <div class="comment-content"> <p> The same should go for if's, switch's, try/catch and perhaps a few other things. While we are at it, spacing is also a good source for changing layout in code, e.g: decimal.Round( amount * 1.0000m, provider.Getdecimals( currency ) ) ). </p> </div> <div class="comment-date">2020-09-07 7:14 UTC</div> </div> <div class="comment" id="ec5cf23c648e4277b851685bc55e258a"> <div class="comment-author">James World</div> <div class="comment-content"> <p> When assigning parameters to fields, I like this one-line option for a null check: </p> <p> <pre>_thing = thing ?? throw new ArgumentNullException(nameof(thing));</pre>. </p> <p> How do you feel about it? You could even use a disard in your example: <pre>_ = thing ?? throw new ArgumentNullException(nameof(thing));</pre> </p> </div> <div class="comment-date">2020-09-07 13:14 UTC</div> </div> <div class="comment" id="e456bf65fe3e426f97639b8855622a7b"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> James, thank you for writing. I'm aware of that option, but rarely use it in my examples. I try to write my example code in a way that they may also be helpful to Java developers, or programmers that use other C-based languages. Obviously, that concern doesn't apply if I discuss something specific to C#, but when I try to explain design patterns or low-level architecture, I try to stay away from language features exclusive to C#. </p> <p> In other contexts, I might or might not use that language feature, but I admit that it jars me. I find that it's a 'clever' solution to something that's not much of a problem. The <code>??</code> operator is intended as a short-circuiting operator one can use to simplify variable assignment. Throwing on the right-hand side never assigns a value. </p> <p> It's as though you would use the <a href="/2018/04/09/coalescing-composite-as-a-monoid">coalescing monoid</a> to pick the left-hand side, but let the program fail on the right-hand side. In Haskell it would look like this: </p> <p> <pre>x = thing &lt;&gt; undefined</pre> </p> <p> The compiler infers that <code>x</code> has the same type as <code>thing</code>, e.g. <code>Semigroup a =&gt; Maybe a</code>. If <code>thing</code>, however, is <code>Nothing</code>, the expression crashes when evaluated. </p> <p> While this compiles in Haskell, I consider it unidiomatic. It too blatantly advertises the existence of the so-called <a href="https://en.wikipedia.org/wiki/Bottom_type">bottom value</a> (⊥). We know it's there, but we prefer to pretend that it's not. </p> <p> That line of reasoning is subjective, and if I find myself in a code base that uses <code>??</code> like you've shown, I'd try to fit in with that style. It doesn't annoy me that much, after all. </p> <p> It's also shorter, I must admit that. </p> </div> <div class="comment-date">2020-09-08 05:57 UTC</div> </div> <div class="comment" id="5c16a687f8b4421fa826ef7c66656767"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> <p> The above default behaviour of the <em>Add null check</em> Quick Action annoyed me because I had to manually remove the brackets. It's one of those things that annoy you a little, but not enough that you throw aside what you're doing to figure out if you can change the situation. </p> <p> Until it annoyed me enough to investigate whether there was something I <em>could</em> do about it. It turns out to be easy to tweak the behaviour. </p> <p> In Visual Studio 2019, go to <em>Tools</em>, <em>Options</em>, <em>Text Editor</em>, <em>C#</em>, <em>Code Style</em>, <em>General</em> and change the <em>Prefer braces</em> option to <em>No</em>: </p> </blockquote> <p> You can do more than that. As suggested by Visual Studio in your second screenshot, you can (after making the change to not prefer braces) generate an <code>.editorconfig</code> file containing your coding style settings. Visual Studio will prompt you to save the file alongside your solution. If you do so, then any developer that opens these files with Visual Studio with also have your setting to not prefer braces. I wrote a <a href="https://tysonwilliams.coding.blog/2020-09-12_editorconfig">short post about EditorConfig</a> that contains more information. </p> </div> <div class="comment-date">2020-09-13 01:53 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Properties for all https://blog.ploeh.dk/2020/08/31/properties-for-all 2020-08-31T08:39:00+00:00 Mark Seemann <div id="post"> <p> <em>Writing test cases for all possible input values.</em> </p> <p> I've noticed that programmers new to automated testing struggle with a fundamental task: how to come up with good test input? </p> <p> There's plenty of design patterns that address that issue, including <a href="http://www.natpryce.com/articles/000714.html">Test Data Builders</a>. Still, test-driven development, when done right, gives you good feedback on the design of your API. I don't consider having to resort to Test Data Builder as much of a compromise, but still, it's even better if you can model the API in question so that it requires no redundant data. </p> <p> Most business processes can be modelled as finite state machines. Once you've understood the problem well enough to enumerate the possible states, you can reverse-engineer an <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data type</a> from that enumeration. </p> <p> While the data carried around in the state machine may be unconstrained, the number of state and state transitions is usually limited. Model the states as enumerable values, and you can cover all input values with simple combinatorics. </p> <h3 id="44553eea5abb4ab58c73ce173a12ddde"> Tennis states <a href="#44553eea5abb4ab58c73ce173a12ddde" title="permalink">#</a> </h3> <p> The <a href="https://codingdojo.org/kata/Tennis">Tennis kata</a> is one of my favourite exercises. Just like a business process, it turns out that you can model the rules as a finite state machine. There's a state where both players have <em>love</em>, 15, 30, or 40 points (although both can't have 40 points; that state is called <em>deuce</em>); there's a state where a player has the <em>advantage</em>; and so on. </p> <p> I've <a href="https://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">previously written about how to design the states of the Tennis kata with types</a>, so here, I'll just limit myself to the few types that turn out to matter as test input: players and points. </p> <p> Here are both types defined as <a href="https://www.haskell.org">Haskell</a> <a href="https://en.wikipedia.org/wiki/Tagged_union">sum types</a>: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;Player&nbsp;=&nbsp;PlayerOne&nbsp;|&nbsp;PlayerTwo&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>,&nbsp;<span style="color:#2b91af;">Read</span>,&nbsp;<span style="color:#2b91af;">Enum</span>,&nbsp;<span style="color:#2b91af;">Bounded</span>) <span style="color:blue;">data</span>&nbsp;Point&nbsp;=&nbsp;Love&nbsp;|&nbsp;Fifteen&nbsp;|&nbsp;Thirty&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>,&nbsp;<span style="color:#2b91af;">Read</span>,&nbsp;<span style="color:#2b91af;">Enum</span>,&nbsp;<span style="color:#2b91af;">Bounded</span>)</pre> </p> <p> Notice that both are instances of the <code>Enum</code> and <code>Bounded</code> type classes. This means that it's possible to enumerate all values that inhabit each type. You can use that to your advantage when writing unit tests. </p> <h3 id="5a6d5d80394b4486bb17adc4cd8cf727"> Properties for every value <a href="#5a6d5d80394b4486bb17adc4cd8cf727" title="permalink">#</a> </h3> <p> Most people think of <a href="/property-based-testing-intro">property-based testing</a> as something that involves a special framework such as <a href="https://hackage.haskell.org/package/QuickCheck">QuickCheck</a>, <a href="https://github.com/hedgehogqa">Hedgehog</a>, or <a href="https://fscheck.github.io/FsCheck/index.html">FsCheck</a>. I often use such frameworks, but as I've <a href="/2015/02/23/property-based-testing-without-a-property-based-testing-framework">written about before</a>, sometimes enumerating all possible input values is simpler and faster than relying on randomly generated values. There's no reason to generate 100 random values of a type inhabited by only three values. </p> <p> Instead, write properties for the entire <a href="https://en.wikipedia.org/wiki/Domain_of_a_function">domain</a> of the function in question. </p> <p> In Haskell, you can define a generic enumeration like this: </p> <p> <pre><span style="color:#2b91af;">every</span>&nbsp;::&nbsp;(<span style="color:blue;">Enum</span>&nbsp;a,&nbsp;<span style="color:blue;">Bounded</span>&nbsp;a)&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;[a] every&nbsp;=&nbsp;[<span style="color:blue;">minBound</span>&nbsp;..&nbsp;<span style="color:blue;">maxBound</span>] </pre> </p> <p> I don't know why this function doesn't already exist in the standard library, but I suppose that most people just rely on the list comprehension syntax <code>[minBound .. maxBound]</code>... </p> <p> With it you can write simple properties of the Tennis game. For example: </p> <p> <pre><span style="color:#a31515;">&quot;Given&nbsp;deuce,&nbsp;when&nbsp;player&nbsp;wins,&nbsp;then&nbsp;that&nbsp;player&nbsp;has&nbsp;advantage&quot;</span>&nbsp;~:&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;player&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;score&nbsp;Deuce&nbsp;player &nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;$&nbsp;Advantage&nbsp;player&nbsp;~=?&nbsp;actual </pre> </p> <p> Here I'm using <a href="/2018/05/07/inlined-hunit-test-lists">inlined HUnit test lists</a>. With the Tennis kata, it's easiest to start with the <em>deuce</em> case, because regardless of player, the new state is that the winning player has advantage. That's what that property asserts. Because of the <code>do</code> notation, the property produces a list of test cases, one for <code>every</code> player. There's only two <code>Player</code> values, so it's only two test cases. </p> <p> The beauty of <code>do</code> notation (or the list monad in general) is that you can combine enumerations, for example like this: </p> <p> <pre><span style="color:#a31515;">&quot;Given&nbsp;forty,&nbsp;when&nbsp;player&nbsp;wins&nbsp;ball,&nbsp;then&nbsp;that&nbsp;player&nbsp;wins&nbsp;the&nbsp;game&quot;</span>&nbsp;~:&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;player&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;opp&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;score&nbsp;(Forty&nbsp;$&nbsp;FortyData&nbsp;player&nbsp;opp)&nbsp;player &nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;$&nbsp;Game&nbsp;player&nbsp;~=?&nbsp;actual </pre> </p> <p> While <code>player</code> is a <code>Player</code> value, <code>opp</code> (<em>Other Player's Point</em>) is a <code>Point</code> value. Since there's two possible <code>Player</code> values and three possible <code>Point</code> values, the combination yields six test cases. </p> <p> I've been doing the Tennis kata a couple of times using this approach. I've also done it in C# with <a href="https://xunit.net">xUnit.net</a>, where the first test of the <em>deuce</em> state looks like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>,&nbsp;<span style="color:#2b91af;">MemberData</span>(<span style="color:blue;">nameof</span>(Player))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;TransitionFromDeuce(<span style="color:#2b91af;">IPlayer</span>&nbsp;player) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Deuce</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.BallTo(player); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Advantage</span>(player); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(expected,&nbsp;actual); }</pre> </p> <p> <a href="/2019/12/16/zone-of-ceremony">C# takes more ceremony than Haskell</a>, but the idea is the same. The <code>Player</code> data source for the <code>[Theory]</code> is defined as an enumeration of the possible values: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:blue;">object</span>[]&gt;&nbsp;Player { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">object</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PlayerOne</span>()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">object</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PlayerTwo</span>()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> All the times I've done the Tennis kata by fully exhausting the domain of the transition functions in question, I've arrived at 40 test cases. I wonder if that's the number of possible state transitions of the game, or if it's just an artefact of the way I've modelled it. I suspect the latter... </p> <h3 id="21ccf5bd3c3f48859bbb8c766b0a0610"> Conclusion <a href="#21ccf5bd3c3f48859bbb8c766b0a0610" title="permalink">#</a> </h3> <p> You can sometimes enumerate all possible inputs to an API. Even if a function takes a Boolean value and a byte as input, the enumeration of all possible combinations is only 2 * 256 = 512 values. You computer will tear through all of those combinations faster than you can say <em>random selection</em>. Consider writing APIs that take algebraic data types as input, and writing properties that exhaust the domain of the functions in question. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Adding REST links as a cross-cutting concern https://blog.ploeh.dk/2020/08/24/adding-rest-links-as-a-cross-cutting-concern 2020-08-24T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Use a piece of middleware to enrich a Data Transfer Object. An ASP.NET Core example.</em> </p> <p> When developing true REST APIs, you should <a href="https://martinfowler.com/articles/richardsonMaturityModel.html">use hypermedia controls</a> (i.e. <em>links</em>) to guide clients to the resources they need. I've always felt that the code that generates these links tends to make otherwise readable Controller methods unreadable. </p> <p> I'm currently experimenting with generating links as a cross-cutting concern. So far, I like it very much. </p> <h3 id="57785b860f604552b099e42b296f6fb6"> Links from home <a href="#57785b860f604552b099e42b296f6fb6" title="permalink">#</a> </h3> <p> Consider an online restaurant reservation system. When you make a <code>GET</code> request against the home resource (which is the only published URL for the API), you should receive a representation like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/reservations&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/13&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> As you can tell, my example just runs on my local development machine, but I'm sure that you can see past that. There's three calendar links that clients can use to <code>GET</code> the restaurant's calendar for the current day, month, or year. Clients can use these resources to present a user with a date picker or a similar user interface so that it's possible to pick a date for a reservation. </p> <p> When a client wants to make a reservation, it can use the URL identified by the <code>rel</code> (<em>relationship type</em>) <code>"urn:reservations"</code> to make a <code>POST</code> request. </p> <h3 id="724d54b2bb4a4071b0f965105541c792"> Link generation as a Controller responsibility <a href="#724d54b2bb4a4071b0f965105541c792" title="permalink">#</a> </h3> <p> I first wrote the code that generates these links directly in the Controller class that serves the home resource. It looked like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IActionResult</span>&nbsp;Get() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;links&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">List</span>&lt;<span style="color:#2b91af;">LinkDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToReservations()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(enableCalendar) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToYear(now.Year)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToMonth(now.Year,&nbsp;now.Month)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToDay(now.Year,&nbsp;now.Month,&nbsp;now.Day)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Ok(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>&nbsp;{&nbsp;Links&nbsp;=&nbsp;links.ToArray()&nbsp;}); }</pre> </p> <p> That doesn't look too bad, but 90% of the code is exclusively concerned with generating links. (<code>enableCalendar</code>, by the way, is a <a href="https://en.wikipedia.org/wiki/Feature_toggle">feature flag</a>.) That seems acceptable in this special case, because there's really nothing else the home resource has to do. For other resources, the Controller code might contain some composition code as well, and then all the link code starts to look like noise that makes it harder to understand the actual purpose of the Controller method. You'll see an example of a non-trivial Controller method later in this article. </p> <p> It seemed to me that enriching a Data Transfer Object (DTO) with links ought to be a cross-cutting concern. </p> <h3 id="1973741c11414414bcd0ca2748bf0f0d"> LinksFilter <a href="#1973741c11414414bcd0ca2748bf0f0d" title="permalink">#</a> </h3> <p> In ASP.NET Core, you can implement cross-cutting concerns with a type of middleware called <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.filters.iasyncactionfilter">IAsyncActionFilter</a>. I added one called <code>LinksFilter</code>: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LinksFilter</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IAsyncActionFilter</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;enableCalendar; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;UrlHelperFactory&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">LinksFilter</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;urlHelperFactory, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">CalendarFlag</span>&nbsp;calendarFlag) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;UrlHelperFactory&nbsp;=&nbsp;urlHelperFactory; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;enableCalendar&nbsp;=&nbsp;calendarFlag.Enabled; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;OnActionExecutionAsync( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutionDelegate</span>&nbsp;next) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ctxAfter&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!(ctxAfter.Result&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>&nbsp;ok)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;UrlHelperFactory.GetUrlHelper(ctxAfter); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">switch</span>&nbsp;(ok.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">case</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>&nbsp;homeDto: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(homeDto,&nbsp;url); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">case</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span>&nbsp;calendarDto: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(calendarDto,&nbsp;url); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">default</span>: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> There's only one method to implement. If you want to run some code <em>after</em> the Controllers have had their chance, you invoke the <code>next</code> delegate to get the resulting context. It should contain the response to be returned. If <code>Result</code> isn't an <code>OkObjectResult</code> there's no content to enrich with links, so the method just returns. </p> <p> Otherwise, it switches on the type of the <code>ok.Value</code> and passes the DTO to an appropriate helper method. Here's the <code>AddLinks</code> overload for <code>HomeDto</code>: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddLinks(<span style="color:#2b91af;">HomeDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(enableCalendar) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToReservations(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToYear(now.Year), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToMonth(now.Year,&nbsp;now.Month), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToDay(now.Year,&nbsp;now.Month,&nbsp;now.Day) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;url.LinkToReservations()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> You can probably recognise the implemented behaviour from before, where it was implemented in the <code>Get</code> method. That method now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;Get() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>()); }</pre> </p> <p> That's clearly much simpler, but you probably think that little has been achieved. After all, doesn't this just move some code from one place to another? </p> <p> Yes, that's the case in this particular example, but I wanted to start with an example that was so simple that it highlights how to move the code to a filter. Consider, then, the following example. </p> <h3 id="9156b0b69f4d485fa373a8c4b06547d5"> A calendar resource <a href="#9156b0b69f4d485fa373a8c4b06547d5" title="permalink">#</a> </h3> <p> The online reservation system enables clients to navigate its calendar to look up dates and time slots. A representation might look like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;previous&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/12&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;next&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/14&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;], &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;year&quot;</span>:&nbsp;2020, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;month&quot;</span>:&nbsp;8, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;day&quot;</span>:&nbsp;13, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;days&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/13&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-08-13&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;entries&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;21:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> This is a JSON representation of the calendar for August 13, 2020. The data it contains is the identification of the date, as well as a series of <code>entries</code> that lists the largest reservation the restaurant can accept for each time slot. </p> <p> Apart from the data, the representation also contains links. There's a general collection of links that currently holds only <code>next</code> and <code>previous</code>. In addition to that, each day has its own array of links. In the above example, only a single day is represented, so the <code>days</code> array contains only a single object. For a month calendar (navigatable via the <code>urn:month</code> link), there'd be between 28 and 31 <code>days</code>, each with its own <code>links</code> array. </p> <p> Generating all these links is a complex undertaking all by itself, so separation of concerns is a boon. </p> <h3 id="8f85a3a882164352aa57d600a21628be"> Calendar links <a href="#8f85a3a882164352aa57d600a21628be" title="permalink">#</a> </h3> <p> As you can see in the above <code>LinksFilter</code>, it branches on the type of value wrapped in an <code>OkObjectResult</code>. If the type is <code>CalendarDto</code>, it calls the appropriate <code>AddLinks</code> overload: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddLinks(<span style="color:#2b91af;">CalendarDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;dto.ToPeriod(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;previous&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PreviousPeriodVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;next&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NextPeriodVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToPeriod(previous,&nbsp;<span style="color:#a31515;">&quot;previous&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToPeriod(next,&nbsp;<span style="color:#a31515;">&quot;next&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto.Days&nbsp;<span style="color:blue;">is</span>&nbsp;{&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">foreach</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;day&nbsp;<span style="color:blue;">in</span>&nbsp;dto.Days) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(day,&nbsp;url); }</pre> </p> <p> It both generates the <code>previous</code> and <code>next</code> links on the <code>dto</code>, as well as the links for each day. While I'm not going to bore you with more of that code, you can tell, I hope, that the <code>AddLinks</code> method calls other helper methods and classes. The point is that link generation involves more than just a few lines of code. </p> <p> You already saw that in the first example (related to <code>HomeDto</code>). The question is whether there's still some significant code left in the Controller class? </p> <h3 id="ab37807dfce0431b8308f86ec18a44aa"> Calendar resource <a href="#ab37807dfce0431b8308f86ec18a44aa" title="permalink">#</a> </h3> <p> The <code>CalendarController</code> class defines three overloads of <code>Get</code> - one for a single day, one for a month, and one for an entire year. Each of them looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;<span style="color:#2b91af;">Period</span>.Month(year,&nbsp;month); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;MakeDays(period).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Year&nbsp;=&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Month&nbsp;=&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Days&nbsp;=&nbsp;days &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> It doesn't look as though much is going on, but at least you can see that it returns a <code>CalendarDto</code> object. </p> <p> While the method looks simple, it's not. Significant work happens in the <code>MakeDays</code> helper method: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">DayDto</span>[]&gt;&nbsp;MakeDays(<span style="color:#2b91af;">IPeriod</span>&nbsp;period) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;firstTick&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FirstTickVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;lastTick&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LastTickVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations(firstTick,&nbsp;lastTick).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DaysVisitor</span>()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Select(d&nbsp;=&gt;&nbsp;MakeDay(d,&nbsp;reservations)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToArray(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;days; }</pre> </p> <p> After having read relevant <code>reservations</code> from the database, it applies complex business logic to allocate them and thereby being able to report on remaining capacity for each time slot. </p> <p> Not having to worry about link generation while doing all that work seems like a benefit. </p> <h3 id="87b164914be0450fbd523735946cf146"> Filter registration <a href="#87b164914be0450fbd523735946cf146" title="permalink">#</a> </h3> <p> You must tell the ASP.NET Core framework about any filters that you add. You can do that in the <code>Startup</code> class' <code>ConfigureServices</code> method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ConfigureServices(<span style="color:#2b91af;">IServiceCollection</span>&nbsp;services) { &nbsp;&nbsp;&nbsp;&nbsp;services.AddControllers(opts&nbsp;=&gt;&nbsp;opts.Filters.Add&lt;<span style="color:#2b91af;">LinksFilter</span>&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> When registered, the filter executes for each HTTP request. When the object represents a <code>200 OK</code> result, the filter populates the DTOs with links. </p> <h3 id="ced9fd76d526420aa1a847da4398e904"> Conclusion <a href="#ced9fd76d526420aa1a847da4398e904" title="permalink">#</a> </h3> <p> By treating RESTful link generation as a cross-cutting concern, you can separate if from the logic of generating the data structure that represents the resource. That's not the only way to do it. You could also write a simple function that populates DTOs, and call it directly from each Controller action. </p> <p> What I like about using a filter is that I don't have to remember to do that. Once the filter is registered, it'll populate all the DTOs it knows about, regardless of which Controller generated them. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="fff391e80e27427e8fc93959deef3768"> <div class="comment-author"><a href="https://twitter.com/MehdiFalamarzi?s=08">Mehdi Falamarzi</a></div> <div class="comment-content"> <p> Thanks for your good and insightful posts. </p> <p> Separation of REST concerns from MVC controller's concerns is a great idea, But in my opinion this solution has two problems: </p> <h3 id="52608f57b49942aaaafdab7aae22c85"> Distance between related REST implementations <a href="#52608f57b49942aaaafdab7aae22c85" title="permalink">#</a> </h3> <p> When implementing REST by MVC pattern often REST archetypes are the reasons for a MVC Controller class to be created. As long as the MVC Controller class describes the archetype, And links of a resource is a part of the response when implementing hypermedia controls, having the archetype and its related links in the place where the resource described is a big advantage in easiness and readability of the design. pulling out link implementations and putting them in separate classes causes higher readability and uniformity of the code abstranction levels in the action method at the expense of making a distance between related REST implementation. </p> <h3 id="fa1203384d3746bfadd276aae6ac860f"> Implementation scalability <a href="#fa1203384d3746bfadd276aae6ac860f" title="permalink">#</a> </h3> <p> There is a switch statement on the ActionExecutingContext's result in the LinksFilter to decide what links must be represented to the client in the response.The DTOs thre are the results of the clients's requests for URIs. If this solution generalised for every resources the API must represent there will be several cases for the switch statement to handle. Beside that every resources may have different implemetations for generating their links. Putting all this in one place leads the LinksFilter to be coupled with too many helper classes and this coupling process never stops. </p> <h3 id="4dd0703da482480abffbaffabdb27741"> Solution <a href="#4dd0703da482480abffbaffabdb27741" title="permalink">#</a> </h3> <p> LinkDescriptor and LinkSubscriber for resources links definition </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">LinkDescriptor</span> { <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Rel { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Href { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Resource { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } } </pre> </div> <p> Letting the MVC controller classes have their resource's links definitions but not in action methods. </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> ActionResult <span style="color: #0066BB; font-weight: bold">Get</span>() { <span style="color: #008800; font-weight: bold">return</span> <span style="color: #008800; font-weight: bold">new</span> <span style="color: #0066BB; font-weight: bold">OkObjectResult</span>(<span style="color: #008800; font-weight: bold">new</span> HomeDto()); } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">RegisterLinks</span>(LinkSubscriber subscriber) { subscriber .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:reservations&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/reservations&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:year&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:month&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020/8&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:day&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020/8/13&quot;</span>); } </pre> </div> <p> Registering resources links by convention </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">MvcBuilderExtensions</span> { <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> IMvcBuilder <span style="color: #0066BB; font-weight: bold">AddRestLinks</span>(<span style="color: #008800; font-weight: bold">this</span> IMvcBuilder mvcBuilder) { <span style="color: #333399; font-weight: bold">var</span> subscriber = <span style="color: #008800; font-weight: bold">new</span> LinkSubscriber(); <span style="color: #333399; font-weight: bold">var</span> linkRegistrationMethods = GetLinkRegistrationMethods(mvcBuilder.Services); PopulateLinks(subscriber, linkRegistrationMethods); mvcBuilder.Services.AddSingleton&lt;IEnumerable&lt;LinkDescriptor&gt;&gt;(subscriber.LinkDescriptors); <span style="color: #008800; font-weight: bold">return</span> mvcBuilder; } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">static</span> List&lt;MethodInfo&gt; GetLinkRegistrationMethods(IServiceCollection services) { <span style="color: #008800; font-weight: bold">return</span> <span style="color: #0066BB; font-weight: bold">typeof</span>(MvcBuilderExtensions).Assembly.ExportedTypes .Where(tp =&gt; <span style="color: #008800; font-weight: bold">typeof</span>(ControllerBase).IsAssignableFrom(tp)) .Select(tp =&gt; tp.GetMethod(<span style="background-color: #fff0f0">&quot;RegisterLinks&quot;</span>, <span style="color: #008800; font-weight: bold">new</span>[] { <span style="color: #008800; font-weight: bold">typeof</span>(LinkSubscriber) })) .Where(mi =&gt; mi != <span style="color: #008800; font-weight: bold">null</span>) .ToList(); } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">PopulateLinks</span>(LinkSubscriber subscriber, List&lt;MethodInfo&gt; linkRegistrationMethods) { <span style="color: #008800; font-weight: bold">foreach</span> (<span style="color: #333399; font-weight: bold">var</span> method <span style="color: #008800; font-weight: bold">in</span> linkRegistrationMethods) { method.Invoke(<span style="color: #008800; font-weight: bold">null</span>, <span style="color: #008800; font-weight: bold">new</span>[] { subscriber }); } } } </pre> </div> <p> Add dependencies and execute procedures for adding links to responses </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">ConfigureServices</span>(IServiceCollection services) { services.AddControllers(conf =&gt; conf.Filters.Add&lt;LinksFilter&gt;()) .AddRestLinks(); } </pre> </div> <p> And Last letting the LinksFilter to dynamicaly add resources links by utilizing ExpandoObject </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">LinksFilter</span> : IAsyncActionFilter { <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">readonly</span> IEnumerable&lt;LinkDescriptor&gt; links; <span style="color: #008800; font-weight: bold">public</span> <span style="color: #0066BB; font-weight: bold">LinksFilter</span>(IEnumerable&lt;LinkDescriptor&gt; links) { <span style="color: #008800; font-weight: bold">this</span>.links = links; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">async</span> Task <span style="color: #0066BB; font-weight: bold">OnActionExecutionAsync</span>(ActionExecutingContext context, ActionExecutionDelegate next) { <span style="color: #333399; font-weight: bold">var</span> relatedLinks = links .Where(lk =&gt; context.HttpContext.Request.Path.Value.ToLower() == lk.Resource); <span style="color: #008800; font-weight: bold">if</span> (relatedLinks.Any()) <span style="color: #008800; font-weight: bold">await</span> <span style="color: #0066BB; font-weight: bold">ManipulateResponseAsync</span>(context, next, relatedLinks); } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">async</span> Task <span style="color: #0066BB; font-weight: bold">ManipulateResponseAsync</span>(ActionExecutingContext context, ActionExecutionDelegate next, IEnumerable&lt;LinkDescriptor&gt; relatedLinks) { <span style="color: #333399; font-weight: bold">var</span> ctxAfter = <span style="color: #008800; font-weight: bold">await</span> next().ConfigureAwait(<span style="color: #008800; font-weight: bold">false</span>); <span style="color: #008800; font-weight: bold">if</span> (!(ctxAfter.Result <span style="color: #008800; font-weight: bold">is</span> ObjectResult objRes)) <span style="color: #008800; font-weight: bold">return</span>; <span style="color: #333399; font-weight: bold">var</span> expandoResult = <span style="color: #008800; font-weight: bold">new</span> ExpandoObject(); FillExpandoWithResultProperties(expandoResult, objRes.Value); FillExpandoWithLinks(expandoResult, relatedLinks); objRes.Value = expandoResult; } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">FillExpandoWithResultProperties</span>(ExpandoObject resultExpando, <span style="color: #333399; font-weight: bold">object</span> <span style="color: #008800; font-weight: bold">value</span>) { <span style="color: #333399; font-weight: bold">var</span> properties = <span style="color: #008800; font-weight: bold">value</span>.GetType().GetProperties(); <span style="color: #008800; font-weight: bold">foreach</span> (<span style="color: #333399; font-weight: bold">var</span> property <span style="color: #008800; font-weight: bold">in</span> properties) { resultExpando.TryAdd(property.Name, property.GetValue(<span style="color: #008800; font-weight: bold">value</span>)); } } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">FillExpandoWithLinks</span>(ExpandoObject resultExpando, IEnumerable&lt;LinkDescriptor&gt; relatedLinks) { <span style="color: #333399; font-weight: bold">var</span> linksToAdd = relatedLinks.Select(lk =&gt; <span style="color: #008800; font-weight: bold">new</span> { Rel = lk.Rel, Href = lk.Href }); resultExpando.TryAdd(<span style="background-color: #fff0f0">&quot;Links&quot;</span>, linksToAdd); } } </pre> </div> <p> If absoulute URI in href field is prefered, IUriHelper can be injected in LinksFilter to create URI paths. </p> </div> <div class="comment-date">2020-08-24 23:39 UTC</div> </div> <div class="comment" id="127d2750e83811ea8dd900155d8065e1"> <div class="comment-author"><a href="https://github.com/JesHansen">Jes Hansen</a></div> <div class="comment-content"> <p> Mark, thanks for figuring out the tricky parts so we don't have to. :-) </p> <p> I did not see a link to a repo with the completed code from this article, and a cursory look around your Github profile didn't give away any obvious clues. Is the example code in the article part of a repo we can clone? If so, could you please provide a link? </p> </div> <div class="comment-date">2020-08-27 07:45 UTC</div> </div> <div class="comment" id="13197699aaff4a90901910a1975c4de0"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Jes, it's part of a larger project that I'm currently working on. Eventually, I hope to publish it, but it's not yet in a state where I wish to do that. </p> <p> Did I leave out essential details that makes it hard to reproduce the idea? </p> </div> <div class="comment-date">2020-08-27 08:07 UTC</div> </div> <div class="comment" id="0e3477f6e84311eaadc700155d8065e1"> <div class="comment-author"><a href="https://github.com/JesHansen">Jes Hansen</a></div> <div class="comment-content"> <p> No, your presentation was fine. Looking forward to see the completed project! </p> </div> <div class="comment-date">2020-08-27 08:57 UTC</div> </div> <div class="comment" id="a5f42d7c00e947a885506abdd2a16628"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Mehdi, thank you for writing. It's true that the <code>LinksFilter</code> implementation code contains a <code>switch</code> statement, and that this is one of multiple possible designs. I do believe that this is a trade-off rather than a problem per se. </p> <p> That <code>switch</code> statement is an implementation detail of the filter, and something I might decide to change in the future. I did choose that implementation, though, because I it was the simplest design that came to my mind. As presented, the <code>switch</code> statement just calls some private helper methods (all called <code>AddLinks</code>), but if one wanted that code close to the rest of the Controller code, one could just move those helper methods to the relevant Controller classes. </p> <p> While you wouldn't need to resort to Reflection to do that, it's true that this would leave that <code>switch</code> statement as a central place where developers would have to go if they add a new resource. It's true that your proposed solution addresses that problem, but doesn't it just shift the burden somewhere else? Now, developers will have to know that they ought to add a <code>RegisterLinks</code> method with a specific signature to their Controller classes. This replaces a design with compile-time checking with something that may fail at run time. How is that an improvement? </p> <p> I think that I understand your other point about the distance of code, but it assumes a particular notion of REST that I find parochial. Most (.NET) developers I've met design REST APIs in a code-centric (if not a database-centric) way. They tend to conflate <em>representations</em> with <em>resources</em> and translate both to Controller classes. </p> <p> The idea behind <em>Representational State Transfer</em>, however, is to decouple state from representation. Resources have state, but can have multiple representations. Vice versa, many resources may share the same representation. In the code base I used for this article, not only do I have three overloaded <code>Get</code> methods on <code>CalendarController</code> that produce <code>CalendarDto</code> representations, I also have a <code>ScheduleController</code> class that does the same. </p> <p> Granted, not all REST API code bases are designed like this. I admit that what actually motivated me to do things like this was to avoid having to inherit from <code>ControllerBase</code>. Moving all the code that relies heavily on the ASP.NET infrastructure keeps the Controller classes lighter, and thus easier to test. I should probably write an article about that... </p> </div> <div class="comment-date">2020-09-01 07:56 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Unit testing is fine https://blog.ploeh.dk/2020/08/17/unit-testing-is-fine 2020-08-17T05:29:00+00:00 Mark Seemann <div id="post"> <p> <em>Unit testing considered harmful? I think not.</em> </p> <p> Once in a while, some article, video, or podcast makes the rounds on social media, arguing that unit testing is bad, overrated, harmful, or the like. I'm not going to link to any specific resources, because this post isn't an attack on any particular piece of work. Many of these are sophisticated, thoughtful, and make good points, but still arrive at the wrong conclusion. </p> <p> The line of reasoning tends to be to show examples of bad unit tests and conclude that, based on the examples, unit tests are bad. </p> <p> The power of examples is great, but clearly, this is a logical fallacy. </p> <h3 id="5dbc2a18fc7f439b9db0282f0613b2a5"> Symbolisation <a href="#5dbc2a18fc7f439b9db0282f0613b2a5" title="permalink">#</a> </h3> <p> In case it isn't clear that the argument is invalid, I'll demonstrate it using the techniques I've learned from <a href="https://bit.ly/predicate-logic-intro">Howard Pospesel's introduction to predicate logic</a>. </p> <p> We can begin by symbolising the natural-language arguments into well-formed formulas. I'll keep this as simple as possible: </p> <p> <pre>∃xBx ⊢ ∀xBx (Domain: unit tests; Bx = x is bad)</pre> </p> <p> Basically, <code>Bx</code> states that <code>x</code> is bad, where <code>x</code> is a unit test. <code>∃xBx</code> is a statement that there exists a unit test <code>x</code> for which <code>x</code> is bad; i.e. <em>bad unit tests exist</em>. The statement <code>∀xBx</code> claims that for all unit tests <code>x</code>, <code>x</code> is bad; i.e. <em>all unit tests are bad</em>. The turnstile symbol <code>⊢</code> in the middle indicates that the antecedent on the left proves the consequent to the right. </p> <p> Translated back to natural language, the claim is this: <em>Because bad unit tests exist, all unit tests are bad.</em> </p> <p> You can trivially prove this <a href="https://en.wikipedia.org/wiki/Sequent">sequent</a> invalid. </p> <h3 id="a5f5e5c8e1f94696b8ffb86a119170b4"> Logical fallacy <a href="#a5f5e5c8e1f94696b8ffb86a119170b4" title="permalink">#</a> </h3> <p> One way to prove the sequent invalid is to use a truth tree: </p> <p> <img src="/content/binary/truth-tree-open-all-unit-tests-bad.png" alt="Truth tree that disproves the sequent about all unit tests being bad."> </p> <p> Briefly, the way this works is that the statements on the left-hand side represent truth, while the ones to the right are false. By placing the antecedent on the left, but the consequent on the right, you're basically assuming the sequent to be wrong. This is is also the way you <em>prove</em> correct sequents true; if the conclusion is assumed false, a logical truth should lead to a contradiction. If it doesn't, the sequent is invalid. That's what happens here. </p> <p> The tree remains <em>open</em>, which means that the original sequent is invalid. It's a logical fallacy. </p> <h3 id="c19e880f945d4a44a40ca1ccb9b96f74"> Counter examples <a href="#c19e880f945d4a44a40ca1ccb9b96f74" title="permalink">#</a> </h3> <p> You probably already knew that. All it takes to counter a universal assertion such as <em>all unit tests are bad</em> is to produce a counter example. One is sufficient, because if just a single good unit test exists, it can't be true that <em>all</em> are bad. </p> <p> Most of the think pieces that argue that unit testing is bad do so by showing examples of bad unit tests. These tests typically involve lots of mocks and stubs; they tend to test the interaction between internal components instead of the components themselves, or the system as a whole. I agree that this often leads to <a href="http://xunitpatterns.com/Fragile%20Test.html">fragile tests</a>. </p> <p> While I still spend my testing energy according to the <a href="https://martinfowler.com/bliki/TestPyramid.html">Test Pyramid</a>, I don't write unit tests like that. I rarely use dynamic mock libraries. Instead, I <a href="/2020/03/02/impureim-sandwich">push impure actions to the boundary of the system</a> and write most of the application code as <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>, which are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. No test-induced damage there. </p> <p> I follow up with <a href="/outside-in-tdd">boundary tests</a> that demonstrate that the <a href="/2015/12/21/integration-testing-composed-functions">functions are integrated into a working system</a>. That's just another layer in the Test Pyramid, but smaller. You don't need that many integration tests when you have a foundation of good unit tests. </p> <p> While I'm currently working on a larger body of work that showcases this approach, this blog <a href="/2019/04/01/an-example-of-state-based-testing-in-c">already has examples of this</a>. </p> <h3 id="414e235a672042a6a07e572027250df0"> Conclusion <a href="#414e235a672042a6a07e572027250df0" title="permalink">#</a> </h3> <p> You often hear or see the claim that unit tests are bad. The supporting argument is that a particular (popular, I admit) style of unit testing is bad. </p> <p> If the person making this claim only knows of that single style of unit testing, it's natural to jump to the conclusion that all unit testing must be bad. </p> <p> That's not the case. I write most of my unit tests in a style dissimilar from the interaction-heavy, mocks-and-stubs-based style that most people use. These test have a low maintenance burden and don't cause test-induced damage. </p> <p> Unit testing is fine. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An ASP.NET Core URL Builder https://blog.ploeh.dk/2020/08/10/an-aspnet-core-url-builder 2020-08-10T06:59:00+00:00 Mark Seemann <div id="post"> <p> <em>A use case for the Immutable Fluent Builder design pattern variant.</em> </p> <p> The <a href="/2020/02/10/builder-isomorphisms">Fluent Builder</a> design pattern is popular in object-oriented programming. Most programmers use the mutable variant, while I favour the immutable alternative. The advantages of Immutable Fluent Builders, however, may not be immediately clear. <blockquote> <p> "I never thought of someone reusing a configured builder (soulds like too big class/SRP violation)." </p> <footer><cite><a href="https://twitter.com/ELuciusFTW/status/1226800015047348224">Guy Buss</a></cite></footer> </blockquote> It inspires me when I encounter a differing perspective. Could I be wrong? Or did I fail to produce a compelling example? </p> <p> It's possible that I'm wrong, but in my <a href="/2020/02/10/builder-isomorphisms">my recent article on Builder isomorphisms</a> I focused on the pattern variations themselves, to the point where a convincing example wasn't my top priority. </p> <p> I recently encountered a good use case for an Immutable Fluent Builder. </p> <h3 id="e2761ee2b9654ad982a2ec1a6264c76f"> Build links <a href="#e2761ee2b9654ad982a2ec1a6264c76f" title="permalink">#</a> </h3> <p> I was developing a REST API and wanted to generate some links like these: </p> <p> <pre>{ "links": [ { "rel": "urn:reservations", "href": "http://localhost:53568/reservations" }, { "rel": "urn:year", "href": "http://localhost:53568/calendar/2020" }, { "rel": "urn:month", "href": "http://localhost:53568/calendar/2020/7" }, { "rel": "urn:day", "href": "http://localhost:53568/calendar/2020/7/7" } ] }</pre> </p> <p> As I recently described, <a href="/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper">the ASP.NET Core Action API is tricky</a>, and since there was some repetition, I was looking for a way to reduce the code duplication. At first I just thought I'd make a few private helper methods, but then it occurred to me that an Immutable Fluent Builder as an <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a> to the <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.urlhelperextensions.action">Action API</a> might offer a fertile alternative. </p> <h3 id="5adf09c648a54679b1b137776e4f1192"> UrlBuilder class <a href="#5adf09c648a54679b1b137776e4f1192" title="permalink">#</a> </h3> <p> The various <code>Action</code> overloads all accept null arguments, so there's effectively no clear invariants to enforce on that dimension. While I wanted an Immutable Fluent Builder, I made all the fields nullable. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;action; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;controller; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">object</span>?&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(<span style="color:blue;">string</span>?&nbsp;action,&nbsp;<span style="color:blue;">string</span>?&nbsp;controller,&nbsp;<span style="color:blue;">object</span>?&nbsp;values) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.action&nbsp;=&nbsp;action; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.controller&nbsp;=&nbsp;controller; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.values&nbsp;=&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> I also gave the <code>UrlBuilder</code> class a public constructor and a private copy constructor. That's the standard way I implement that pattern. </p> <p> Most of the modification methods are straightforward: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithAction(<span style="color:blue;">string</span>&nbsp;newAction) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(newAction,&nbsp;controller,&nbsp;values); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithValues(<span style="color:blue;">object</span>&nbsp;newValues) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(action,&nbsp;controller,&nbsp;newValues); }</pre> </p> <p> I wanted to encapsulate <a href="/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper">the suffix-handling behaviour I recently described</a> in the appropriate method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithController(<span style="color:blue;">string</span>&nbsp;newController) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(newController&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(newController)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;controllerSuffix&nbsp;=&nbsp;<span style="color:#a31515;">&quot;controller&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;index&nbsp;=&nbsp;newController.LastIndexOf( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controllerSuffix, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">StringComparison</span>.OrdinalIgnoreCase); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(0&nbsp;&lt;=&nbsp;index) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newController&nbsp;=&nbsp;newController.Remove(index); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(action,&nbsp;newController,&nbsp;values); }</pre> </p> <p> The <code>WithController</code> method handles both the case where <code>newController</code> is suffixed by <code>"Controller"</code> and the case where it isn't. I also wrote unit tests to verify that the implementation works as intended. </p> <p> Finally, a Builder should have a method to build the desired object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Uri</span>&nbsp;BuildAbsolute(<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(url&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(url)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actionUrl&nbsp;=&nbsp;url.Action( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;action, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controller, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.ActionContext.HttpContext.Request.Host.ToUriComponent()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(actionUrl); }</pre> </p> <p> One could imagine also defining a <code>BuildRelative</code> method, but I didn't need it. </p> <h3 id="bec08c507ce7475eb0038723c6f6f8a8"> Generating links <a href="#bec08c507ce7475eb0038723c6f6f8a8" title="permalink">#</a> </h3> <p> Each of the objects shown above are represented by a simple Data Transfer Object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LinkDto</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Rel&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Href&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> My next step was to define an extension method on <code>Uri</code>, so that I could turn a URL into a link: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;Link(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">Uri</span>&nbsp;uri,&nbsp;<span style="color:blue;">string</span>&nbsp;rel) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;{&nbsp;Rel&nbsp;=&nbsp;rel,&nbsp;Href&nbsp;=&nbsp;uri.ToString()&nbsp;}; }</pre> </p> <p> With that function I could now write code like this: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;CreateYearLink() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithAction(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithController(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.BuildAbsolute(Url) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Link(<span style="color:#a31515;">&quot;urn:year&quot;</span>); }</pre> </p> <p> It's acceptable, but verbose. This only creates the <code>urn:year</code> link; to create the <code>urn:month</code> and <code>urn:day</code> links, I needed similar code. Only the <code>WithValues</code> method calls differed. The calls to <code>WithAction</code> and <code>WithController</code> were identical. </p> <h3 id="e003c737cf604f83bd61dfba8045754b"> Shared Immutable Builder <a href="#e003c737cf604f83bd61dfba8045754b" title="permalink">#</a> </h3> <p> Since <code>UrlBuilder</code> is immutable, I can trivially define a shared instance: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;calendar&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithAction(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithController(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>));</pre> </p> <p> This enabled me to write more succinct methods for each of the relationship types: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToYear(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:year&quot;</span>); } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToMonth(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year,&nbsp;month&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:month&quot;</span>); } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToDay(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month,&nbsp;<span style="color:blue;">int</span>&nbsp;day) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year,&nbsp;month,&nbsp;day&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:day&quot;</span>); }</pre> </p> <p> This is possible exactly because <code>UrlBuilder</code> is immutable. Had the Builder been mutable, such sharing would have created an <a href="https://en.wikipedia.org/wiki/Aliasing_(computing)">aliasing</a> bug, as I <a href="/2020/02/10/builder-isomorphisms">previously described</a>. Immutability enables reuse. </p> <h3 id="cc70ea76298547e893d883bcb9983f92"> Conclusion <a href="#cc70ea76298547e893d883bcb9983f92" title="permalink">#</a> </h3> <p> I got my first taste of functional programming around 2010. Since then, when I'm not programming in <a href="https://fsharp.org">F#</a> or <a href="https://www.haskell.org">Haskell</a>, I've steadily worked on identifying good ways to enjoy the benefits of functional programming in C#. </p> <p> Immutability is a fairly low-hanging fruit. It requires more boilerplate code, but apart from that, it's easy to make classes immutable in C#, and Visual Studio has plenty of refactorings that make it easier. </p> <p> Immutability is one of those features you're unlikely to realise that you're missing. When it's not there, you work around it, but when it's there, it simplifies many tasks. </p> <p> The example you've seen in this article relates to the Fluent Builder pattern. At first glance, it seems as though a mutable Fluent Builder has the same capabilities as a corresponding Immutable Fluent Builder. You can, however, build upon shared Immutable Fluent Builders, which you can't with mutable Fluent Builders. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Using the nameof C# keyword with ASP.NET 3 IUrlHelper https://blog.ploeh.dk/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper 2020-08-03T10:01:00+00:00 Mark Seemann <div id="post"> <p> <em>How to generate links to other resources in a refactoring-friendly way.</em> </p> <p> I recently spent a couple of hours yak-shaving, and despite much Googling couldn't find any help on the internet. I'm surprised that the following problem turned out to be so difficult to figure out, so it may just be that I'm ignorant or that my web search skills failed me that day. On the other hand, if this really is as difficult as I found it, perhaps this article can save some other poor soul an hour or two. </p> <p> I was developing a REST API and wanted to generate some links like these: </p> <p> <pre>{ "links": [ { "rel": "urn:reservations", "href": "http://localhost:53568/reservations" }, { "rel": "urn:year", "href": "http://localhost:53568/calendar/2020" }, { "rel": "urn:month", "href": "http://localhost:53568/calendar/2020/7" }, { "rel": "urn:day", "href": "http://localhost:53568/calendar/2020/7/7" } ] }</pre> </p> <p> Like previous incarnations of the framework, ASP.NET Core 3 has <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.urlhelperextensions.action">an API for generating links to a method on a Controller</a>. I just couldn't get it to work. </p> <h3 id="6c7ec5c13e9942098dc18cbcc43b345c"> Using nameof <a href="#6c7ec5c13e9942098dc18cbcc43b345c" title="permalink">#</a> </h3> <p> I wanted to generate a URL like <code>http://localhost:53568/calendar/2020</code> in a refactoring-friendly way. While ASP.NET wants you to define HTTP resources as methods (<em>actions</em>) on <em>Controllers</em>, the various <code>Action</code> overloads want you to identify these actions and Controllers as strings. What happens if someone renames one of those methods or Controller classes? </p> <p> That's what the C# <code>nameof</code> keyword is for. </p> <p> I naively called the <code>Action</code> method like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;href&nbsp;=&nbsp;Url.Action( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Host.ToUriComponent());</pre> </p> <p> Looks good, doesn't it? </p> <p> I thought so, but it didn't work. In the time-honoured tradition of mainstream programming languages, the method just silently fails to return a value and instead returns null. That's not helpful. What might be the problem? No clue is provided. It just doesn't work. </p> <h3 id="3f7f01514e5a4c518ddd37226352970e"> Strip off the suffix <a href="#3f7f01514e5a4c518ddd37226352970e" title="permalink">#</a> </h3> <p> It turns out that the <code>Action</code> method expects the <code>controller</code> argument to <em>not</em> contain the <code>Controller</code> suffix. Not surprisingly, <code>nameof(CalendarController)</code> becomes the string <code>"CalendarController"</code>, which doesn't work. </p> <p> It took me some time to figure out that I was supposed to pass a string like <code>"Calendar"</code>. <em>That</em> works! </p> <p> As a first pass at the problem, then, I changed my code to this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;controllerName&nbsp;=&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>); <span style="color:blue;">var</span>&nbsp;controller&nbsp;=&nbsp;controllerName.Remove( &nbsp;&nbsp;&nbsp;&nbsp;controllerName.LastIndexOf( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Controller&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">StringComparison</span>.Ordinal)); <span style="color:blue;">var</span>&nbsp;href&nbsp;=&nbsp;Url.Action( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get), &nbsp;&nbsp;&nbsp;&nbsp;controller, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Host.ToUriComponent());</pre> </p> <p> That also works, and is more refactoring-friendly. You can rename both the Controller class and the method, and the link should still work. </p> <h3 id="8285d8ffd53f44808a899c33b8712bda"> Conclusion <a href="#8285d8ffd53f44808a899c33b8712bda" title="permalink">#</a> </h3> <p> The <code>UrlHelperExtensions.Action</code> methods expect the <code>controller</code> to be the 'semantic' name of the Controller, if you will - not the actual class name. If you're calling it with values produced with the <code>nameof</code> keyword, you'll have to strip the <code>Controller</code> suffix away. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Task asynchronous programming as an IO surrogate https://blog.ploeh.dk/2020/07/27/task-asynchronous-programming-as-an-io-surrogate 2020-07-27T08:27:00+00:00 Mark Seemann <div id="post"> <p> <em>Is task asynchronous programming a substitute for the IO container? An article for C# programmers.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In the previous articles, you've seen how a type like <code>IO&lt;T&gt;</code> can be used to distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. While it's an effective and elegant solution to the problem, it depends on a convention: that all impure actions return <code>IO</code> objects, which are opaque to pure functions. </p> <p> In reality, .NET base class library methods don't do that, and it's unrealistic that this is ever going to happen. It'd require a breaking reset of the entire .NET ecosystem to introduce this design. </p> <p> A comparable reset did, however, happen a few years ago. </p> <h3 id="dce87c53f2fa4b1db4c05b48e8d65870"> TAP reset <a href="#dce87c53f2fa4b1db4c05b48e8d65870" title="permalink">#</a> </h3> <p> Microsoft introduced the <a href="https://docs.microsoft.com/dotnet/csharp/programming-guide/concepts/async">task asynchronous programming</a> (TAP) model some years ago. Operations that involve <a href="https://en.wikipedia.org/wiki/Input/output">I/O</a> got a new return type. Not <code>IO&lt;T&gt;</code>, but <code>Task&lt;T&gt;</code>. </p> <p> The .NET framework team began a long process of adding asynchronous alternatives to existing APIs that involve I/O. Not as breaking changes, but by adding new, asynchronous methods side-by-side with older methods. <a href="https://docs.microsoft.com/dotnet/api/system.data.sqlclient.sqlcommand.executereaderasync">ExecuteReaderAsync</a> as an alternative to <a href="https://docs.microsoft.com/dotnet/api/system.data.sqlclient.sqlcommand.executereader">ExecuteReader</a>, <a href="https://docs.microsoft.com/dotnet/api/system.io.file.readalllinesasync">ReadAllLinesAsync</a> side by side with <a href="https://docs.microsoft.com/dotnet/api/system.io.file.readalllines">ReadAllLines</a>, and so on. </p> <p> Modern APIs exclusively with asynchronous methods appeared. For example, the <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpclient">HttpClient</a> class only affords asynchronous I/O-based operations. </p> <p> The TAP reset was further strengthened by the move from .NET to .NET Core. Some frameworks, most notably ASP.NET, were redesigned on a fundamentally asynchronous core. </p> <p> In 2020, most I/O operations in .NET are easily recognisable, because they return <code>Task&lt;T&gt;</code>. </p> <h3 id="3e84ba1d588a42689d7e8741315d22b5"> Task as a surrogate IO <a href="#3e84ba1d588a42689d7e8741315d22b5" title="permalink">#</a> </h3> <p> I/O operations are impure. Either you're receiving input from outside the running process, which is consistently non-deterministic, or you're writing to an external resource, which implies a side effect. It might seem natural to think of <code>Task&lt;T&gt;</code> as a replacement for <code>IO&lt;T&gt;</code>. <a href="https://twitter.com/SzymonPobiega/status/713001580077965312">Szymon Pobiega had a similar idea in 2016</a>, and I <a href="/2016/04/11/async-as-surrogate-io">investigated his idea in an article</a>. This was based on <a href="https://fsharp.org">F#</a>'s <code>Async&lt;'a&gt;</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a>, which is <a href="/2018/09/24/asynchronous-functors">equivalent to <code>Task&lt;T&gt;</code></a> - except when it comes to <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>. </p> <p> Unfortunately, <code>Task&lt;T&gt;</code> is far from a perfect replacement of <code>IO&lt;T&gt;</code>, because the .NET base class library (BCL) still contains plenty of impure actions that 'look' pure. Examples include <a href="https://docs.microsoft.com/dotnet/api/system.console.writeline">Console.WriteLine</a>, the parameterless <a href="https://docs.microsoft.com/dotnet/api/system.random.-ctor">Random constructor</a>, <a href="https://docs.microsoft.com/dotnet/api/system.guid.newguid">Guid.NewGuid</a>, and <a href="https://docs.microsoft.com/dotnet/api/system.datetime.now">DateTime.Now</a> (arguably a candidate for the worst-designed API in the BCL). None of those methods return tasks, which they ought to if tasks should serve as easily recognisable signifiers of impurity. </p> <p> Still, you could write asynchronous <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapters</a> over such APIs. Your <code>Console</code> Adapter might present this API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Console</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">string</span>&gt;&nbsp;ReadLine(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;WriteLine(<span style="color:blue;">string</span>&nbsp;value); }</pre> </p> <p> Moreover, the <code>Clock</code> API might look like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Clock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;GetLocalTime(); }</pre> </p> <p> Modern versions of C# enable you to write asynchronous <a href="https://en.wikipedia.org/wiki/Entry_point">entry points</a>, so the <a href="https://en.wikipedia.org/wiki/%22Hello,_World!%22_program">hello world</a> example shown in this article series becomes: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;What&#39;s&nbsp;your&nbsp;name?&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;name&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.ReadLine(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#2b91af;">Greeter</span>.Greet(now,&nbsp;name); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(greeting); }</pre> </p> <p> That's nice <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C# code, so what's not to like? </p> <h3 id="8a11bbfe9cbc4e72bd90ea6b787c0037"> No referential transparency <a href="#8a11bbfe9cbc4e72bd90ea6b787c0037" title="permalink">#</a> </h3> <p> The above <code>Main</code> example is probably as good as it's going to get in C#. I've nothing against that style of C# programming, but you shouldn't believe that this gives you compile-time checking of referential transparency. It doesn't. </p> <p> Consider a simple function like this, written using the <code>IO</code> container shown in previous articles: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;No,&nbsp;I&#39;m&nbsp;not.&quot;</span>; }</pre> </p> <p> Is this method referentially transparent? Surprisingly, despite the apparent side effect, it is. The reason becomes clearer if you write the code so that it explicitly ignores the return value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;No,&nbsp;I&#39;m&nbsp;not.&quot;</span>; }</pre> </p> <p> The <code>Console.WriteLine</code> method returns an object that represents a computation that might take place. This <code>IO&lt;Unit&gt;</code> object, however, never escapes the method, and thus never runs. The side effect never takes place, which means that the method is referentially transparent. You can replace <code>AmIEvil()</code> with its return value <code>"No, I'm not."</code>, and your program would behave exactly the same. </p> <p> Consider what happens when you replace <code>IO</code> with <code>Task</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;Yes,&nbsp;I&nbsp;am.&quot;</span>; }</pre> </p> <p> Is this method a pure function? No, it's not. The problem is that the most common way that .NET libraries return tasks is that the task is already running when it's returned. This is also the case here. As soon as you call this version of <code>Console.WriteLine</code>, the task starts running on a background thread. Even though you ignore the task and return a plain <code>string</code>, the side effect sooner or later takes place. You can't replace a call to <code>AmIEvil()</code> with its return value. If you did, the side effect wouldn't happen, and that would change the behaviour of your program. </p> <p> Contrary to <code>IO</code>, tasks don't guarantee referential transparency. </p> <h3 id="aff1e4cb382b494ea0624ef63311da81"> Conclusion <a href="#aff1e4cb382b494ea0624ef63311da81" title="permalink">#</a> </h3> <p> While it'd be technically possible to make C# distinguish between pure and impure code at compile time, it'd require such a breaking reset to the entire .NET ecosystem that it's unrealistic to hope for. It seems, though, that there's enough overlap with the design of <code>IO&lt;T&gt;</code> and task asynchronous programming that the latter might fill that role. </p> <p> Unfortunately it doesn't, because it fails to guarantee referential transparency. It's better than nothing, though. Most C# programmers have now learned that while <code>Task</code> objects come with a <a href="https://docs.microsoft.com/dotnet/api/system.threading.tasks.task-1.result">Result</a> property, you shouldn't use it. Instead, you should write your entire program using <code>async</code> and <code>await</code>. That, at least, takes you halfway towards where you want to be. </p> <p> The compiler, on the other hand, doesn't help you when it comes to those impure actions that look pure. Neither does it protect you against asynchronous side effects. Diligence, code reviews, and programming discipline are still required if you want to separate pure functions from impure actions. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Closing database connections during test teardown https://blog.ploeh.dk/2020/07/20/closing-database-connections-during-test-teardown 2020-07-20T07:20:00+00:00 Mark Seemann <div id="post"> <p> <em>How to close database connections to SQL Server during integration testing teardown.</em> </p> <p> Whenever I need to run integration tests that involve SQL Server, I have a standard approach that <a href="https://docs.microsoft.com/en-us/archive/blogs/ploeh/sqlexpressdatabaseinstaller">I've evolved since 2007</a>. It involves <ol> <li>setting up a SQL Server Express database before each test</li> <li>running the test</li> <li>tearing down the database</li> </ol> I've explained the basic process in details in my <a href="/outside-in-tdd">Outside-In Test-Driven Development</a> Pluralsight course. You can also see it in action in <a href="https://github.com/ploeh/dependency-injection-revisited">this GitHub repository</a>. </p> <p> One problem with that approach is that SQL Server doesn't allow you to delete a database if it has existing connections. </p> <h3 id="7de0c4f8bd3d40e99740a39b11570593"> Turn off connection pooling <a href="#7de0c4f8bd3d40e99740a39b11570593" title="permalink">#</a> </h3> <p> I usually solve the problem by turning off connection pooling. For an integration test suite, this is fine. I usually use integration testing to verify functionality - not performance. </p> <p> Turning off connection pooling is easily done by setting the flag to <code>false</code> in the connection string: </p> <p> <code>Server=(LocalDB)\MSSQLLocalDB;Database=Booking;Integrated Security=true;Pooling=false</code> </p> <p> This means that when you get to the teardown phase of the test, you can issue a DDL statement to the <code>master</code> database: </p> <p> <pre>IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'Booking') DROP DATABASE [Booking]</pre> </p> <p> When connection pooling is turned off, no other connections are open when you attempt to do that, and the database (here named <code>Booking</code>) is deleted. </p> <h3 id="b599a9242a7d44f1b23241341db054b8"> Forcibly close other connections <a href="#b599a9242a7d44f1b23241341db054b8" title="permalink">#</a> </h3> <p> Recently, however, I ran into a testing scenario where connection pooling had to be turned on. When you turn on connection pooling, however, the above <code>DROP DATABASE</code> statement fails because at least one connection from the pool is still connected to the database. </p> <p> To solve that issue, I forcibly close other connections during teardown: </p> <p> <pre>IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'Booking') BEGIN -- This closes existing connections: ALTER DATABASE [Booking] SET SINGLE_USER WITH ROLLBACK IMMEDIATE DROP DATABASE [Booking] END</pre> </p> <p> Surprisingly, turning on connection pooling like this makes the integration tests slower. I suppose it's because throwing other connections off the database involves a bit of negotiation between the server and the clients, and that takes some time. </p> <p> While slower, it does enable you to run the integration tests with connection pooling turned on. When you need it, you need it. </p> <h3 id="6a3cf6b38c0f49a3abb1aa8ccbf50644"> Summary <a href="#6a3cf6b38c0f49a3abb1aa8ccbf50644" title="permalink">#</a> </h3> <p> You can run integration tests against a SQL Server Express database. People do it in various ways, but I've found that setting up and tearing down a pristine database for each test case is a robust and maintainable solution to the problem. </p> <p> SQL Server will not, however, allow you to delete a database if other connections exist. The easiest and fastest solution to that problem is to turn off connection pooling. </p> <p> Sometimes, you can't do that, so instead, you can expand your database teardown script so that it closes existing connections before it deletes the database. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="0a9ea1fffa1c41ae96981ff4625c4985"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> This sounds like a great approach. I have been on projects with tests that involved the database, but none of them were designed as well as this. I will be sure to come back to this post when we add a database to my current project. </p> <p> My understanding is that <a href="https://expressdb.io/sql-server-express-vs-localdb.html#what-is-localdb">SQL Server Express and LocalDB are not the same thing</a>. Are you using SQL Server Express or LocalDB? Do you prefer one over the other for this database testing approach of yours? </p> </div> <div class="comment-date">2020-07-25 19:58 UTC</div> </div> <div class="comment" id="4fd6901499a44fc084d58673aa762cc1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. It's not really my area of expertise. I use the one bundled with Visual Studio, so I suppose that's actually LocalDB, and not SQL Server Express. </div> <div class="comment-date">2020-07-26 13:33 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Implementation of the C# IO container https://blog.ploeh.dk/2020/07/13/implementation-of-the-c-io-container 2020-07-13T06:02:00+00:00 Mark Seemann <div id="post"> <p> <em>Implementation details of the C# IO container.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In the previous articles, you've seen how a type like <code>IO&lt;T&gt;</code> can be used to distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. </p> <p> The point of the article series is to illustrate the concept that <a href="https://www.haskell.org">Haskell</a> uses to impose the <a href="/2018/11/19/functional-architecture-a-definition">functional interaction law</a> at compile time. The implementation details really aren't important. Still, I believe that I know my readership well enough that a substantial fraction would be left unsatisfied if I didn't share the implementation details. </p> <p> I consider this an appendix to the article series. It's not really what this is all about, but here it is, nonetheless. </p> <h3 id="dcf2521a1b8c40df9d45e97a43cfc546"> Constructor <a href="#dcf2521a1b8c40df9d45e97a43cfc546" title="permalink">#</a> </h3> <p> Based on the public API <a href="/2020/06/15/io-container-in-a-parallel-c-universe">already published</a>, the constructor implementation hardly comes as a surprise. </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; }</pre> </p> <p> The <code>IO&lt;T&gt;</code> class is little more than a wrapper around a lazily evaluated function, with the restriction that while you can put a <code>Func&lt;T&gt;</code> object into an <code>IO</code> object, you can never get it out again. Thus, the <code>item</code> is a <code>private</code> class field instead of a public property. </p> <h3 id="50d90cc423f04f5bb4e31a0d6c9b12c8"> SelectMany <a href="#50d90cc423f04f5bb4e31a0d6c9b12c8" title="permalink">#</a> </h3> <p> The <code>SelectMany</code> method is a little more tricky: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;selector(item()).item()); }</pre> </p> <p> To guarantee <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>, we don't want the method to trigger evaluation of the lazy value, so the <code>selector</code> has to run inside a new lazy computation. This produces a lazy <code>IO</code> value that the method then has to unwrap inside another lazy computation. Such a translation from <code>Func&lt;IO&lt;TResult&gt;&gt;</code> to a new <code>IO</code> object with a <code>Func&lt;TResult&gt;</code> inside it is reminiscent of what in Haskell is known as a <em>traversal</em>. </p> <h3 id="b6c088710f6340689376b0ddf3b7df01"> UnsafePerformIO <a href="#b6c088710f6340689376b0ddf3b7df01" title="permalink">#</a> </h3> <p> Finally, the <code>UnsafePerformIO</code> method isn't part of the API, but as explained in the <a href="/2020/07/06/referential-transparency-of-io">previous article</a>, this is the special method that the hypothetical parallel-world framework calls on the <code>IO&lt;Unit&gt;</code> returned by <code>Main</code> methods. </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item(); }</pre> </p> <p> Since only the framework is supposed to call this method, it's <code>internal</code> by design. The only thing it does is to force evaluation of the lazy <code>item</code>. </p> <h3 id="38830b00fbbf4911b8a7ebd723e138ea"> Conclusion <a href="#38830b00fbbf4911b8a7ebd723e138ea" title="permalink">#</a> </h3> <p> Most of the implementation of <code>IO&lt;T&gt;</code> is straightforward, with the single exception of <code>SelectMany</code>, which has to jump through a few hoops to keep the behaviour lazy until it's activated. </p> <p> Once more I want to point out that the purpose of this article series is to explain how a type system like Haskell's guarantees referential transparency. C# could do the same, but it'd require that all impure actions in all libraries in the entire .NET ecosystem were to return <code>IO</code> values. That's not going to happen, but something similar already has happened. Read on in the next article. </p> <p> <strong>Next:</strong> <a href="/2020/07/27/task-asynchronous-programming-as-an-io-surrogate">Task asynchronous programming as an IO surrogate</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="8fea4b2ac22d4211bd91d352c7b4e55e"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> In your <a href="https://blog.ploeh.dk/2020/07/13/implementation-of-the-c-io-container/">previous post</a>, you said </p> <blockquote> Haskell is a lazily evaluated language. That's an important piece of the puzzle, so I've modelled the <code>IO&lt;T&gt;</code> example so that it only wraps <code>Lazy</code> values. That emulates Haskell's behaviour in C#. </blockquote> <p> After several days, I finally feel like I fully understand this. </p> <p> The concept of lazy has serveral slightly different definitions depending on the context. Haskell is a lazily evaluated language in the sense that its evaluation strategy is <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_need">call by need</a>. In C#, both <code>Lazy&lt;T&gt;</code> and <code>Func&lt;T&gt;</code> are lazy in the sense that neither actually contains a <code>T</code>, but both could produce a <code>T</code> if asked to do so. The difference is the presence or absence of caching. I remember all this by saying that <code>Lazy&lt;T&gt;</code> is lazy with caching and <code>Func&lt;T&gt;</code> is lazy without caching. So <code>Lazy&lt;T&gt;</code> is to call by need as <code>Func&lt;T&gt;</code> is to <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name">call by name</a>. </p> <p> Therefore, <code>Lazy&lt;T&gt;</code> is the correct choice if we want to model or emulate the evaluation strategy of Haskell in C#. What about Haskell's <code>IO&lt;T&gt;</code>? Is it lazy with caching or lazy without caching? My guess was lazy without caching, but I finally installed the ghc Haskell compiler and compiled Haskell on my machine for the first time in order to test this. I think this example shows that Haskell's <code>IO&lt;T&gt;</code> is lazy without caching. </p> <p> <pre>-- output: xx main = let io = putStr "x" in do { io ; io }</pre> </p> <p> I think this would be equivalent C# code in this parallel world that you have created. </p> <p> <pre>// output: x static IO<Unit> MainIO(string[] args) { var io = Console.Write("x"); return from _1 in io from _2 in io select Unit.Instance; }</pre> </p> <p> What makes me think that I fully understand this now is that I think I see where you are going. I think you already knew all this and decided to model Haskell's <code>IO&lt;T&gt;</code> using <code>Lazy&lt;T&gt;</code> anyway because <code>Task&lt;T&gt;</code> is also lazy with caching just like <code>Lazy&lt;T&gt;</code>, and your next post will discuss using <code>Task&lt;T&gt;</code> as a surrogate for Haskell's <code>IO&lt;T&gt;</code>. I think you want your C# implementation of <code>IO&lt;T&gt;</code> to be more like C#'s <code>Task&lt;T&gt;</code> than Haskell's <code>IO&lt;T&gt;</code>. </p> <p> Thank you for including such a gem for me to think about...and enough motivation to finally put Haskell on my machine! </p> </div> <div class="comment-date">2020-07-13 20:46 UTC</div> </div> <div class="comment" id="ba8de284a15c49529bb10390c33d2aa1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. You've almost turned my blog into a peer-reviewed journal, and you've just pointed out a major blunder of mine 👍 </p> <p> I think I was mislead by the name <code>Lazy</code>, my attention was elsewhere, and I completely forgot about the memoisation that both <code>Lazy&lt;T&gt;</code> and <code>Task&lt;T&gt;</code> employ. It does turn out to be problematic in this context. Take the following example: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;getTime&nbsp;=&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getTime.SelectMany(t1&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(t1.Ticks.ToString()).Select(u1&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Thread</span>.Sleep(2); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">Unit</span>.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}).SelectMany(u2&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getTime).SelectMany(t2&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(t2.Ticks.ToString()))); }</pre> </p> <p> Notice that this example <em>reuses</em> <code>getTime</code> twice. We'd like any <code>IO&lt;T&gt;</code> value to represent an impure computation, so evaluating it twice with a 2 millisecond delay in between ought to yield two different results. </p> <p> Due to the memoisation built into <code>Lazy&lt;T&gt;</code>, the first value is reused. That's not the behaviour we'd like to see. </p> <p> While this is a rather big error on my part, it's fortunately only of a technical nature (I think). As you imply, the correct implementation is to use <code>Func&lt;T&gt;</code> rather than <code>Lazy&lt;T&gt;</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;selector(item()).item()); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This addresses the above reuse bug. With this implementation, the above <code>Main</code> method prints two different values, even though it reuses <code>getTime</code>. </p> <p> Haskell <code>IO</code> doesn't memoise values, so this <code>Func</code>-based implementation better emulates the Haskell behaviour, which is actually what I wanted to do all along. </p> <p> This mistake of mine is potentially so confusing that I think that it's best if I go back and edit the other posts in this articles series. Before I do that, though, I'd like to get your feedback. </p> <p> Does this look better to you, or do you see other problems with it? </p> </div> <div class="comment-date">2020-07-19 14:32 UTC</div> </div> <div class="comment" id="f14bcdf1d46245278ef8694fd6b73d3b"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> Tyson, thank you for writing. You've almost turned my blog into a peer-reviewed journal, and you've just pointed out a major blunder of mine 👍 </blockquote> <p> You're welcome. I am certainly a peer, and I benefit greatly from closly reviewing your posts. They always give me so much to think about. I am happy that we both benefit from this :) </p> <blockquote> <p> While this is a rather big error on my part, it's fortunately only of a technical nature (I think). ... </p> <p> Haskell <code>IO</code> doesn't memoise values, so this <code>Func</code>-based implementation better emulates the Haskell behaviour, which is actually what I wanted to do all along. </p> <p> This mistake of mine is potentially so confusing that I think that it's best if I go back and edit the other posts in this articles series. Before I do that, though, I'd like to get your feedback. </p> <p> Does this look better to you, or do you see other problems with it? </p> </blockquote> <p> Yes, your <code>Func&lt;T&gt;</code>-based implementation better emulates Haskell's <code>IO&lt;T&gt;</code>. My guess was that you had used <code>Lazy&lt;T&gt;</code> with its caching behavior in mind. I do think it is a minor issue. I can't think of any code that I would write on purpose that would depend on this difference. </p> <p> I think editing the previous posts depends on exactly how you want to suggesst <code>Task&lt;T&gt;</code> as an <code>IO&lt;T&gt;</code> surrogate. </p> <p> From a purely teaching perspective, I think I prefer to first implement Haskell's <code>IO&lt;T&gt;</code> in C# using <code>Func&lt;T&gt;</code>, then suggest this implemention is essentialy the same as <code>Task&lt;T&gt;</code>, then point out the caching difference for those that are still reading. It would be a shame to lose some readers eariler by pointing out the difference too soon. I wouldn't expect you lost any readres in your current presentation that includes the caching difference but without mentioning it. </p> <p> <code>Func&lt;T&gt;</code>, <code>Lazy&lt;T&gt;</code>, and <code>Task&lt;T&gt;</code> are all lazy (in the sense that none contain a <code>T</code> but all could produce one if requested. Here are their differences: <ul> <li><code>Func&lt;T&gt;</code> is synchronous without caching,</li> <li><code>Lazy&lt;T&gt;</code> is synchronous with caching, and</li> <li><code>Task&lt;T&gt;</code> is asynchronous with caching.</li> </ul> We are missing a type that is asynchronous without caching. We can create such behavior though using <code>Func&lt;T&gt;</code> and <code>Task&lt;T&gt;</code>. The nested type <code>Func&lt;Task&lt;T&gt;&gt;</code> can asynchronously produce a <code>T</code> without caching. Maybe this would be helpful in this article series. </p> <p> Overall though, I don't know of any other potential changes to consider. </p> </div> <div class="comment-date">2020-07-19 21:19 UTC</div> </div> <div class="comment" id="689c56123f7b469a9796e0793f48b294"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> For any reader following the discussion after today (July 24, 2020), it may be slightly confusing. Based on <a href="#8fea4b2ac22d4211bd91d352c7b4e55e">Tyson Williams' feedback</a>, I've edited the article series with <a href="#ba8de284a15c49529bb10390c33d2aa1">the above implementation</a>. The previous incarnation of the article series had this implementation of <code>IO&lt;T&gt;</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;res&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;(()&nbsp;=&gt;&nbsp;selector(item.Value)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;res.Value.item.Value)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item.Value; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As this discussion reveals, the memoisation performed by <code>Lazy&lt;T&gt;</code> causes problems. After thinking it through, I've decided to retroactively change the articles in the series. This is something that I rarely do, but in this case I think is the best way forward, as the <code>Lazy</code>-based implementation could be confusing to new readers. </p> <p> Readers interested in the history of these articles can peruse <a href="https://github.com/ploeh/ploeh.github.com">the Git log</a>. </p> </div> <div class="comment-date">2020-07-24 8:22 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Referential transparency of IO https://blog.ploeh.dk/2020/07/06/referential-transparency-of-io 2020-07-06T05:56:00+00:00 Mark Seemann <div id="post"> <p> <em>How the IO container guarantees referential integrity. An article for object-oriented programmers.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In <a href="/2020/06/15/io-container-in-a-parallel-c-universe">a previous article</a> you got a basic introduction to the <code>IO&lt;T&gt;</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> I use to explain how a type system like <a href="https://www.haskell.org">Haskell</a>'s distinguishes between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. </p> <p> The whole point of the <code>IO</code> container is to effectuate the <a href="/2018/11/19/functional-architecture-a-definition">functional interaction law</a>: <em>a pure function mustn't be able to invoke an impure activity.</em> This rule follows from <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>. </p> <p> The practical way to think about it is to consider the two rules of pure functions: <ul> <li>Pure functions must be deterministic</li> <li>Pure functions may have no side effects</li> </ul> In this article, you'll see how <code>IO&lt;T&gt;</code> imposes those rules. </p> <h3 id="3b648ec5b32a42e4b608dbc2e8861a86"> Determinism <a href="#3b648ec5b32a42e4b608dbc2e8861a86" title="permalink">#</a> </h3> <p> Like in the previous articles in this series, you must imagine that you're living in a parallel universe where all impure library functions return <code>IO&lt;T&gt;</code> objects. By elimination, then, methods that return naked values must be pure functions. </p> <p> Consider the <code>Greet</code> function from the <a href="/2020/06/15/io-container-in-a-parallel-c-universe">previous article</a>. Since it returns a plain <code>string</code>, you can infer that it must be a pure function. What prevents it from doing something impure? </p> <p> What if you thought that passing <code>now</code> as an argument is a silly design. Couldn't you just call <code>Clock.GetLocalTime</code> from the method? </p> <p> Well, yes, in fact you can: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;now1&nbsp;=&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> This compiles, but is only the first refactoring step you have in mind. Next, you want to extract the <code>DateTime</code> from <code>now1</code> so that you can get rid of the <code>now</code> parameter. Alas, you now run into an insuperable barrier. <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">How do you get the DateTime out of the IO?</a> </p> <p> You can't. By design. </p> <p> While you can call the <code>GetLocalTime</code> method, you can't <em>use</em> the return value. The only way you can use it is by composing it with <code>SelectMany</code>, but that still accomplishes nothing unless you return the resulting <code>IO</code> object. If you do that, though, you've now turned the entire <code>Greet</code> method into an impure action. </p> <p> You can't perform any non-deterministic behaviour inside a pure function. </p> <h3 id="9f62e432e0c74fda93aff427774c0183"> Side effects <a href="#9f62e432e0c74fda93aff427774c0183" title="permalink">#</a> </h3> <p> How does <code>IO&lt;T&gt;</code> protect against side effects? In the <code>Greet</code> method, couldn't you just write to the console, like the following? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> This also compiles, despite our best efforts. That's unfortunate, but, as you'll see in a moment, it doesn't violate referential transparency. </p> <p> In Haskell or <a href="https://fsharp.org">F#</a> equivalent code would make the compiler complain. Those compilers don't have special knowledge about <code>IO</code>, but they can see that an action returns a value. F# generates a compiler warning if you ignore a return value. In Haskell the story is a bit different, but the result is the same. Those compilers complain because you try to ignore the return value. </p> <p> You can get around the issue using the language's wildcard pattern. This tells the compiler that you're actively ignoring the result of the action. You can do the same in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> The situation is now similar to the above treatment of non-determinism. While there's no value of interest in an <code>IO&lt;Unit&gt;</code>, the fact that there's an object at all is a hint. Like <code>Lazy&lt;T&gt;</code>, that value isn't a result. It's a placeholder for a computation. </p> <p> If there's a way to make the C# compiler complain about ignored return values, I'm not aware of it, so I don't know if we can get closer to how Haskell works than this. Regardless, keep in mind that I'm not trying to encourage you to write C# like this; I'm only trying to explain how Haskell enforces referential transparency at the type level. </p> <h3 id="4f40c1e1a7a142b693d050dcab466d90"> Referential transparency <a href="#4f40c1e1a7a142b693d050dcab466d90" title="permalink">#</a> </h3> <p> While the above examples compile without warnings in C#, both are still referentially transparent! </p> <p> This may surprise you. Particularly the second example that includes <code>Console.WriteLine</code> looks like it has a side effect, and thus violates referential transparency. </p> <p> Keep in mind, though, what referential transparency means. It means that you can replace a particular function call with its value. For example, you should be able to replace the function call <code>Greeter.Greet(new DateTime(2020, 6, 4, 12, 10, 0), "Nearth")</code> with its return value <code>"Hello, Nearth."</code>, or the function call <code>Greeter.Greet(new DateTime(2020, 6, 4, 7, 30, 0), "Bru")</code> with <code>"Good morning, Bru."</code>, without changing the behaviour of the software. This property still holds. </p> <p> Even when the <code>Greet</code> method includes the <code>Console.WriteLine</code> call, that side effect never happens. </p> <p> The reason is that an <code>IO</code> object represents a potential computation that may take place (also known as a <em>thunk</em>). Notice that the <code>IO&lt;T&gt;</code> constructor takes a <code>Func</code> as argument. It's basically just a lazily evaluated function wrapped in such a way that you can't force evaluation. </p> <p> Instead, you should imagine that <em>after</em> <code>Main</code> returns its <code>IO&lt;Unit&gt;</code> object, the parallel-universe .NET framework executes it. </p> <p> <img src="/content/binary/parallel-universe-fx-calling-unsafeperformio.png" alt="Diagram showing that the parrallel-universe framework executes the IO value after Main returns."> </p> <p> The framework supplies command-line arguments to the <code>Main</code> method. Once the method returns an <code>IO&lt;Unit&gt;</code> object, the framework executes it with a special method that only the framework can invoke. Any other <code>IO</code> values that may have been created (e.g. the above <code>Console.WriteLine</code>) never gets executed, because they're not included in the return value. </p> <h3 id="d5fe0e97d9ea4442b3b4fb13ad28aab6"> Conclusion <a href="#d5fe0e97d9ea4442b3b4fb13ad28aab6" title="permalink">#</a> </h3> <p> The <code>IO</code> container makes sure that pure functions maintain referential transparency. The underlying assumption that makes all of this work is that <em>all</em> impure actions return <code>IO</code> objects. That's not how the .NET framework works, but that's how Haskell works. Since the <code>IO</code> container is opaque, pure functions can't see the contents of <code>IO</code> boxes. </p> <p> Program <a href="https://en.wikipedia.org/wiki/Entry_point">entry points</a> are all impure. The return value of the entry point must be <code>IO&lt;Unit&gt;</code>. The hypothetical parallel-universe framework executes the <code>IO</code> value returned by <code>Main</code>. </p> <p> Haskell is a lazily evaluated language. That's an important piece of the puzzle, so I've modelled the <code>IO&lt;T&gt;</code> example so that it only wraps lazily evaluated <code>Func</code> values. That emulates Haskell's behaviour in C#. In the next article, you'll see how I wrote the C# code that supports these articles. </p> <p> <strong>Next:</strong> <a href="/2020/07/13/implementation-of-the-c-io-container">Implementation of the C# IO container</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="52e8a987dc234e839d259cc3296176d7"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> In a <a href="https://blog.ploeh.dk/2020/02/24/discerning-and-maintaining-purity/">previous post</a>, you said </p> <blockquote> ...a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a> has to obey two rules: <ul> <li>The same input always produces the same output.</li> <li>Calling it causes no side effects.</li> </ul> </blockquote> <p> In this post, you said </p> <blockquote>...the two rules of pure functions: <ul> <li>Pure functions must be deterministic</li> <li>Pure functions may have no side effects</li> </ul> </blockquote> <p> The first two items are not the same. Is this difference intentional? </p> </div> <div class="comment-date">2020-07-06 11:07 UTC</div> </div> <div class="comment" id="73bfc62f4f464ac2b5a6131e95884c11"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. Indeed, the words er verbatim not the same, but I do intend them to carry the same meaning. </p> <p> If one wants to define purity in a way that leaves little ambiguity, one has to use more words. Just look at the linked Wikipedia article. I link to Wikipedia for the benefit of those readers who'd like to get a more rigorous definition of the term, while at the same time I enumerate the two rules as a summary for the benefit of those readers who just need a reminder. </p> <p> Does that answer your question? </p> </div> <div class="comment-date">2020-07-06 11:26 UTC</div> </div> <div class="comment" id="1338dd35c4c6405f861ffa26034b13e6"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> ...I do intend them to carry the same meaning. </blockquote> <p> I don't think they don't mean the same thing. That is part of the discussion we are having on <a href="https://blog.ploeh.dk/2020/02/24/discerning-and-maintaining-purity/">this previous post</a>. I think the simplest example to see the difference is randomzied quicksort. For each input, the output is always the same. However, randomized quicksort is not deterministic because it uses randomness. </p> <p> Do you still think they mean the same thing? </p> </div> <div class="comment-date">2020-07-06 19:35 UTC</div> </div> <div class="comment" id="247a2c05d83a485a99225057829afa71"> <div class="comment-author"><a href="http://www.raboof.com/">Atif Aziz</a></div> <div class="comment-content"> <blockquote> You can get around the issue using the language's wildcard pattern. This tells the compiler that you're actively ignoring the result of the action. </blockquote> <p> You made a standard variable declaration and I believe you meant to use a <a href="https://docs.microsoft.com/en-us/dotnet/csharp/discards#a-standalone-discard">stand-alone discard</a> rather than a wildcard pattern? Like below? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> </div> <div class="comment-date">2020-07-09 08:18 UTC</div> </div> <div class="comment" id="dcd18dda3cf742ffb0bba38ce0326b66"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, I apologise in advance for my use of <a href="https://en.wikipedia.org/wiki/Weasel_word">weasel words</a>, but 'in the literature', a function that always returns the same output when given the same input is called 'deterministic'. I can't give you a comprehensive list of 'the literature' right now, but <a href="https://en.wikipedia.org/wiki/Deterministic_algorithm">here's at least one example</a>. </p> <p> I'm well aware that this might be confusing. One could argue that querying a database is deterministic, because the output is completely determined by the state of the database. The same goes for reading the contents of a file. Such operations, however, may return different outputs for the same inputs, as the state of the resource changes. There's no stochastic process involved in such state changes, but we still consider such actions non-deterministic. </p> <p> In the same vein, in this jargon, 'deterministic' doesn't imply the absence of internal randomness, as <a href="/2020/02/24/discerning-and-maintaining-purity#a06da27b57944c89aefc16914234939c">you have so convincingly argued</a>. In the present context, 'deterministic' is <em>defined</em> as the property that a given input value always produces the same output value. </p> <p> That's the reason I tend to use those phrases interchangeably. In this context, they literally mean the same. I can see how this might be confusing, though. </p> </div> <div class="comment-date">2020-07-09 10:57 UTC</div> </div> <div class="comment" id="d0dc32b4548a4d4ab5d55652a7b42502"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Atif, thank you for writing. I didn't know about that language construct. Thank you for pointing it out to me! </p> </div> <div class="comment-date">2020-07-09 18:11 UTC</div> </div> <div class="comment" id="956af671404548038caa603fede12e88"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> <p> [<a href="https://en.wikipedia.org/wiki/Deterministic_algorithm">Wikipedia</a> says that] a function that always returns the same output when given the same input is called 'deterministic'. </p> <p> In the same vein, in this jargon, 'deterministic' doesn't imply the absence of internal randomness, as <a href="/2020/02/24/discerning-and-maintaining-purity#a06da27b57944c89aefc16914234939c">you have so convincingly argued</a>. In the present context, 'deterministic' is <em>defined</em> as the property that a given input value always produces the same output value. </p> <p> That's the reason I tend to use those phrases interchangeably. In this context, they literally mean the same. I can see how this might be confusing, though. </p> </blockquote> <p> I am certainly still confused.  I can't tell if you are purposefully providing your own definition for determinism, if you are accidentally misunderstanding the actual definition of determinism, if you think the definition of determinism is equivalent to the definition you gave due to being in some context or operating with an additional assumption (of which I am unsure), or maybe something else. </p> <p> If I had to guess, then I think you do see the difference between the two definitions but are claiming that are equivalent due to being in some context or operating with an additional assumption.  If this guess is correct, then what is the additional assumption you are making? </p> <p> To ensure that you do understand determinism, I will interpret your statements as literally as possible and then respond to them.  I apologize in advance if this is not the correct interpretation and for any harshness in the tone of my response. </p> <p> You misquoted the Wikipedia article.  Here is the exact text (after replacing "algorithm" with "function" since the difference between these is not important to us now). </p> <blockquote> a deterministic function is a function which, given a particular input, will always produce the same output, with the underlying machine always passing through the same sequence of states. </blockquote> <p> Your quotation essentially excludes the phrase "with the underlying machine always passing through the same sequence of states".  Let's put these two definitions next to each other. </p> <p> Let <code>f</code> be a function.  Consider the following two statements about <code>f</code> that might or might not be true (depending on the exact value of <code>f</code>). <ol> <li>Given a particular input, <code>f</code> will always produce the same output.</li> <li>Given a particular input, <code>f</code> will always produce the same output, with the underlying machine always passing through the same sequence of states.</li> </ol> </p> <p> Suppose some <code>f</code> satisfies statement 2.  Then <code>f</code> clearly satisfies statement 1 as well.  These two sentences together are equivalent to saying that statement 2 implies statement 1.  The <a href="https://en.wikipedia.org/wiki/Contraposition">contrapositive</a> then says that not satisfying statement 1 implies not satisfying statement 2.   Also, Wikipedia says that such an <code>f</code> is said to be deterministic. </p> <p> Now suppose some <code>f</code> satisfies statement 1.  Then <code>f</code> need not also satisfy statement 2.  An example that proves this is randomized quicksort (as I pointed out in <a href="https://blog.ploeh.dk/2020/02/24/discerning-and-maintaining-purity/#a06da27b57944c89aefc16914234939c">this comment</a>).  This means that statement 1 does not imply statement 2. </p> <p> "Wikipedia gave" a name to functions that satisfy statement 2.  I do not recall ever seeing anyone give a name to functions that satisfy statement 1.  In <a href="https://blog.ploeh.dk/2020/02/24/discerning-and-maintaining-purity/#9f9664e29c3f4fafa786abae039eb20e">this comment</a>, you asked me about what I meant by "weak determinism".  I am trying to give a name to functions that satisfy statement 1.  I am suggesting that we call them weakly deterministic.  This name allows us to say that a deterministic function is also weakly deterministic, but a weakly deterministic function might not be deterministic.  Furthermore, not being a weakly deterministic implies not being deterministic. </p> <blockquote> One could argue that querying a database is deterministic, because the output is completely determined by the state of the database. The same goes for reading the contents of a file. Such operations, however, may return different outputs for the same inputs, as the state of the resource changes. There's no stochastic process involved in such state changes, but we still consider such actions non-deterministic. </blockquote> <p> Indeed.  I agree.  If we distinguish determinism from weak determinism, then we can say that such a function is not weakly deterministic, which implies that it is not deterministic. </p> </div> <div class="comment-date">2020-07-10 01:36 UTC</div> </div> <div class="comment" id="c53873f100b2480296bfc0ea0709e97d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, I'm sorry, I picked a bad example. It's possible that my brain is playing a trick with me. I'm not <em>purposefully</em> providing my own definition of determinism. </p> <p> I've learned all of this from diverse sources, and I don't recall all of them. Some of them are books, some were conference talks I attended, some may have been conversations, and some online resources. All of that becomes an amalgam of knowledge. <em>Somewhere</em> I've picked up the shorthand that 'deterministic' is the same as 'the same input produces the same output'; I can't tell you the exact source of that habit, and it may, indeed, be confusing. </p> <p> It seems that I'm not the only person with that habit, though: <blockquote> <p> "Deterministic: They return the same output for the same input." </p> <footer><cite><a href="https://twitter.com/jdegoes/status/936301872066977792">John A De Goes</a></cite></footer> </blockquote> I'm not saying that John A De Goes is an authority you should unquestionably accept; I'm only pointing to that tweet to illustrate that I'm not the only person who occasionally use 'deterministic' in that way. And I don't think that John A De Goes picked up the habit from me. </p> <p> The key to all of this is referential transparency. In an ideal world, I wouldn't need to use the terms 'pure function' or 'deterministic'. I can't, however, write these articles and only refer to referential transparency. The articles are my attempt to share what I have learned with readers who would also like to learn. The problem with referential transparency is that it's an abstract concept: <em>can I replace a function call with its output?</em> This may be a tractable notion to pick up, but how do you evaluate that in practice? </p> <p> I believe that it's easier for readers to learn that they have to look for two properties: <ul> <li>Does the same input always produce the same output?</li> <li>Are there no side effects?</li> </ul> As far as I can tell, these two properties are enough to guarantee referential transparency. Again: this isn't a claim that <em>I'm</em> making; it's just my understanding based on what I've picked up so far. </p> <p> As all programmers know: language is imprecise. Even the above two bullets are vague. What's a side effect? Does the rule about input and output apply to all values in the function's domain? </p> <p> I don't think that it's possible to write perfectly precise and unambiguous prose. Mathematical notation was developed as an attempt to be more precise and unambiguous. Source code has the same quality. </p> <p> I'm not writing mathematical treatises, but I use C#, F#, and Haskell source code to demonstrate concepts as precisely as I can. The surrounding prose is my attempt at explaining what the code does, and why it's written the way it is. The prose will be ambiguous; I can't help it. </p> <p> Sometimes, I need a shorthand to remind the reader about referential transparency in a way that (hopefully) assists him or her. Sometimes, this is best done by using a few adjectives, such as <em>"a deterministic and side-effect-free function"</em>. It's not a definition, it's a reading aid. </p> </div> <div class="comment-date">2020-07-25 8:38 UTC</div> </div> <div class="comment" id="fa3e2af2ea6b498cb8b94baf594d1e6f"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Mark, I am also sorry. As I reread my comment, I think I was too critical. I admire your great compassion and humility, especially in your writing. I have been trying to write more like you, but my previous comment shows that I still have room for improvement. Your blog is still my favorite place to read and learn about software development and functional programming. It is almost scary how much I agree with you. I hope posting questions about my confusion or comments about our differences don't overshadow all the ways in which we agree and sour our relationship. </p> </div> <div class="comment-date">2020-07-25 15:15 UTC</div> </div> <div class="comment" id="dcd28380339f4db8955efd235ef125b6"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, don't worry about it. Questions teach me where there's room for improvement. You also sometimes <a href="/2020/07/13/implementation-of-the-c-io-container#8fea4b2ac22d4211bd91d352c7b4e55e">point out genuine mistakes that I make</a>. If you didn't do that, I might never realise my mistakes, and then I wouldn't be able to correct them. </p> <p> I appreciate the feedback because it improves the content and teaches me things that I didn't know. On the other hand, as the cliché goes, <em>all errors are my own.</em> </p> </div> <div class="comment-date">2020-07-29 9:56 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Syntactic sugar for IO https://blog.ploeh.dk/2020/06/29/syntactic-sugar-for-io 2020-06-29T05:49:00+00:00 Mark Seemann <div id="post"> <p> <em>How to make use of the C# IO container less ugly.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In <a href="/2020/06/15/io-container-in-a-parallel-c-universe">the previous article</a> you saw a basic C# <a href="https://en.wikipedia.org/wiki/%22Hello,_World!%22_program">hello world</a> program using <code>IO&lt;T&gt;</code> to explicitly distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. The code wasn't as pretty as you could hope for. In this article, you'll see how to improve the aesthetics a bit. </p> <p> The code isn't going to be perfect, but I think it'll be better. </p> <h3 id="c919d892a9374cd4a816693c62840620"> Sugared version <a href="#c919d892a9374cd4a816693c62840620" title="permalink">#</a> </h3> <p> The <code>IO&lt;T&gt;</code> container is an imitation of the <a href="https://www.haskell.org">Haskell</a> <code>IO</code> type. In Haskell, <code>IO</code> is a monad. This isn't a monad tutorial, and I hope that you're able to read the article without a deep understanding of monads. I only mention this because when you compose monadic values with each other, you'll sometimes have to write some 'noisy' code - even in Haskell. To alleviate that pain, Haskell offers syntactic sugar in the form of so-called <code>do</code> notation. </p> <p> Likewise, <a href="https://fsharp.org">F#</a> comes with <a href="https://docs.microsoft.com/dotnet/fsharp/language-reference/computation-expressions">computation expressions</a>, which also gives you syntactic sugar over monads. </p> <p> C#, too, comes with syntactic sugar over monads. This is <em>query syntax</em>, but it's not as powerful as Haskell <code>do</code> notation or F# computation expressions. It's powerful enough, though, to enable you to improve the <code>Main</code> method from the previous article: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">from</span>&nbsp;&nbsp;&nbsp;&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;What&#39;s&nbsp;your&nbsp;name?&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;name&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Console</span>.ReadLine() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;&nbsp;now&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#2b91af;">Greeter</span>.Greet(now,&nbsp;name) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;&nbsp;res&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(greeting) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;res; }</pre> </p> <p> If you use C# query syntax at all, you may think of it as exclusively the realm of object-relational mapping, but in fact it works for any monad. There's no data access going on here - just the interleaving of pure and impure code (in an <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>, even). </p> <h3 id="326b00adfdd347e684f5adfc90b001e6"> Infrastructure <a href="#326b00adfdd347e684f5adfc90b001e6" title="permalink">#</a> </h3> <p> For the above code to compile, you must add a pair of methods to the <code>IO&lt;T&gt;</code> API. You can write them as extension methods if you like, but here I've written them as instance methods on <code>IO&lt;T&gt;</code>. </p> <p> When you have multiple <code>from</code> keywords in the same query expression, you must supply a particular overload of <code>SelectMany</code>. This is an oddity of the implementation of the query syntax language feature in C#. You don't have to do anything similar to that in F# or Haskell. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">U</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">U</span>&gt;&gt;&nbsp;k,&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">U</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;s) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;SelectMany(x&nbsp;=&gt;&nbsp;k(x).SelectMany(y&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;s(x,&nbsp;y)))); }</pre> </p> <p> Once you've implemented such overloads a couple of times, they're more tedious than challenging to write. They always follow the same template. First use <code>SelectMany</code> with <code>k</code>, and then <code>SelectMany</code> again with <code>s</code>. The only marginally stimulating part of the implementation is figuring out how to wrap the return value from <code>s</code>. </p> <p> You're also going to need <code>Select</code> as shown in the article about <a href="/2020/06/22/the-io-functor">IO as a functor</a>. </p> <h3 id="589ab47b68244e21bafd2f16c4422800"> Conclusion <a href="#589ab47b68244e21bafd2f16c4422800" title="permalink">#</a> </h3> <p> C#'s query syntax offers limited syntactic sugar over <a href="/2018/03/22/functors">functors</a> and monads. Compared with F# and Haskell, the syntax is odd and its functionality limited. The most galling lacuna is that you can't branch (e.g. use <code>if</code> or <code>switch</code>) inside query expressions. </p> <p> The point of these articles is (still) not to endorse this style of programming. While the code I show in this article series is working C# code that runs and passes its tests, I'm pretending that all impure actions in C# return <code>IO</code> results. To be clear, the <code>Console</code> class this code interacts with isn't <a href="https://docs.microsoft.com/dotnet/api/system.console">the Console class from the base class library</a>. It's a class that pretends to be such a class from a parallel universe. </p> <p> So far in these articles, you've seen how to compose impure actions with pure functions. What I haven't covered yet is the motivation for it all. We want the compiler to enforce the <a href="/2018/11/19/functional-architecture-a-definition">functional interaction law</a>: a pure function shouldn't be able to invoke an impure action. That's the topic for the next article. </p> <p> <strong>Next:</strong> <a href="/2020/07/06/referential-transparency-of-io">Referential transparency of IO</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The IO functor https://blog.ploeh.dk/2020/06/22/the-io-functor 2020-06-22T06:23:00+00:00 Mark Seemann <div id="post"> <p> <em>The IO container forms a functor. An article for object-oriented programmers.</em> </p> <p> This article is an instalment in <a href="/2018/03/22/functors">an article series about functors</a>. Previous articles have covered <a href="/2018/03/26/the-maybe-functor">Maybe</a>, <a href="/2018/09/10/the-lazy-functor">Lazy</a>, and other functors. This article provides another example. </p> <h3 id="cde6cc0f422941c6a89cc7717f3b62b5"> Functor <a href="#cde6cc0f422941c6a89cc7717f3b62b5" title="permalink">#</a> </h3> <p> In a recent article, I gave an example of <a href="/2020/06/15/io-container-in-a-parallel-c-universe">what <em>IO</em> might look like in C#</a>. The <code>IO&lt;T&gt;</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> already has sufficient API to make it a functor. All it needs is a <code>Select</code> method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;Select&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;SelectMany(x&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;selector(x))); }</pre> </p> <p> This is an instance method on <code>IO&lt;T&gt;</code>, but you can also write it as an extension method, if that's more to your liking. </p> <p> When you call <code>selector(x)</code>, the return value is an object of the type <code>TResult</code>. The <code>SelectMany</code> method, however, wants you to return an object of the type <code>IO&lt;TResult&gt;</code>, so you use the <code>IO</code> constructor to wrap that return value. </p> <h3 id="0f17690c09b34160a7cc843a461d836d"> Haskell <a href="#0f17690c09b34160a7cc843a461d836d" title="permalink">#</a> </h3> <p> The C# <code>IO&lt;T&gt;</code> container is an illustration of how <a href="https://www.haskell.org">Haskell</a>'s <code>IO</code> type works. It should come as no surprise to Haskellers that <code>IO</code> is a functor. In fact, it's a monad, and all monads are also functors. </p> <p> The C# <code>IO&lt;T&gt;</code> API is based around a constructor and the <code>SelectMany</code> method. The constructor wraps a plain <code>T</code> value in <code>IO&lt;T&gt;</code>, so that corresponds to Haskell's <code>return</code> method. The <code>SelectMany</code> method corresponds to Haskell's monadic <em>bind</em> operator <code>&gt;&gt;=</code>. When you have lawful <code>return</code> and <code>&gt;&gt;=</code> implementations, you can have a <code>Monad</code> instance. When you have a <code>Monad</code> instance, you not only <em>can</em> have <code>Functor</code> and <code>Applicative</code> instances, you <em>must</em> have them. </p> <h3 id="2c8de38e7bd04656855c405529c71908"> Conclusion <a href="#2c8de38e7bd04656855c405529c71908" title="permalink">#</a> </h3> <p> IO forms a functor, among other abstractions. In C#, this manifests as a proper implementation of a <code>Select</code> method. </p> <p> <strong>Next:</strong> <a href="/2020/10/19/monomorphic-functors">Monomorphic functors</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="a571ffe129e44ddbb2ecec0bf4d108ac"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> The constructor wraps a plain <code>T</code> value in <code>IO&lt;T&gt;</code> </blockquote> <p> Did you mean to say that the constructor wraps a <code>Lazy&lt;T&gt;</code> value in <code>IO&lt;T&gt;</code>? </p> </div> <div class="comment-date">2020-06-22 14:05 UTC</div> </div> <div class="comment" id="6c7f026e825942429254efc19440b2e2"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. Well, yes, that's technically what happens... I'm deliberately being imprecise with the language because I'm trying to draw a parallel to Haskell. In Haskell, <code>return</code> takes a value and wraps it in <code>IO</code> (the type is effectively <code>a -&gt; IO a</code>). In Haskell, however, computation is lazy by default. This means that the value you wrap in <code>IO</code> is already lazy. This turns out to be important, as I'll explain in a future article, so in C# we have to first make sure that the value is lazy. </p> <p> The concept, however, involves taking a 'bare' value and wrapping it in a container, and that's the reason I chose my words as I did. </p> </div> <div class="comment-date">2020-06-22 14:45 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. IO container in a parallel C# universe https://blog.ploeh.dk/2020/06/15/io-container-in-a-parallel-c-universe 2020-06-15T05:55:00+00:00 Mark Seemann <div id="post"> <p> <em>A C# model of IO at the type level.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. The previous article provided a conceptual overview. In this article you'll see C# code examples. </p> <h3 id="03b3ac7f1b2542b583e7f122f1c8da7d"> In a world... <a href="#03b3ac7f1b2542b583e7f122f1c8da7d" title="permalink">#</a> </h3> <p> Imagine a parallel universe where a C# <a href="https://en.wikipedia.org/wiki/Entry_point">entry point</a> is supposed to look like this: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args)</pre> </p> <p> Like another Neo, you notice that something in your reality is odd, so you decide to follow the white rabbit. <code>Unit</code>? What's that? Navigating to its definition, you see this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Unit</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Unit</span>&nbsp;Instance; }</pre> </p> <p> There's not much to see here. <code>Unit</code> is a type that serves the same role as the <code>void</code> keyword. <a href="/2018/01/15/unit-isomorphisms">They're isomorphic</a>, but <code>Unit</code> has the advantage that it's a proper type. This means that it can be a type parameter in a generically typed container like <code>IO</code>. </p> <p> So what's <code>IO</code>? When you view its definition, this is what you see: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) }</pre> </p> <p> There's a constructor you can initialise with a lazily evaluated value, and a <code>SelectMany</code> method that looks strikingly like something out of LINQ. </p> <p> You'll probably notice right away that while you can put a value into the <code>IO</code> container, you can't get it back. As the introductory article explained, this is by design. Still, you may think: <em>What's the point?</em> Why would I ever want to use this class? </p> <p> You get part of the answer when you try to implement your program's entry point. In order for it to compile, the <code>Main</code> method must return an <code>IO&lt;Unit&gt;</code> object. Thus, the simplest <code>Main</code> method that compiles is this: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;(()&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Unit</span>.Instance); };</pre> </p> <p> That's only a roundabout <a href="https://en.wikipedia.org/wiki/NOP_(code)">no-op</a>. What if you want write <em>real</em> code? Like <a href="https://en.wikipedia.org/wiki/%22Hello,_World!%22_program">hello world</a>? </p> <h3 id="01dbc48c2136426680432d9b290ed281"> Impure actions <a href="#01dbc48c2136426680432d9b290ed281" title="permalink">#</a> </h3> <p> You'd like to write a program that asks the user about his or her name. Based on the answer, and the time of day, it'll write <em>Hello, Nearth,</em> or <em>Good evening, Kate.</em> You'd like to know how to take user input and write to the standard output stream. In this parallel world, the <code>Console</code> API looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Console</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:blue;">string</span>&gt;&nbsp;ReadLine(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;WriteLine(<span style="color:blue;">string</span>&nbsp;value); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;here...</span> }</pre> </p> <p> You notice that both methods return <code>IO</code> values. This immediately tells you that they're impure. This is hardly surprising, since <code>ReadLine</code> is non-deterministic and <code>WriteLine</code> has a side effect. </p> <p> You'll also need the current time of day. How do you get that? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Clock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;GetLocalTime(); }</pre> </p> <p> Again, <code>IO</code> signifies that the returned <code>DateTime</code> value is impure; it's non-deterministic. </p> <h3 id="7c7d0747805541fbb5c31f4257f5ab6e"> Pure logic <a href="#7c7d0747805541fbb5c31f4257f5ab6e" title="permalink">#</a> </h3> <p> A major benefit of functional programming is the natural <a href="https://en.wikipedia.org/wiki/Separation_of_concerns">separation of concerns</a>; separation of business logic from implementation details (a.k.a. the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a>). </p> <p> Write the logic of the program as a pure function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> You can tell that this is a pure function from its return type. In this parallel universe, <em>all</em> impure library methods look like the above <code>Console</code> and <code>Clock</code> methods. Thus, by elimination, a method that <em>doesn't</em> return <code>IO</code> is pure. </p> <h3 id="a16d564dc28149eaa462813173276fbd"> Composition <a href="#a16d564dc28149eaa462813173276fbd" title="permalink">#</a> </h3> <p> You have impure actions you can invoke, and a pure piece of logic. You can use <code>ReadLine</code> to get the user's name, and <code>GetLocalTime</code> to get the local time. When you have those two pieces of data, you can call the <code>Greet</code> method. </p> <p> This is where most people run aground. <em>"Yes, but Greet needs a string, but I have an IO&lt;string&gt;. <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">How do I get the string out of the IO?</a></em> </p> <p> If you've been following the plot so far, you know the answer: <a href="https://en.wikipedia.org/wiki/Mu_(negative)">mu</a>. You don't. You compose all the things with <code>SelectMany</code>: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;What&#39;s&nbsp;your&nbsp;name?&quot;</span>).SelectMany(_&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.ReadLine().SelectMany(name&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime().SelectMany(now&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#2b91af;">Greeter</span>.Greet(now,&nbsp;name); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(greeting); &nbsp;&nbsp;&nbsp;&nbsp;}))); }</pre> </p> <p> I'm not going to walk through all the details of how this works. There's <a href="https://cleancoders.com/videos/humane-code-real">plenty of monad tutorials out there</a>, but take a moment to contemplate the <code>SelectMany</code> method's <code>selector</code> argument. It takes a <em>plain</em> <code>T</code> value as input, but must return an <code>IO&lt;TResult&gt;</code> object. That's what each of the above lambda expressions do, but that means that <code>name</code> and <code>now</code> are <em>unwrapped values;</em> i.e. <code>string</code> and <code>DateTime</code>. </p> <p> That means that you can call <code>Greet</code> with <code>name</code> and <code>now</code>, which is exactly what happens. </p> <p> Notice that the above lambda expressions are nested. With <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> formatting, they'd exhibit the dreaded <a href="http://wiki.c2.com/?ArrowAntiPattern">arrow shape</a>, but with this formatting, it looks more like the sequential composition that it actually is. </p> <h3 id="03f739d929874049be990b3c90c2d7eb"> Conclusion <a href="#03f739d929874049be990b3c90c2d7eb" title="permalink">#</a> </h3> <p> The code you've seen here all works. The only difference between this hypothetical C# and the real C# is that your <code>Main</code> method can't look like that, and impure library methods don't return <code>IO</code> values. </p> <p> The point of the article isn't to recommend this style of programming. You can't, really, since it relies on the counter-factual assumption that all impure library methods return <code>IO</code>. The point of the article is to explain how a language like <a href="https://www.haskell.org">Haskell</a> uses the type system to distinguish between pure functions and impure actions. </p> <p> Perhaps the code in that <code>Main</code> method isn't the prettiest code you've ever seen, but we can make it a little nicer. That's the topic of the next article in the series. </p> <p> <strong>Next:</strong> <a href="/2020/06/29/syntactic-sugar-for-io">Syntactic sugar for IO</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="c0bb2d1ce63d41adba638e489f926ee1"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Why is the input of the IO constructor of type <code>Lazy&lt;T&gt;</code> instead of just type <code>T</code>? </p> </div> <div class="comment-date">2020-06-22 12:16 UTC</div> </div> <div class="comment" id="8ff4e7987f3e46f4bffe328ec2706679"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. A future article in this article series will answer that question 😀 </p> </div> <div class="comment-date">2020-06-22 12:25 UTC</div> </div> <div class="comment" id="43ca1bf8d83a4124a4a9ccd33343e864"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> FWIW, the promised article is <a href="/2020/07/06/referential-transparency-of-io">now available</a>. </p> </div> <div class="comment-date">2020-07-06 6:01 UTC</div> </div> <div class="comment" id="5870eefe47424f4f925660991e257fac"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Ah, sure. I was thinking that <code>IO&lt;T&gt;</code> is a monad (in <code>T</code>), so there should be a constructor with argument <code>T</code>. However, the function doens't need to be a constructor. The lambda expression <code>t =&gt; new IO&lt;T&gt;(new Lazy&lt;T&gt;(() =&gt; t))</code> satisifies the requirement. </p> </div> <div class="comment-date">2020-07-13 19:35 UTC</div> </div> <div class="comment" id="3178b75249e749a59fd6e08ee7738daa"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Yes 👍 </p> </div> <div class="comment-date">2020-07-14 6:19 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The IO Container https://blog.ploeh.dk/2020/06/08/the-io-container 2020-06-08T05:53:00+00:00 Mark Seemann <div id="post"> <p> <em>How a type system can distinguish between pure and impure code.</em> </p> <p> <a href="https://en.wikipedia.org/wiki/Referential_transparency">Referential transparency</a> is the foundation of <a href="/2018/11/19/functional-architecture-a-definition">functional architecture</a>. If you categorise all operations into <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions, then most other traits associated with functional programming follow. </p> <p> Unfortunately, <a href="/2020/02/24/discerning-and-maintaining-purity">mainstream programming languages don't distinguish between pure functions and impure actions</a>. Identifying pure functions is tricky, and the knowledge is fleeting. What was a pure functions today may become impure next time someone changes the code. </p> <p> Separating pure and impure code is important. It'd be nice if you could automate the process. Perhaps you could run some tests, or, <a href="/2011/04/29/Feedbackmechanismsandtradeoffs">even better</a>, make the compiler do the work. That's what <a href="https://www.haskell.org">Haskell</a> and a few other languages do. </p> <p> In Haskell, the distinction is made with a <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> called <code>IO</code>. This static type enforces the functional interaction law at compile time: pure functions can't invoke impure actions. </p> <h3 id="5bb03bce8e314e7696aefa1800f94637"> Opaque container <a href="#5bb03bce8e314e7696aefa1800f94637" title="permalink">#</a> </h3> <p> Regular readers of this blog know that I often use Haskell to demonstrate principles of functional programming. If you don't know Haskell, however, its ability to guarantee the functional interaction law at compile time may seem magical. It's not. </p> <p> Fortunately, the design is so simple that it's easy to explain the fundamental concept: Results of impure actions are always enclosed in an opaque container called <em>IO</em>. You can think of it as a box with a label. </p> <p> <img src="/content/binary/io-box-with-int-inside.png" alt="A cardboard box with the label 'int inside' on the side."> </p> <p> The label only tells you about the static type of the value inside the box. It could be an <code>int</code>, a <code>DateTime</code>, or your own custom type, say <code>Reservation</code>. While you know what type of value is inside the box, you can't see what's in it, and you can't open it. </p> <h3 id="803d5847d88b4b51bebcecddb2581f3a"> Name <a href="#803d5847d88b4b51bebcecddb2581f3a" title="permalink">#</a> </h3> <p> The container itself is called <code>IO</code>, but don't take the word too literally. While all <a href="https://en.wikipedia.org/wiki/Input/output">I/O (input/output)</a> is inherently impure, other operations that you don't typically think of as I/O is impure as well. Generation of random numbers (including GUIDs) is the most prominent example. Random number generators rely on the system clock, which you <em>can</em> think of as an input device, although I think many programmers don't. </p> <p> I could have called the container <em>Impure</em> instead, but I chose to go with <em>IO</em>, since this is the word used in Haskell. It also has the advantage of being short. </p> <h3 id="8964b332bd1a423ba2f9729d679b6225"> What's in the boooox? <a href="#8964b332bd1a423ba2f9729d679b6225" title="permalink">#</a> </h3> <p> A question frequently comes up: <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">How do I get the value out of my IO?</a> As always, the answer is <a href="https://en.wikipedia.org/wiki/Mu_(negative)">mu</a>. You don't. You inject the desired behaviour into the container. This goes for all monads, including IO. </p> <p> But naturally you wonder: If you can't see the value inside the IO box then <em>what's the point?</em> </p> <p> The point is to enforce the functional interaction law at the type level. A pure function that calls an impure action will receive a sealed, opaque IO box. There's no API that enables a pure function to extract the contents of the container, so this effectively enforces the rule that pure functions can't call impure actions. </p> <p> The other three types of interactions are still possible. <ul> <li>Pure functions should be able to call pure functions. Pure functions return 'normal' values (i.e. values not hidden in IO boxes), so they can call each other as usual.</li> <li>Impure actions should be able to call pure functions. This becomes possible because you can inject pure behaviour into any monad. You'll see example of that in later articles in this series.</li> <li>Impure actions should be able to call other impure actions. Likewise, you can compose many IO actions into one IO action via the IO API.</li> </ul> When you're outside of all IO boxes, there's no way to open the box, and you can't see its contents. </p> <p> On the other hand, <em>if you're already inside the box</em>, you can see the contents. And there's one additional rule: If you're already inside an IO box, you <em>can</em> open other IO boxes and see their contents! </p> <p> In subsequent articles in this article series, you'll see how all of this manifests as C# code. This article gives a high-level view of the concept. I suggest that you go back and re-read it once you've seen the code. </p> <h3 id="2f29062933db44278c87649c31704c9a"> The many-worlds interpretation <a href="#2f29062933db44278c87649c31704c9a" title="permalink">#</a> </h3> <p> If you're looking for metaphors or other ways to understand what's going on, there's two perspectives I find useful. None of them offer the full picture, but together, I find that they help. </p> <p> A common interpretation of IO is that it's like the box in which you put <a href="https://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s_cat">Schrödinger's cat</a>. <code>IO&lt;Cat&gt;</code> can be viewed as the superposition of the two states of cat (assuming that <code>Cat</code> is basically a <a href="https://en.wikipedia.org/wiki/Tagged_union">sum type</a> with the cases <code>Alive</code> and <code>Dead</code>). Likewise, <code>IO&lt;int&gt;</code> represents the superposition of all 4,294,967,296 32-bit integers, <code>IO&lt;string&gt;</code> the superposition of infinitely many strings, etcetera. </p> <p> Only when you observe the contents of the box does the superposition collapse to a single value. </p> <p> But... you can't observe the contents of an IO box, can you? </p> <h3 id="0f85d7e2346d4805ae05045965f9231d"> The black hole interpretation <a href="#0f85d7e2346d4805ae05045965f9231d" title="permalink">#</a> </h3> <p> The IO container represents an impenetrable barrier between the outside and the inside. It's like a <a href="https://en.wikipedia.org/wiki/Black_hole">black hole</a>. Matter can fall into a black hole, but no information can escape its <a href="https://en.wikipedia.org/wiki/Event_horizon">event horizon</a>. </p> <p> In high school I took cosmology, among many other things. I don't know if the following is still current, but we learned a formula for calculating the density of a black hole, based on its mass. When you input the estimated mass of the universe, the formula suggests a density near vacuum. Wait, what?! Are we actually living inside a black hole? Perhaps. Could there be other universes 'inside' black holes? </p> <p> The analogy to the IO container seems apt. You can't see into a black hole from the outside, but once <a href="https://amzn.to/2BlYK6D">beyond the blue event horizon</a>, you can observe everything that goes on in that interior universe. You can't escape to the original universe, though. </p> <p> As with all metaphors, this one breaks down if you think too much about it. Code running in IO can unpack other IO boxes, even nested boxes. There's no reason to believe that if you're inside a black hole that you can then gaze beyond the event horizon of nested black holes. </p> <h3 id="524b6fbfa7c54eb2b26ec8212b7db64e"> Code examples <a href="#524b6fbfa7c54eb2b26ec8212b7db64e" title="permalink">#</a> </h3> <p> In the next articles in this series, you'll see C# code examples that illustrate how this concept might be implemented. The purpose of these code examples is to give you a sense for how <code>IO</code> works in Haskell, but with more familiar syntax. <ul> <li><a href="/2020/06/15/io-container-in-a-parallel-c-universe">IO container in a parallel C# universe</a></li> <li><a href="/2020/06/29/syntactic-sugar-for-io">Syntactic sugar for IO</a></li> <li><a href="/2020/07/06/referential-transparency-of-io">Referential transparency of IO</a></li> <li><a href="/2020/07/13/implementation-of-the-c-io-container">Implementation of the C# IO container</a></li> <li><a href="/2020/07/27/task-asynchronous-programming-as-an-io-surrogate">Task asynchronous programming as an IO surrogate</a></li> </ul> These code examples are <em>illustrations</em>, not recommendations. </p> <h3 id="729688b622f64684b3f5ad2816eadaca"> Conclusion <a href="#729688b622f64684b3f5ad2816eadaca" title="permalink">#</a> </h3> <p> When you saw the title, did you think that this would be an article about IoC Containers? It's not. The title isn't a typo, and I never use the term <em>IoC Container</em>. As <a href="https://blogs.cuttingedge.it/steven/">Steven</a> and I explain in <a href="/dippp">our book</a>, <em>Inversion of Control</em> (IoC) is a broader concept than Dependency Injection (DI). It's called a <em>DI Container</em>. </p> <p> IO, on the other hand, is a container of impure values. Its API enables you to 'build' bigger structures (programs) from smaller IO boxes. You can compose IO actions together and inject pure functions into them. The boxes, however, are opaque. Pure functions can't see their contents. This effectively enforces the functional interaction law at the type level. </p> <p> <strong>Next:</strong> <a href="/2020/06/15/io-container-in-a-parallel-c-universe">IO container in a parallel C# universe</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header">Comments</h2> <div class="comment" id="33c7fbbfa5ba409a885296b880592e80"> <div class="comment-author"><a href="https://www.relativisticramblings.com/">Christer van der Meeren</a></div> <div class="comment-content"> <p> Come on, you know there is a perfect metaphor. <a href="https://blog.plover.com/prog/burritos.html">Monads are like burritos.</a> </p> </div> <div class="comment-date">2020-06-08 11:08 UTC</div> </div> <div class="comment" id="d5b99040cf664be5a7b3ae5bfc2b6394"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Christer, I appreciate that this is all in good fun 🤓 </p> <p> For the benefit of readers who may still be trying to learn these concepts, I'll point out that just as this isn't an article about <em>IoC containers</em>, it's not a monad tutorial. It's an explanation of a particular API called <em>IO</em>, which, among other traits, also forms a monad. I'm trying to downplay that aspect here, because I hope that you can understand most of this and the next articles without knowing what a monad is. </p> </div> <div class="comment-date">2020-06-08 11:27 UTC</div> </div> <div class="comment" id="6d8a99b9728b452193311fadec795cf4"> <div class="comment-author"><a href="https://github.com/DunetsNM/">Nick Dunets</a></div> <div class="comment-content"> <p> "While you know what type of value is inside the box, you can't see what's in it, and you can't open it." </p> <p> Well you technically can, with unsafePerformIO ;) although it defeats the whole purpose. </p> </div> <div class="comment-date">2020-06-10 02:31 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Retiring old service versions https://blog.ploeh.dk/2020/06/01/retiring-old-service-versions 2020-06-01T09:36:00+00:00 Mark Seemann <div id="post"> <p> <em>A few ideas on how to retire old versions of a web service.</em> </p> <p> I was recently listening to <a href="https://dotnetrocks.com/?show=1688">a .NET Rocks! episode on web APIs</a>, and one of the topics that came up was how to retire old versions of a web service. It's not easy to do, but not necessarily impossible. </p> <p> The best approach to API versioning is to never break compatibility. As long as there's no breaking changes, you don't have to version your API. It's rarely possible to completely avoid breaking changes, but the fewer of those that you introduce, the fewer API version you have to maintain. I've <a href="/2015/06/22/rest-implies-content-negotiation">previously described how to version REST APIs</a>, but this article doesn't assume any particular versioning strategy. </p> <p> Sooner or later you'll have an old version that you'd like to retire. How do you do that? </p> <h3 id="66f738fcae73491988edafb0eb766001"> Incentives <a href="#66f738fcae73491988edafb0eb766001" title="permalink">#</a> </h3> <p> First, take a minute to understand why some clients keep using old versions of your API. It may be obvious, but I meet enough programmers who never give incentives a thought that I find it worthwhile to point out. </p> <p> When you introduce a breaking change, by definition this is a change that <em>breaks</em> clients. Thus, upgrading from an old version to a newer version of your API is likely to give client developers extra work. If what they have already works to their satisfaction, why should they upgrade their clients? </p> <p> You might argue that your new version is 'better' because it has more features, or is faster, or whatever it is that makes it better. Still, if the old version is good enough for a particular purpose, some clients are going to stay there. The client maintainers have no incentives to upgrade. There's only cost, and no benefit, to upgrading. </p> <p> Even if the developers who maintain those clients would like to upgrade, they may be prohibited from doing so by their managers. If there's no business reason to upgrade, efforts are better used somewhere else. </p> <h3 id="ccca66feb014455288e12ee519c56ada"> Advance warning <a href="#ccca66feb014455288e12ee519c56ada" title="permalink">#</a> </h3> <p> Web services exist in various contexts. Some are only available on an internal network, while others are publicly available on the internet. Some require an authentication token or API key, while others allow anonymous client access. </p> <p> With some services, you have a way to contact every single client developer. With other services, you don't know who uses your service. </p> <p> Regardless of the situation, when you wish to retire a version, you should first try to give clients advance warning. If you have an address list of all client developers, you can simply write them all, but you can't expect that everyone reads their mail. If you don't know the clients, you can publish the warning. If you have a blog or a marketing site, you can publish the warning there. If you run a mailing list, you can write about the upcoming change there. You can tweet it, post it on Facebook, or dance it on TikTok. </p> <p> Depending on SLAs and contracts, there may even be a legally valid communications channel that you can use. </p> <p> Give advance warning. That's the decent thing to do. </p> <h3 id="bfa13823398b42a8aa1f1ef3de2c6b29"> Slow it down <a href="#bfa13823398b42a8aa1f1ef3de2c6b29" title="permalink">#</a> </h3> <p> Even with advance warning, not everyone gets the message. Or, even if everyone got the message, some people deliberately decide to ignore it. Consider their incentives. They may gamble that as long as your logs show that they use the old version, you'll keep it online. What do you do then? </p> <p> You can, of course, just take the version off line. That's going to break clients that still depend on that version. That's rarely the best course of action. </p> <p> Another option is to degrade the performance of that version. Make it slower. You can simply add a constant delay when clients access that service, or you can gradually degrade performance. </p> <p> Many HTTP client libraries have long timeouts. For example, the <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpclient.timeout">default HttpClient timeout is 100 seconds</a>. Imagine that you want to introduce a gradual performance degradation that starts at no delay on June 1, 2020 and reaches 100 seconds after one year. You can use the formula <code>d = 100 s * (t - t0) / 1 year</code>, where <code>d</code> is the delay, <code>t</code> is the current time, and <code>t0</code> is the start time (e.g. June 1, 2020). This'll cause requests for resources to gradually slow down. After a year, clients still talking to the retiring version will begin to time out. </p> <p> You can think of this as another way to give advance warning. With the gradual deterioration of performance, users will likely notice the long wait times well before calls actually time out. </p> <p> When client developers contact you about the bad performance, you can tell them that the issue is fixed in more recent versions. You've just given the client organisation an incentive to upgrade. </p> <h3 id="741bec1a72bf46cc9e26f6e7600a6fab"> Failure injection <a href="#741bec1a72bf46cc9e26f6e7600a6fab" title="permalink">#</a> </h3> <p> Another option is to deliberately make the service err now and then. Randomly return a <code>500 Internal Server Error</code> from time to time, even if the service <em>can</em> handle the request. </p> <p> Like deliberate performance degradation, you can gradually make the deprecated version more and more unreliable. Again, end users are likely to start complaining about the unreliability of the system long before it becomes entirely useless. </p> <h3 id="bc73620aa71141b6a74f4a4aaf395d75"> Reverse proxy <a href="#bc73620aa71141b6a74f4a4aaf395d75" title="permalink">#</a> </h3> <p> One of the many benefits of HTTP-based services is that you can put a <a href="https://en.wikipedia.org/wiki/Reverse_proxy">reverse proxy</a> in front of your application servers. I've no idea how to configure or operate <a href="https://en.wikipedia.org/wiki/Nginx">NGINX</a> or <a href="https://en.wikipedia.org/wiki/Varnish_(software)">Varnish</a>, but from talking to people who do know, I get the impression that they're quite scriptable. </p> <p> Since the above ideas are independent of actual service implementation or behaviour, it's a generic problem that you should seek to address with general-purpose software. </p> <p> <img src="/content/binary/reverse-proxy-based-delay.png" alt="Sequence diagram of a client, reverse proxy, and application server."> </p> <p> Imagine having a general-purpose reverse proxy that detects whether incoming HTTP requests are for the version you'd like to retire (<em>version 1</em> in the diagram) or another version. If the proxy detects that the request is for a retiring version, it inserts a delay before it forward the request to the application server. For all requests for current versions, it just forwards the request. </p> <p> I could imagine doing something similar with failure injections. </p> <h3 id="a0426537b459414b831db955de4cca26"> Legal ramifications <a href="#a0426537b459414b831db955de4cca26" title="permalink">#</a> </h3> <p> All of the above are only ideas. If you can use them, great. Consider the implications, though. You may be legally obliged to keep an SLA. In that case, you can't degrade the performance or reliability below the SLA level. </p> <p> In any case, I don't think you should do any of these things lightly. Involve relevant stakeholders before you engage in something like the above. </p> <p> Legal specialists are as careful and conservative as traditional operations teams. Their default reaction to any change proposal is to say <em>no</em>. That's not a judgement on their character or morals, but simply another observation based on incentives. As long as everything works as it's supposed to work, any change represents a risk. Legal specialists, like operations teams, are typically embedded in incentive systems that punish risk-taking. </p> <p> To counter other stakeholders' reluctance to engage in unorthodox behaviour, you'll have to explain why retiring an old version of the service is important. It works best if you can quantify the reason. If you can, measure how much extra time you waste on maintaining the old version. If the old version runs on separate hardware, or a separate cloud service, quantify the cost overhead. </p> <p> If you can't produce a compelling argument to retire an old version, then perhaps it isn't that important after all. </p> <h3 id="725c16c05869474a8cf58da8f0705b7e"> Logs <a href="#725c16c05869474a8cf58da8f0705b7e" title="permalink">#</a> </h3> <p> Server logs can be useful. They can tell you how many requests the old version serves, which IP addresses they come from, at which times or dates you have most traffic, and whether the usage trend is increasing or decreasing. </p> <p> These measures can be used to support your argument that a particular version should be retired. </p> <h3 id="226e48b1d4ef4a14bbf778a9fd6983f7"> Conclusion <a href="#226e48b1d4ef4a14bbf778a9fd6983f7" title="permalink">#</a> </h3> <p> Versioning web services is already a complex subject, but once you've figured it out, you'll sooner or later want to retire an old version of your API. If some clients still make good use of that version, you'll have to give them incentive to upgrade to a newer version. </p> <p> It's best if you can proactively make clients migrate, so prioritise amiable solutions. Sometimes, however, you have no way to reach all client developers, or no obvious way to motivate them to upgrade. In those cases, gentle and gradual performance or reliability degradation of deprecated versions could be a way. </p> <p> I present these ideas as nothing more than that: ideas. Use them if they make sense in your context, but think things through. The responsibility is yours. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="c7751c3a6d5b4facbf7a552eeaa62792"> <div class="comment-author"><a href="http://blog.strobaek.org">Karsten Strøbæk</a></div> <div class="comment-content"> <p> Hi Mark. As always an excellent piece. A few comments if I may. </p> <p> An assumption seems to be, that the client is able to update to a new version of the API, but is not inclined to do so for various reasons. I work with organisations where updating a client if nearly impossible. Not because of lack of will, but due to other factors such as government regulations, physical location of client, hardware version or capabilities of said client to name just a few. </p> <p> We have a tendency in the software industry to see updates as a question of 'running Windows update' and then all is good. Most likely because that is the world we live in. If we wish to update a program or even an OS, it is fairly easy even your above considerations taken into account. </p> <p> In the 'physical' world of manufacturing (or pharmacy or mining or ...) the situation is different. The lifetime of the 'thing' running the client is regularly measured in years or even decades and not weeks or months as it is for a piece of software. </p> <p> Updates are often equal to bloating of resource requirements meaning you have to replace the hardware. This might not always be possible again for various reasons. Cost (company is pressed on margin or client is located in Outer Mongolia) or risk (client is located in Syria or some other hotspot) are some I've personally come across. </p> <p> REST/HTTP is not used. I acknowledge that the original podcast from .NET Rocks! was about updating a web service. This does not change the premises of your arguments, but it potentially adds a layer of complication. </p> </div> <div class="comment-date">2020-06-02 14:47 UTC</div> </div> <div class="comment" id="3e0454135e9b49f1b27bec577318484a"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Karsten, thank you for writing. You are correct that the underlying assumption is that you can retire old versions. </p> <p> I, too, have written REST APIs where retiring service versions weren't an option. These were APIs that consumer-grade hardware would communicate with. We had no way to assure that consumers would upgrade their hardware. Those boxes wouldn't have much of a user-interface. Owners might not realise that firmware updates were available, even if they were. </p> <p> This article does, indeed, assume that the reader has made an informed decision that it's fundamentally acceptable to retire a service version. I should have made that more explicit. </p> </div> <div class="comment-date">2020-06-04 5:32 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Where's the science? https://blog.ploeh.dk/2020/05/25/wheres-the-science 2020-05-25T05:50:00+00:00 Mark Seemann <div id="post"> <p> <em>Is a scientific discussion about software development possible?</em> </p> <p> Have you ever found yourself in a heated discussion about a software development topic? Which is best? Tabs or spaces? Where do you put the curly brackets? <a href="/2020/05/04/significant-whitespace-is-dry">Is significant whitespace a good idea?</a> Is Python better than Go? Does test-driven development yield an advantage? <a href="/2019/07/01/yes-silver-bullet">Is there a silver bullet?</a> Can you <a href="https://martinfowler.com/bliki/CannotMeasureProductivity.html">measure software development productivity?</a> </p> <p> I've had plenty of such discussions, and I'll have them again in the future. </p> <p> While some of these discussions may resemble debates on <a href="https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F">how many angels can dance on the head of a pin</a>, other discussions might be important. Ex ante, it's hard to tell which is which. </p> <p> Why don't we settle these discussions with science? </p> <h3 id="dc849de642834dd998e0fdce57b91550"> A notion of science <a href="#dc849de642834dd998e0fdce57b91550" title="permalink">#</a> </h3> <p> I love science. Why don't I apply scientific knowledge instead of arguments based on <a href="https://martinfowler.com/bliki/AnecdotalEvidence.html">anecdotal evidence</a>? </p> <p> To answer such questions, we must first agree on a definition of science. I favour <a href="https://en.wikipedia.org/wiki/Karl_Popper">Karl Popper</a>'s description of <a href="https://en.wikipedia.org/wiki/Falsifiability">empirical falsifiability</a>. A hypothesis that makes successful falsifiable predictions of the future is a good scientific theory. Such a a theory has <a href="https://en.wikipedia.org/wiki/Predictive_power">predictive power</a>. </p> <p> Newton's theory of gravity had ample predictive power, but Einstein's theory of general relativity supplanted it because its predictive power was even better. </p> <p> <a href="https://en.wikipedia.org/wiki/Gregor_Mendel">Mendel</a>'s theory of inheritance had predictive power, but was refined into what is modern-day genetics which yields much greater predictive power. </p> <p> Is predictive power the only distinguishing trait of good science? I'm already venturing into controversial territory by taking this position. I've met people in the programming community who consider my position naive or reductionist. </p> <p> What about <em>explanatory power?</em> If a theory satisfactorily explains observed phenomena, doesn't that count as a proper scientific theory? </p> <h3 id="080ec1ebb66c498f8d21bfce5939e480"> Controversy <a href="#080ec1ebb66c498f8d21bfce5939e480" title="permalink">#</a> </h3> <p> I don't believe in explanatory power as a sufficient criterion for science. Without predictive power, we have little evidence that an explanation is correct. An explanatory theory can even be internally consistent, and yet we may not know if it describes reality. </p> <p> Theories with explanatory power are the realm of politics or religion. Consider the observation that some people are rich and some are poor. You can believe in a theory that explains this by claiming structural societal oppression. You can believe in another theory that views poor people as fundamentally lazy. Both are (somewhat internally consistent) political theories, but they have yet to demonstrate much predictive power. </p> <p> Likewise, you may believe that some deity created the universe, but that belief produces no predictions. You can apply <a href="https://en.wikipedia.org/wiki/Occam%27s_razor">Occam's razor</a> and explain the same phenomena without a god. A belief in one or more gods is a religious theory, not a scientific theory. </p> <p> It seems to me that there's a correlation between explanatory power and controversy. Over time, theories with predictive power become uncontroversial. Even if they start out controversial (such as Einstein's theory of general relativity), the dust soon settles because it's hard to argue with results. </p> <p> Theories with mere explanatory power, on the other hand, can fuel controversy forever. Explanations can be compelling, and without evidence to refute them, the theories linger. </p> <p> Ironically, you might argue that Popper's theory of scientific discovery itself is controversial. It's a great explanation, but does it have predictive power? Not much, I admit, but I'm also not aware of competing views on science with better predictive power. Thus, you're free to disagree with everything in this article. I admit that it's a piece of philosophy, not of science. </p> <h3 id="1f26035f0b3c40d4b0681a6b2f45176c"> The practicality of experimental verification <a href="#1f26035f0b3c40d4b0681a6b2f45176c" title="permalink">#</a> </h3> <p> We typically see our field of software development as one of the pillars of <a href="https://en.wikipedia.org/wiki/Science,_technology,_engineering,_and_mathematics">STEM</a>. Many of us have STEM educations (I don't; I'm an economist). Yet, we're struggling to come to grips with the lack of scientific methodology in our field. It seems to me that we suffer from <a href="https://en.wikipedia.org/wiki/Physics_envy">physics envy</a>. </p> <p> It's really hard to compete with physics when it comes to predictive power, but even modern physics struggle with experimental verification. Consider an establishment like <a href="https://home.cern">CERN</a>. It takes billions of euros of investment to make today's physics experiments possible. The only reason we make such investments, I think, is that physics so far has had a good track record. </p> <p> What about another fairly 'hard' science like medicine? In order to produce proper falsifiable predictions, medical science have evolved the process of the <a href="https://en.wikipedia.org/wiki/Randomized_controlled_trial">randomised controlled trial</a>. It works well when you're studying short-term effects. Does this medicine cure this disease? Does this surgical procedure improve a patient's condition? How does lack of REM sleep for three days affect your ability to remember strings of numbers? </p> <p> When a doctor tells you that a particular medicine helps, or that surgery might be warranted, he or she is typically on solid scientific grounds. </p> <p> Here's where things start to become less clear, though, What if <a href="https://en.wikipedia.org/wiki/Atkins_diet">a doctor tells you that a particular diet</a> will improve your expected life span? Is he or she on scientific solid ground? </p> <p> That's rarely the case, because you can't make randomised controlled trials about life styles. Or, rather, a totalitarian society might be able to do that, but we'd consider it unethical. Consider what it would involve: You'd have to randomly select a significant number of <em>babies</em> and group them into those who must follow a particular life style, and those who must not. Then you'll somehow have to force those people to stick to their randomly assigned life style for the entirety of their lives. This is not only unethical, but the experiment also takes the most of a century to perform. </p> <p> What life-style scientists instead do is resort to demographic studies, with all the problems of statistics they involve. Again, the question is whether scientific theories in this field offer predictive power. Perhaps they do, but it takes decades to evaluate the results. </p> <p> My point is that medicine isn't exclusively a hard science. Some medicine is, and some is closer to social sciences. </p> <p> I'm an economist by education. Economics is generally not considered a hard science, although it's a field where it's trivial to make falsifiable predictions. Almost all of economics is about numbers, so making a falsifiable prediction is trivial: <em>The MSFT stock will be at 200 by January 1 2021. The unemployment rate in Denmark will be below 5% in third quarter of 2020.</em> The problem with economics is that most of such predictions turn out to be no better than the toss of a coin - even when made by economists. You can make falsifiable predictions in economics, but most of them do, in fact, get falsified. </p> <p> On the other hand, with the advances in such disparate fields as DNA forensics, satellite surveys, and computer-aided image processing, a venerable 'art' like archaeology is gaining predictive power. <em>We predict that if we dig here, we'll find artefacts from the iron age. We predict that if we make a DNA test of these skeletal remains, they'll show that the person buried was a middle-aged women.</em> And so on. </p> <p> One thing is the ability to produce falsifiable predictions. Another things is whether or not the associated experiment is practically possible. </p> <h3 id="1dda923d8e8f4eecbf6cdd79cdc0e27c"> The science of software development <a href="#1dda923d8e8f4eecbf6cdd79cdc0e27c" title="permalink">#</a> </h3> <p> Do we have a science of software development? I don't think that we have. </p> <p> There's <a href="https://en.wikipedia.org/wiki/Computer_science">computer science</a>, but that's not quite the same. That field of study has produced many predictions that hold. <em>In general, <a href="https://en.wikipedia.org/wiki/Quicksort">quicksort</a> will be faster than <a href="https://en.wikipedia.org/wiki/Bubble_sort">bubble sort</a>. There's an algorithm for finding the shortest way through a network.</em> That sort of thing. </p> <p> You will notice that these result are hardly controversial. It's not those topics that we endlessly debate. </p> <p> We debate whether certain ways to organise work is more 'productive'. The entire productivity debate revolves around an often implicit context: that what we discuss is long-term productivity. We don't much argue how to throw together something during a weekend hackaton. We argue whether we can complete a nine-month software project safer with test-driven development. We argue whether a code base can better sustain its organisation year after year if it's written in <a href="https://fsharp.org">F#</a> or JavaScript. </p> <p> There's <a href="https://www.goodreads.com/review/show/1956107880">little scientific evidence on those questions</a>. </p> <p> The main problem, as I see it, is that it's impractical to perform experiments. Coming up with falsifiable predictions is easy. </p> <p> Let's consider an example. Imagine that your hypothesis is that test-driven development makes you more productive in the middle and long run. You'll have to turn that into a falsifiable claim, so first, pick a software development project of sufficient complexity. Imagine that you can find a project that someone estimates will take around ten months to complete for a team of five people. This has to be a real project that someone needs done, complete with vague, contradictory, and changing requirements. Now you formulate your falsifiable prediction, for example: <em>"This project will be delivered one month earlier with test-driven development."</em> </p> <p> Next, you form teams to undertake the work. One team to perform the work <em>with</em> test-driven development, and one team to do the work without it. Then you measure when they're done. </p> <p> This is already impractical, because who's going to pay for two teams when one would suffice? </p> <p> Perhaps, if you're an exceptional proposal writer, you could get a research grant for that, but alas, that wouldn't be good enough. </p> <p> With two competing teams of five people each, it might happen that <a href="/2019/09/30/10x-developers">one team member exhibits productivity orders of magnitudes different from the others</a>. That could skew the experimental outcome, so you'd have to go for a proper randomised controlled trial. This would involve picking numerous teams and assigning a methodology at random: either they do test-driven development, or they don't. Nothing else should vary. They should all use the same programming language, the same libraries, the same development tools, and work the same hours. Furthermore, no-one outside the team should know which teams follow which method. </p> <p> Theoretically possible, but impractical. It would require finding and paying many software teams for most of a year. One such experiment would cost millions of euros. </p> <p> If you did such an experiment, it would tell you something, but it'd still be open to interpretation. You might argue that the programming language used caused the outcome, but that one can't extrapolate from that result to other languages. Or perhaps there was something special about the project that you think doesn't generalise. Or perhaps you take issue with the pool from which the team members were drawn. You'd have to repeat the experiment while varying one of the other dimensions. That'll cost millions more, and take another year. </p> <p> Considering the billions of euros/dollars/pounds the world's governments pour into research, you'd think that we could divert a few hundred millions to do proper research in software development, but it's unlikely to happen. That's the reason we have to contend ourselves with arguing from anecdotal evidence. </p> <h3 id="6af72b5a24aa4caba250da2373a975d9"> Conclusion <a href="#6af72b5a24aa4caba250da2373a975d9" title="permalink">#</a> </h3> <p> I can imagine how scientific inquiry into software engineering could work. It'd involve making a falsifiable prediction, and then set up an experiment to prove it wrong. Unfortunately, to be on a scientifically sound footing, experiments should be performed with randomised controlled trials, with a statistically significant number of participants. It's not too hard to conceive of such experiments, but they'd be prohibitively expensive. </p> <p> In the meantime, the software development industry moves forward. We share ideas and copy each other. Some of us are successful, and some of us fail. Slowly, this might lead to improvements. </p> <p> That process, however, looks more like evolution than scientific progress. The fittest software development organisations survive. They need not be the best, as they could be stuck in local maxima. </p> <p> When we argue, when we write blog posts, when we speak at conferences, when we appear on podcasts, we exchange ideas and experiences. Not genes, but <a href="https://en.wikipedia.org/wiki/Meme">memes</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="5eec3680e62d40ca8d6e6ab7cc3cf34f"> <div class="comment-author">Sergey Petrov</div> <div class="comment-content"> <p> That topic is something many software developers think about, at least I do from time to time. </p> <p> Your post reminded me of that conference talk <a href="https://www.youtube.com/watch?v=WELBnE33dpY">Intro to Empirical Software Engineering: What We Know We Don't Know</a> by Hillel Wayne. Just curious - have you seen the talk and if so - what do you think? Researches mentioned in the talk are not proper scientific expreiments as you describe, but anyway looked really interesting to me. </p> </div> <div class="comment-date">2020-05-28 12:40 UTC</div> </div> <div class="comment" id="c223fed5e0294187a5faaca6c885d945"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Sergey, thank you for writing. I didn't know about that talk, but Hillel Wayne regularly makes an appearance in my Twitter feed. I've now seen the talk, and I think it offers a perspective close to mine. </p> <p> I've already read <em>The Leprechauns of Software Engineering</em> (above, I linked to my review), but while I was aware of <a href="https://amzn.to/2XQPHSV">Making Software</a>, I've yet to read it. Several people have reacted to my article by recommending that book, so it's now on it's way to me in the mail. </p> </div> <div class="comment-date">2020-06-02 10:26 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Modelling versus shaping reality https://blog.ploeh.dk/2020/05/18/modelling-versus-shaping-reality 2020-05-18T07:08:00+00:00 Mark Seemann <div id="post"> <p> <em>How does software development relate to reality?</em> </p> <p> I recently <a href="https://dotnetrocks.com/?show=1685">appeared as a guest on the <em>.NET Rocks!</em> podcast</a> where we discussed <a href="https://en.wikipedia.org/wiki/Fred_Brooks">Fred Brooks</a>' 1986 essay <a href="https://en.wikipedia.org/wiki/No_Silver_Bullet">No Silver Bullet</a>. As a reaction, <a href="https://jonsmusings.com">Jon Suda</a> wrote <a href="https://jonsmusings.com/Full-Silver-Jacket">a thoughtful piece of his own</a>. That made me think some more. </p> <h3 id="80fd2177df7e4569a292b795ec32a1a3"> Beware of Greeks... <a href="#80fd2177df7e4569a292b795ec32a1a3" title="permalink">#</a> </h3> <p> Brooks' central premise is <a href="https://en.wikipedia.org/wiki/Aristotle">Aristotle</a>'s distinction between <em>essential</em> and <em>accidental</em> complexity. I've already examined the argument in <a href="/2019/07/01/yes-silver-bullet">my previous article on the topic</a>, but in summary, Brooks posits that complexity in software development can be separated into those two categories: </p> <p> <pre>c = E + a</pre> </p> <p> Here, <code>c</code> is the total complexity, <code>E</code> is the essential complexity, and <code>a</code> the accidental complexity. I've deliberately represented <code>c</code> and <code>a</code> with lower-case letters to suggest that they represent variables, whereas I mean to suggest with the upper-case <code>E</code> that the essential complexity is constant. That's Brooks' argument: Every problem has an essential complexity that can't be changed. Thus, your only chance to reduce total complexity <code>c</code> is to reduce the accidental complexity <code>a</code>. </p> <p> Jon Suda writes that <blockquote> <p> "Mark doesn’t disagree with the classification of problems" </p> <footer><cite><a href="https://jonsmusings.com/Full-Silver-Jacket">Jon Suda</a></cite></footer> </blockquote> That got me thinking, because actually I do. When I wrote <a href="/2019/07/01/yes-silver-bullet">Yes silver bullet</a> I wanted to engage with Brooks' essay on its own terms. </p> <p> I do think, however, that one should be sceptical of any argument originating from the ancient Greeks. These people believed that all matter was composed of <a href="https://en.wikipedia.org/wiki/Classical_element">earth, water, air, and fire</a>. They believed in the <a href="https://en.wikipedia.org/wiki/Emission_theory_(vision)">extramission theory of vision</a>. They practised medicine by <a href="https://en.wikipedia.org/wiki/Humorism">trying to balance blood, phlegm, yellow, and black bile</a>. Aristotle believed in <a href="https://en.wikipedia.org/wiki/Spontaneous_generation">spontaneous generation</a>. He also <a href="https://en.wikipedia.org/wiki/History_of_neuroscience">believed that the brain was a radiator while the heart was the seat of intelligence</a>. </p> <p> I think that Aristotle's distinction between essential and accidental complexity is a <a href="https://en.wikipedia.org/wiki/False_dilemma">false dichotomy</a>, at least in the realm of software development. </p> <p> The problem is the assumption that there's a single, immutable underlying reality to be modelled. </p> <h3 id="006f597e772546c2a1083d98ec21f87c"> Modelling reality <a href="#006f597e772546c2a1083d98ec21f87c" title="permalink">#</a> </h3> <p> Jon Suda touches on this as well: <blockquote> <p> "Conceptual (or business) problems are rooted in the problem domain (i.e., the real world)[...]" </p> <p> "Dealing with the "messy real world" is what makes software development hard these days" </p> <footer><cite><a href="https://jonsmusings.com/Full-Silver-Jacket">Jon Suda</a></cite></footer> </blockquote> I agree that the 'real world' (whatever it is) is often messy, and our ability to deal with the mess is how we earn our keep. It seems to me, though, that there's an underlying assumption that there's a single real world to be modelled. </p> <p> I think that a more fruitful perspective is to question that assumption. <a href="http://udidahan.com/2012/03/05/dont-try-to-model-the-real-world-it-doesnt-exist">Don’t try to model the real world, it doesn’t exist.</a> </p> <p> I've mostly worked with business software. Web shops, <a href="https://twitter.com/ploeh/status/530320252790669313">BLOBAs</a>, and other software to support business endeavours. This is the realm implicitly addressed by <a href="http://amzn.to/WBCwx7">Domain-Driven Design</a> and a technique like <a href="https://en.wikipedia.org/wiki/Behavior-driven_development">behaviour-driven development</a>. The assumption is that there's one or more <em>domain experts</em> who know how the business operates, and the job of the software development team is to translate that understanding into working software. </p> <p> This is often the way it happens. Lord knows that I've been involved in enough fixed-requirements contracts to know that sometimes, all you can do is try to implement the requirements as best you can, regardless of how messy they are. In other words, I agree with Jon Suda that messy problems are a reality. </p> <p> Where I disagree with Brooks and Aristotle is that business processes contain essential complexity. In the old days, before computers, businesses ran according to a set of written and unwritten rules. Routine operations might be fairly consistent, but occasionally, something out of the ordinary would happen. Organisations don't have a script for every eventuality. When the unexpected happens, people wing it. </p> <p> A business may have a way it sells its goods. It may have a standard price list, and a set of discounts that salespeople are authorised to offer. It may have standard payment terms that customers are expected to comply with. It may even have standard operating procedures for dealing with missing payments. </p> <p> Then one day, say, the German government expresses interest in placing an order greater than the business has ever handled before. The Germans, however, want a special discount, and special terms of payment. What happens? Do the salespeople refuse because those requests don't comply with the way the organisation does business? Of course not. The case is escalated to people with the authority to change the rules, and a deal is made. </p> <p> Later, the French government comes by, and a similar situation unfolds, but with the difference that someone else handles the French, and the deal struck is different. </p> <p> The way these two cases are handled could be internally inconsistent. Decisions are made based on concrete contexts, but with incomplete information and based on gut feelings. </p> <p> While there may be a system to how an organisation does routine business, there's no uniform reality to be modelled. </p> <p> You can model standard operating procedures in software, but I think it's a mistake to think that it's a model of reality. It's just an interpretation on how routine business is conducted. </p> <p> There's no single, unyielding essential complexity, because the essence isn't there. </p> <h3 id="e9af89838d33486d8c983b50e154e389"> Shaping reality <a href="#e9af89838d33486d8c983b50e154e389" title="permalink">#</a> </h3> <p> <a href="https://www.infoq.com/presentations/Fowler-North-Crevasse-of-Doom">Dan North tells a story</a> of a project where a core business requirement was the ability to print out data. When investigating the issue, it turned out that users needed to have the printout next to their computers so that they could type the data into another system. When asked whether they wouldn't rather prefer to have the data just appear in the other system, they incredulously replied, <em>"You can do that?!</em> </p> <p> <a href="https://twitter.com/ploeh/status/1259832000095059968">This turns out to be a common experience</a>. Someone may tell you about an essential requirement, but when you investigate, it turns out to be not at all essential. There may not be any essential complexity. </p> <p> There's likely to be complexity, but the distinction between essential and accidental complexity seems artificial. While software can model business processes, it can also shape reality. Consider a company like Amazon. The software that supports Amazon wasn't developed <em>after</em> the company was already established. Amazon developed it concurrently with setting up business. </p> <p> Consider companies like Google, Facebook, Uber, or Airbnb. Such software doesn't model reality; it <em>shapes</em> reality. </p> <p> In the beginning of the IT revolution, the driving force behind business software development was to automate existing real-world processes, but this is becoming increasingly rare. New companies enter the markets digitally born. Older organisations may be on their second or third system since they went digital. Software not only models 'messy' reality - it shapes 'messy' reality. </p> <h3 id="8b73389dded94ac593f577969cbdfbea"> Conclusion <a href="#8b73389dded94ac593f577969cbdfbea" title="permalink">#</a> </h3> <p> It may look as though I fundamentally disagree with Jon Suda's blog post. I don't. I agree with almost all of it. It did, however, inspire me to put my thoughts into writing. </p> <p> My position is that I find the situation more nuanced than Fred Brooks suggests by setting off from Aristotle. I don't think that the distinction between essential and accidental complexity is the whole story. Granted, it provides a fruitful and inspiring perspective, but while we find it insightful, we shouldn't accept it as gospel. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="10c9a14dbcd64f28835e7c136d4f9a93"> <div class="comment-author"><a href="https://jonsmusings.com/">Jon Suda</a></div> <div class="comment-content"> <p> I think I agree with virtually everything said here – if not <em>actually</em> everything 😊 </p> <p> As (virtually, if not <em>actually</em>) always, though, there are a few things I’d like to add, clarify, and elaborate on 😊 </p> <p> I fully share your reluctance to accept ancient Greek philosophy. I once discussed No Silver Bullet (and drank Scotch) with a (non-developer) “philosopher” friend of mine. (Understand: He has a BA in philosophy 😊) He said something to the effect of <em>do you understand this essence/accident theory is considered obsolete?</em> I answered that it was beside the point here – at least as far as I was concerned. For me, the dichotomy serves as an inspiration, a metaphor perhaps, not a rigorous theoretical framework – that’s one of the reasons why I adapted it into the (informal) conceptual/technical distinction. </p> <p> I also 100% agree with the notion that software shouldn’t merely “model” or “automate” reality. In fact, I now remember that back in university, this was one of the central themes of my “thesis” (or whatever it was). I pondered the difference/boundary between analysis and design activities and concluded that the idea of creating a model of the “real world” during analysis and then building a technical representation of this model as part of design/implementation couldn’t adequately describe many of the more “inventive” software projects and products. </p> <p> I don’t, however, believe this makes the conceptual/technical (essential/accidental) distinction go away. Even though the point of a project may not (and should not) be a mere automation of a preexisting reality, you still need to know what you want to achieve (i.e., “invent”), <em>conceptually</em>, to be able to fashion a <em>technical</em> implementation of it. And yes, your conceptual model should be based on what’s possible with all available technology – which is why you’ll hopefully end up with something <em>way</em> better than the old solution. (Note that in my post, I never talk about “modeling” the “messy real world” but rather “dealing with it”; even your revolutionary new solution will have to coexist with the messy outside world.) </p> <p> For me, one of the main lessons of No Silver Bullet, the moral of the story, is this: Developers tend to spend inordinate amounts of time discussing and brooding over geeky technical stuff; perhaps they should, instead, talk to their users a bit more and learn something about the problem domain; that’s where the biggest room for improvement is, in my opinion. </p> </div> <div class="comment-date">2020-05-20 16:40 UTC</div> </div> <div class="comment" id="15e5f7c1886343cdbde0bcd30f7cfbc4"> <div class="comment-author"><a href="https://jonsmusings.com/">Jon Suda</a></div> <div class="comment-content"> <p> FWIW </p> <p> <a href="https://jonsmusings.com/Transmutation-of-Reality">https://jonsmusings.com/Transmutation-of-Reality</a> </p> <p> Let the inspiration feedback loop continue 😊 I agree with more-or-less everything you say; I just don’t necessarily think that your ideas are incompatible with those expressed in No Silver Bullet. That, to a significant extent (but not exclusively), is what my new post is about. As I said in my previous comment, your article reminded me of my university “thesis,” and I felt that the paper I used as its “conceptual framework” was worth presenting in a “non-academic” form. It can mostly be used to support your arguments – I certainly use it that way. </p> </div> <div class="comment-date">2020-05-29 17:28 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. AFK https://blog.ploeh.dk/2020/05/11/afk 2020-05-11T07:04:00+00:00 Mark Seemann <div id="post"> <p> <em>Software development productivity tip: regularly step away from the computer.</em> </p> <p> In these days of covid-19, people are learning that productivity is imperfectly correlated to the amount of time one is physically present in an office. Indeed, the time you spend in front of you computer is hardly correlated to your productivity. After all, <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">programming productivity isn't measured by typing speed</a>. Additionally, you can waste much time at the computer, checking Twitter, watching cat videos on YouTube, etc. </p> <h3 id="63ec07d591da4ac0b8f47501ba65eb6a"> Pomodorobut <a href="#63ec07d591da4ac0b8f47501ba65eb6a" title="permalink">#</a> </h3> <p> I've worked from home for years, so I thought I'd share some productivity tips. I only report what works for me. If you can use some of the tips, then great. If they're not for you, that's okay too. I don't pretend that I've found any secret sauce or universal truth. </p> <p> A major problem with productivity is finding the discipline to work. I use <em>Pomodorobut</em>. It's like <a href="https://www.scrum.org/resources/what-scrumbut">Scrumbut</a>, but for the <a href="https://en.wikipedia.org/wiki/Pomodoro_Technique">Pomodoro technique</a>. For years I thought I was using the Pomodoro technique, but <a href="https://twitter.com/arialdomartini/status/1238426025429794818">Arialdo Martini makes a compelling case that I'm not</a>. He suggests just calling it <em>timeboxing</em>. </p> <p> I think it's a little more than that, though, because the way I use Pomodorobut has two equally important components: <ul> <li>The 25-minute work period</li> <li>The 5-minute break</li> </ul> The 25 minutes is a sufficiently short period that even if you loathe the task ahead of you, you can summon the energy to do it for 25 minutes. Once you get started, it's usually not that bad. </p> <p> When you program, you often run into problems: <ul> <li>There's a bug that you don't understand.</li> <li>You can't figure out the correct algorithm to implement a particular behaviour.</li> <li>Your code doesn't compile, and you don't understand why.</li> <li>A framework behaves inexplicably.</li> </ul> When you're in front of your computer, you can be stuck at such problems for hours on end. </p> <p> The solution: take a break. </p> <h3 id="402a30606d684e179e8a0379c967a31a"> Breaks give a fresh perspective <a href="#402a30606d684e179e8a0379c967a31a" title="permalink">#</a> </h3> <p> I take my Pomodorobut breaks seriously. My rule is that I must leave my office during the break. I usually go get a glass of water or the like. The point is to get out of the chair and <em>afk</em> (away from keyboard). </p> <p> While I'm out the room, it often happens that I get an insight. If I'm stuck on something, I may think of a potential solution, or I may realise that the problem is irrelevant, because of a wider context I forgot about when I sat in front of the computer. </p> <p> You may have heard about <a href="https://en.wikipedia.org/wiki/Rubber_duck_debugging">rubber ducking</a>. Ostensibly <blockquote> <p> "By having to verbalize [...], you may suddenly gain new insight into the problem." </p> <footer><cite><a href="http://bit.ly/the-pragmatic-programmer">Andy Hunt and Dave Thomas</a></cite></footer> </blockquote> I've tried it enough times: you ask a colleague if he or she has a minute, launch into an explanation of your problem, only to break halfway through and say: <em>"Never mind! I suddenly realised what the problem is. Thank you for your help."</em> </p> <p> Working from home, I haven't had a colleague I could disturb like that for years, and I don't actually use a rubber duck. In my experience, getting out of my chair works equally well. </p> <p> The Pomodorobut technique makes me productive because the breaks, and the insights they spark, reduce the time I waste on knotty problems. When I'm in danger of becoming stuck, I often find a way forward in less than 30 minutes: at most 25 minutes being stuck, and a 5-minute break to get unstuck. </p> <h3 id="814a2859670247dc8add4a6d1a2cd29c"> Hammock-driven development <a href="#814a2859670247dc8add4a6d1a2cd29c" title="permalink">#</a> </h3> <p> Working from home gives you extra flexibility. I have a regular routine where I go for a run around 10 in the morning. I also routinely go grocery shopping around 14 in the afternoon. Years ago, when I still worked in an office, I'd ride my bicycle to and from work every day. I've had my good ideas during those activities. </p> <p> In fact, I can't recall ever having had a profound insight in front of the computer. They always come to me when I'm away from it. For instance, I distinctly remember walking around in my apartment doing other things when I realised that <a href="/2018/06/25/visitor-as-a-sum-type">the Visitor design pattern is just another encoding of a sum type</a>. </p> <p> Insights don't come for free. As Rich Hickey points out in his talk about <a href="https://youtu.be/f84n5oFoZBc">hammock-driven development</a>, you must feed your 'background mind'. That involves deliberate focus on the problem. </p> <p> Good ideas don't come if you're permanently away from the computer, but neither do they arrive if all you do is stare at the screen. It's the variety that makes you productive. </p> <h3 id="3bcce686b9ed4deba31c7e9f7bb27893"> Conclusion <a href="#3bcce686b9ed4deba31c7e9f7bb27893" title="permalink">#</a> </h3> <p> Software development productivity is weakly correlated with the time you spend in front of the computer. I find that I'm most productive when I can vary my activities during the day. Do a few 25-minute sessions, rigidly interrupted by breaks. Go for a run. Work a bit more on the computer. Have lunch. Do one or two more time-boxed sessions. Go grocery shopping. Conclude with a final pair of work sessions. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Significant whitespace is DRY https://blog.ploeh.dk/2020/05/04/significant-whitespace-is-dry 2020-05-04T10:02:00+00:00 Mark Seemann <div id="post"> <p> <em>Languages with explicit scoping encourage you to repeat yourself.</em> </p> <p> When the talk falls on significant whitespace, the most common reaction I hear seems to be one of nervous <a href="https://en.wikipedia.org/wiki/Apotropaic_magic">apotropaic</a> deflection. <blockquote> <p> "Indentation matters? Oh, no! I'm horrified." </p> <footer><cite><a href="https://dotnetrocks.com/?show=1665">Carl Franklin</a></cite></footer> </blockquote> I've always wondered why people react like that. What's the problem with significant whitespace? </p> <p> If given a choice, I'd <em>prefer</em> indentation to matter. In that way, I don't have to declare scope more than once. </p> <h3 id="d4077d55ab644786b54672af077f4729"> Explicit scope <a href="#d4077d55ab644786b54672af077f4729" title="permalink">#</a> </h3> <p> If you had to choose between the following three C# implementations of the <a href="http://codingdojo.org/kata/FizzBuzz/">FizzBuzz kata</a>, which one would you choose? </p> <p> <em>a:</em> </p> <p> <pre><span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Program</span> { <span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Main</span>() { <span style="font-weight:bold;color:#8f08c4;">for</span>&nbsp;(<span style="color:blue;">int</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;=&nbsp;1;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;&lt;&nbsp;100;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>++) { <span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;15&nbsp;==&nbsp;0) { <span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;FizzBuzz&quot;</span>); } <span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;3&nbsp;==&nbsp;0) { <span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Fizz&quot;</span>); } <span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;5&nbsp;==&nbsp;0) { <span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Buzz&quot;</span>); } <span style="font-weight:bold;color:#8f08c4;">else</span> { <span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="font-weight:bold;color:#1f377f;">i</span>); } } } }</pre> </p> <p> <em>b:</em> </p> <p> <pre><span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Program</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Main</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">for</span>&nbsp;(<span style="color:blue;">int</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;=&nbsp;1;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;&lt;&nbsp;100;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>++) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;15&nbsp;==&nbsp;0) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;FizzBuzz&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;3&nbsp;==&nbsp;0) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Fizz&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;5&nbsp;==&nbsp;0) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Buzz&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="font-weight:bold;color:#1f377f;">i</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> <em>c:</em> </p> <p> <pre><span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Program</span>&nbsp;{&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Main</span>()&nbsp;{&nbsp;<span style="font-weight:bold;color:#8f08c4;">for</span>&nbsp;(<span style="color:blue;">int</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;=&nbsp;1;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;&lt;&nbsp;100;&nbsp;<span style="font-weight:bold;color:#1f377f;">i</span>++)&nbsp;{&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;15&nbsp;==&nbsp;0)&nbsp;{&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;FizzBuzz&quot;</span>);&nbsp;}&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;3&nbsp;==&nbsp;0)&nbsp;{&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Fizz&quot;</span>);&nbsp;}&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">i</span>&nbsp;%&nbsp;5&nbsp;==&nbsp;0)&nbsp;{&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="color:#a31515;">&quot;Buzz&quot;</span>);&nbsp;}&nbsp;<span style="font-weight:bold;color:#8f08c4;">else</span>&nbsp;{&nbsp;<span style="color:#2b91af;">Console</span>.<span style="color:#74531f;">WriteLine</span>(<span style="font-weight:bold;color:#1f377f;">i</span>);&nbsp;}&nbsp;}&nbsp;}&nbsp;}</pre> </p> <p> Which of these do you prefer? <em>a</em>, <em>b</em>, or <em>c?</em> </p> <p> You prefer <em>b.</em> Everyone does. </p> <p> Yet, those three examples are equivalent. Not only do they behave the same - except for whitespace, they're identical. They produce the same effective abstract syntax tree. </p> <p> Even though a language like C# has explicit scoping and statement termination with its curly brackets and semicolons, indentation still matters. It doesn't matter to the compiler, but it matters to humans. </p> <p> When you format code like option <em>b</em>, you express scope twice. Once for the compiler, and once for human readers. You're repeating yourself. </p> <h3 id="f0433a7047ca4809ae347082f9ca6726"> Significant whitespace <a href="#f0433a7047ca4809ae347082f9ca6726" title="permalink">#</a> </h3> <p> Some languages dispense with the <a href="/2019/12/16/zone-of-ceremony">ceremony</a> and let indentation indicate scope. The most prominent is <a href="https://www.python.org">Python</a>, but I've more experience with <a href="https://fsharp.org">F#</a> and <a href="https://www.haskell.org">Haskell</a>. In F#, you could write FizzBuzz like this: </p> <p> <pre>[&lt;EntryPoint&gt;] <span style="color:blue;">let</span>&nbsp;main&nbsp;argv&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;i&nbsp;<span style="color:blue;">in</span>&nbsp;[1..100]&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;i&nbsp;%&nbsp;15&nbsp;=&nbsp;0&nbsp;<span style="color:blue;">then</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;FizzBuzz&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;i&nbsp;%&nbsp;3&nbsp;=&nbsp;0&nbsp;<span style="color:blue;">then</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;Fizz&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;i&nbsp;%&nbsp;5&nbsp;=&nbsp;0&nbsp;<span style="color:blue;">then</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;Buzz&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;%i&quot;</span>&nbsp;i &nbsp;&nbsp;&nbsp;&nbsp;0</pre> </p> <p> You don't have to explicitly scope variables or expressions. The scope is automatically indicated by the indentation. You <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">don't repeat yourself</a>. Scope is expressed once, and both compiler and human understand it. </p> <p> I've programmed in F# for a decade now, and I don't find its use of significant whitespace to be a problem. I'd recommend, however, to turn on the feature in your IDE of choice that shows whitespace characters. </p> <h3 id="31e1f00baf044bab8504cf4e2a492773"> Summary <a href="#31e1f00baf044bab8504cf4e2a492773" title="permalink">#</a> </h3> <p> Significant whitespace is a good language feature. You're going to indent your code anyway, so why not let the indentation carry meaning? In that way, you don't repeat yourself. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An F# implementation of the Maître d' kata https://blog.ploeh.dk/2020/04/27/an-f-implementation-of-the-maitre-d-kata 2020-04-27T14:41:00+00:00 Mark Seemann <div id="post"> <p> <em>This article walks you through the Ma&icirc;tre d' kata done in F#.</em> </p> <p> In a previous article, I presented <a href="/2020/01/27/the-maitre-d-kata">the Ma&icirc;tre d' kata</a> and promised to publish a walkthrough. Here it is. </p> <h3 id="f7bc269790014e9d9510c4c034e9a1bb"> Preparation <a href="#f7bc269790014e9d9510c4c034e9a1bb" title="permalink">#</a> </h3> <p> I used <a href="/2019/10/21/a-red-green-refactor-checklist">test-driven development</a> and <a href="https://fsharp.org">F#</a> for both unit tests and implementation. As usual, my test framework was <a href="https://xunit.net">xUnit.net</a> (2.4.0) with <a href="https://github.com/SwensenSoftware/unquote">Unquote</a> (5.0.0) as the assertion library. </p> <p> I could have done the exercise with a <a href="/property-based-testing-intro">property-based testing</a> framework like <a href="https://fscheck.github.io/FsCheck">FsCheck</a> or <a href="https://github.com/hedgehogqa/fsharp-hedgehog">Hedgehog</a>, but I chose instead to take my own medicine. In the kata description, I suggested some test cases, so I wanted to try and see if they made sense. </p> <p> The entire code base is <a href="https://github.com/ploeh/maitred-kata-in-fsharp-1">available on GitHub</a>. </p> <h3 id="adc8a50bbe72453c87853ae566b9a185"> Boutique restaurant <a href="#adc8a50bbe72453c87853ae566b9a185" title="permalink">#</a> </h3> <p> I wrote the first suggested test case this way: </p> <p> <pre>[&lt;Fact&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;test&nbsp;&lt;@&nbsp;canAccept&nbsp;12&nbsp;[]&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;1&nbsp;}&nbsp;@&gt;</pre> </p> <p> This uses Unquote's <code>test</code> function to verify that a Boolean expression is true. The expression is a function call to <code>canAccept</code> with the capacity <code>12</code>, no existing reservations, and a reservation with <code>Quantity = 1</code>. </p> <p> The simplest thing that could possibly work was this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Reservation&nbsp;=&nbsp;{&nbsp;Quantity&nbsp;:&nbsp;int&nbsp;} <span style="color:blue;">let</span>&nbsp;canAccept&nbsp;_&nbsp;_&nbsp;_&nbsp;=&nbsp;<span style="color:blue;">true</span></pre> </p> <p> The <code>Reservation</code> type was required to make the test compile, but the <code>canAccept</code> function didn't have to consider its arguments. It could simply return <code>true</code>. </p> <h3 id="d169683993034e3c86db04a1dfaca279"> Parametrisation <a href="#d169683993034e3c86db04a1dfaca279" title="permalink">#</a> </h3> <p> The next test case made me turn the test function into a parametrised test: </p> <p> <pre>[&lt;Theory&gt;] [&lt;InlineData(&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>)&gt;] [&lt;InlineData(13,&nbsp;<span style="color:blue;">false</span>)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;quantity&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;canAccept&nbsp;12&nbsp;[]&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}</pre> </p> <p> So far, the only test parameters were <code>quantity</code> and the <code>expected</code> result. I could no longer use <code>test</code> to verify the result of calling <code>canAccept</code>, since I added variation to the <code>expected</code> result. I changed <code>test</code> into Unquote's <code>=!</code> (<em>must equal</em>) operator. </p> <p> The simplest passing implementation I could think of was: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;_&nbsp;_&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q&nbsp;}&nbsp;=&nbsp;q&nbsp;=&nbsp;1</pre> </p> <p> It ignored the capacity and instead checked whether <code>q</code> is <code>1</code>. That passed both tests. </p> <h3 id="43fddd9f6ad845aa85b4f04357815659"> Test data API <a href="#43fddd9f6ad845aa85b4f04357815659" title="permalink">#</a> </h3> <p> Before adding another test case, I decided to refactor my test code a bit. When working with a real domain model, you often have to furnish test data in order to make code compile - even if that data isn't relevant to the test. I wanted to demonstrate how to deal with this issue. My first step was to introduce an 'arbitrary' <code>Reservation</code> value in the spirit of <a href="/2017/09/11/test-data-without-builders">Test data without Builders</a>. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;aReservation&nbsp;=&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;1&nbsp;}</pre> </p> <p> This enabled me to rewrite the test: </p> <p> <pre>[&lt;Theory&gt;] [&lt;InlineData(&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>)&gt;] [&lt;InlineData(13,&nbsp;<span style="color:blue;">false</span>)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;quantity&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;canAccept&nbsp;12&nbsp;[]&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}</pre> </p> <p> This doesn't look like an immediate improvement, but it made it possible to make the <code>Reservation</code> record type more realistic without damage to the test: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Reservation&nbsp;=&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;:&nbsp;DateTime &nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;:&nbsp;string &nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;:&nbsp;string &nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;:&nbsp;int&nbsp;}</pre> </p> <p> I added some fields that a real-world reservation would have. The <code>Quantity</code> field will be useful later on, but the <code>Name</code> and <code>Email</code> fields are irrelevant in the context of the kata. </p> <p> This is the type of API change that often gives people grief. To create a <code>Reservation</code> value, you <em>must</em> supply all four fields. This often adds noise to tests. </p> <p> Not here, though, because the only concession I had to make was to change <code>aReservation</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;aReservation&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Date&nbsp;=&nbsp;DateTime&nbsp;(2019,&nbsp;11,&nbsp;29,&nbsp;12,&nbsp;0,&nbsp;0) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;}</pre> </p> <p> The test code remained unaltered. </p> <p> With that in place, I could add the third test case: </p> <p> <pre>[&lt;Theory&gt;] [&lt;InlineData(&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>)&gt;] [&lt;InlineData(13,&nbsp;<span style="color:blue;">false</span>)&gt;] [&lt;InlineData(12,&nbsp;&nbsp;<span style="color:blue;">true</span>)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;quantity&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;canAccept&nbsp;12&nbsp;[]&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}</pre> </p> <p> The simplest passing implementation I could think of was: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;_&nbsp;_&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q&nbsp;}&nbsp;=&nbsp;q&nbsp;&lt;&gt;&nbsp;13</pre> </p> <p> This implementation still ignored the restaurant's capacity and simply checked that <code>q</code> was different from <code>13</code>. That was enough to pass all three tests. </p> <h3 id="b25e3ea455db4fc6a0a18ed84d3be679"> Refactor test case code <a href="#b25e3ea455db4fc6a0a18ed84d3be679" title="permalink">#</a> </h3> <p> Adding the next suggested test case proved to be a problem. I wanted to write a single <code>[&lt;Theory&gt;]</code>-driven test function fed by all the <em>Boutique restaurant</em> test data. To do that, I'd have to supply arrays of test input, but unfortunately, <a href="https://github.com/dotnet/fsharp/issues/7916">that wasn't possible in F#</a>. </p> <p> Instead I decided to refactor the test case code to use <code>ClassData</code>-driven test cases. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(13,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;<span style="color:blue;">true</span>) [&lt;Theory;&nbsp;ClassData(typeof&lt;BoutiqueTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;quantity&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;canAccept&nbsp;12&nbsp;[]&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}</pre> </p> <p> These are the same test cases as before, but now expressed by a class inheriting from <code>TheoryData&lt;int, bool&gt;</code>. The implementing code remains the same. </p> <h3 id="d1c356e19bd940cd8e37c24848b4f896"> Existing reservation <a href="#d1c356e19bd940cd8e37c24848b4f896" title="permalink">#</a> </h3> <p> The next suggested test case includes an existing reservation. To support that, I changed the test case base class to <code>TheoryData&lt;int, int list, int, bool&gt;</code>, and passed empty lists for the first three test cases. For the new, fourth test case, I supplied a number of seats. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;13,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;12,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;[2],&nbsp;&nbsp;3,&nbsp;<span style="color:blue;">false</span>) [&lt;Theory;&nbsp;ClassData(typeof&lt;BoutiqueTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;capacity&nbsp;reservatedSeats&nbsp;quantity&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;List.map&nbsp;(<span style="color:blue;">fun</span>&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;s&nbsp;})&nbsp;reservatedSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;capacity&nbsp;rs&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> This also forced me to to change the body of the test function. At this stage, it could be prettier, but it got the job done. I soon after improved it. </p> <p> My implementation, as usual, was <em>the simplest thing that could possibly work</em>. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;_&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;q&nbsp;&lt;&gt;&nbsp;13&nbsp;&amp;&amp;&nbsp;Seq.isEmpty&nbsp;reservations</pre> </p> <p> Notice that although the fourth test case varied the <code>capacity</code>, I still managed to pass all tests without looking at it. </p> <h3 id="b08aade19008419cab173c9ab720cfca"> Accept despite existing reservation <a href="#b08aade19008419cab173c9ab720cfca" title="permalink">#</a> </h3> <p> The next test case introduced another existing reservation, but this time with enough capacity to accept a new reservation. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;13,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;[],&nbsp;12,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;[2],&nbsp;&nbsp;3,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;[2],&nbsp;&nbsp;3,&nbsp;&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> The test function remained unchanged. </p> <p> In the spirit of the <a href="/2019/10/07/devils-advocate">Devil's advocate technique</a>, I actively sought to avoid a correct implementation. I came up with this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;capacity&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;Seq.tryHead&nbsp;reservations&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;0 &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> Since all test cases supplied at most one existing reservation, it was enough to consider the first reservation, if present. </p> <p> To many people, it may seem strange to actively seek out incorrect implementations like this. An incorrect implementation that passes all tests does, however, demonstrate the need for more tests. </p> <h3 id="587afbd29a9d41879994d2663c26f8bb"> The sum of all reservations <a href="#587afbd29a9d41879994d2663c26f8bb" title="permalink">#</a> </h3> <p> I then added another test case, this time with three existing reservations: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;&nbsp;1,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;13,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;12,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2],&nbsp;&nbsp;3,&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2],&nbsp;&nbsp;3,&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;[3;2;3],&nbsp;&nbsp;3,&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> Again, I left the test function untouched. </p> <p> On the side of the implementation, I couldn't think of more hoops to jump through, so I finally gave in and provided a 'proper' implementation: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;capacity&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> Not only does it look simpler that before, but I also felt that the implementation was warranted. <blockquote> <p> “As the tests get more specific, the code gets more generic.” </p> <footer><cite><a href="https://blog.cleancoder.com/uncle-bob/2013/05/27/TheTransformationPriorityPremise.html">Robert C. Martin</a></cite></footer> </blockquote> Although I'd only tested <code>canAccept</code> with lists, I decided to implement it with <code>Seq</code>. This was a decision I later regretted. </p> <h3 id="987a4fc4119a4ab29bc968712659bdc2"> Another date <a href="#987a4fc4119a4ab29bc968712659bdc2" title="permalink">#</a> </h3> <p> The last <em>Boutique restaurant</em> test case was to supply an existing reservation on another date. The <code>canAccept</code> function should only consider existing reservations on the date in question. </p> <p> First, I decided to model the two separate dates as two values: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;d1&nbsp;=&nbsp;DateTime&nbsp;(2023,&nbsp;9,&nbsp;14) <span style="color:blue;">let</span>&nbsp;d2&nbsp;=&nbsp;DateTime&nbsp;(2023,&nbsp;9,&nbsp;15)</pre> </p> <p> I hoped that it would make my test cases more readable, because the dates would have a denser representation. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(&nbsp;1,&nbsp;d1),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(13,&nbsp;d1),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(12,&nbsp;d1),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;[(3,&nbsp;d1);(2,&nbsp;d1);(3,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d2)],&nbsp;(&nbsp;3,&nbsp;d1),&nbsp;&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> I changed the representation of a reservation from just an <code>int</code> to a tuple of a number and a date. I also got tired of looking at that noisy unit test, so I introduced a test-specific helper function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;reserve&nbsp;(q,&nbsp;d)&nbsp;=&nbsp;{&nbsp;aReservation&nbsp;<span style="color:blue;">with</span>&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}</pre> </p> <p> Since it takes a tuple of a number and a date, I could use it to simplify the test function: </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;BoutiqueTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;(capacity,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;reserve&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;capacity&nbsp;reservations&nbsp;(reserve&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> The <code>canAccept</code> function now had to filter the <code>reservations</code> on <code>Date</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;capacity&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;relevantReservations&nbsp;=&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date&nbsp;=&nbsp;d)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;relevantReservations &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> This implementation specifically compared dates, though, so while it passed all tests, it'd behave incorrectly if the dates were as much as nanosecond off. That implied that another test case was required. </p> <h3 id="535fc318a9b74ffda89e4e02992efe41"> Same date, different time <a href="#535fc318a9b74ffda89e4e02992efe41" title="permalink">#</a> </h3> <p> The final test case for the <em>Boutique restaurant</em>, then, was to use two <code>DateTime</code> values on the same date, but with different times. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;BoutiqueTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(&nbsp;1,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(13,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(12,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(12,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(10,&nbsp;[(3,&nbsp;d1);(2,&nbsp;d1);(3,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d2)],&nbsp;(&nbsp;3,&nbsp;d1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;4,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d1)],&nbsp;(&nbsp;3,&nbsp;d1.AddHours&nbsp;1.),&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> I just added a new test case as a new line and lined up the data. The test function, again, didn't change. </p> <p> To address the new test case, I generalised the first filter. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;capacity&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;relevantReservations&nbsp;=&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date.Date&nbsp;=&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;relevantReservations &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> An expression like <code>r.Date.Date</code> looks a little odd. <code>DateTime</code> values have a <code>Date</code> property that represents its date part. The first <code>Date</code> is the <code>Reservation</code> field, and the second is the date part. </p> <p> I was now content with the <em>Boutique restaurant</em> implementation. </p> <h3 id="ea07bcad3ac8429194c55edddb7187d7"> Haute cuisine <a href="#ea07bcad3ac8429194c55edddb7187d7" title="permalink">#</a> </h3> <p> In the next phase of the kata, I now had to deal with a configuration of more than one table, so I introduced a type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Table&nbsp;=&nbsp;{&nbsp;Seats&nbsp;:&nbsp;int&nbsp;}</pre> </p> <p> It's really only a glorified wrapper around an <code>int</code>, but with a real domain model in place, I could make its constructor private and instead afford a smart constructor that only accepts positive integers. </p> <p> I changed the <code>canAccept</code> function to take a list of tables, instead of <code>capacity</code>. This also required me to change the existing test function to take a singleton list of tables: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;[table&nbsp;capacity]&nbsp;reservations&nbsp;(reserve&nbsp;r)</pre> </p> <p> where <code>table</code> is a test-specific helper function: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;table&nbsp;s&nbsp;=&nbsp;{&nbsp;Seats&nbsp;=&nbsp;s&nbsp;}</pre> </p> <p> I also added a new test function and a single test case: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;d3&nbsp;=&nbsp;DateTime&nbsp;(2024,&nbsp;6,&nbsp;&nbsp;7) <span style="color:blue;">type</span>&nbsp;HauteCuisineTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;[],&nbsp;(4,&nbsp;d3),&nbsp;<span style="color:blue;">true</span>) [&lt;Theory;&nbsp;ClassData(typeof&lt;HauteCuisineTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Haute&nbsp;cuisine``&nbsp;(tableSeats,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;tables&nbsp;=&nbsp;List.map&nbsp;table&nbsp;tableSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;reserve&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;tables&nbsp;reservations&nbsp;(reserve&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> The change to <code>canAccept</code> is modest: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;tables&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;t.Seats)&nbsp;tables &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;relevantReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date.Date&nbsp;=&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;relevantReservations &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> It still works by looking at a total capacity as if there was just a single communal table. Now it just calculates <code>capacity</code> from the sequence of <code>tables</code>. </p> <h3 id="334eab19b5864f72a92290cfe5031b66"> Reject reservation that doesn't fit largest table <a href="#334eab19b5864f72a92290cfe5031b66" title="permalink">#</a> </h3> <p> Then I added the next test case to the new test function: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;HauteCuisineTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;[],&nbsp;(4,&nbsp;d3),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;[],&nbsp;(5,&nbsp;d3),&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> This one attempts to make a reservation for five people. The largest table only fits four people, so this reservation should be rejected. The current implementation just considered the total capacity of all tables, to it accepted the reservation, and thereby failed the test. </p> <p> This change to <code>canAccept</code> passes all tests: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;tables&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;=&nbsp;tables&nbsp;|&gt;&nbsp;Seq.map&nbsp;(<span style="color:blue;">fun</span>&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;t.Seats)&nbsp;|&gt;&nbsp;Seq.max &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;relevantReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date.Date&nbsp;=&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;relevantReservations &nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> The function now only considered the largest table in the restaurant. While it's incorrect to ignore all other tables, all tests passed. </p> <h3 id="54e03a3eeb744db3bc82b75aace9556f"> Accept when there's still a remaining table <a href="#54e03a3eeb744db3bc82b75aace9556f" title="permalink">#</a> </h3> <p> Only considering the largest table is obviously wrong, so I added another test case where there's an existing reservation. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;HauteCuisineTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(4,&nbsp;d3),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(5,&nbsp;d3),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;&nbsp;[2;2;4],&nbsp;[(2,&nbsp;d3)],&nbsp;(4,&nbsp;d3),&nbsp;&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> While <code>canAccept</code> should accept the reservation, it didn't when I added the test case. In a variation of the <a href="/2019/10/07/devils-advocate">Devil's Advocate</a> technique, I came up with this implementation to pass all tests: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;tables&nbsp;reservations&nbsp;{&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;largestTable&nbsp;=&nbsp;tables&nbsp;|&gt;&nbsp;Seq.map&nbsp;(<span style="color:blue;">fun</span>&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;t.Seats)&nbsp;|&gt;&nbsp;Seq.max &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;capacity&nbsp;=&nbsp;tables&nbsp;|&gt;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;t.Seats) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;relevantReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date.Date&nbsp;=&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;=&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;relevantReservations &nbsp;&nbsp;&nbsp;&nbsp;q&nbsp;&lt;=&nbsp;largestTable&nbsp;&amp;&amp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity</pre> </p> <p> This still wasn't the correct implementation. It represented a return to looking at the total capacity of all tables, with the extra rule that you couldn't make a reservation larger than the largest table. At least one more test case was needed. </p> <h3 id="d900c55b4ebc4bcbba572cd8cbca33ed"> Accept when remaining table is available <a href="#d900c55b4ebc4bcbba572cd8cbca33ed" title="permalink">#</a> </h3> <p> I added another test case to the <em>haute cuisine</em> test cases. This one came with one existing reservation for three people, effectively reserving the four-person table. While the remaining tables have an aggregate capacity of four, it's two separate tables. Therefore, a reservation for four people should be rejected. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;HauteCuisineTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(4,&nbsp;d3),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;([2;2;4;4],&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[],&nbsp;(5,&nbsp;d3),&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;&nbsp;[2;2;4],&nbsp;[(2,&nbsp;d3)],&nbsp;(4,&nbsp;d3),&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;(&nbsp;&nbsp;[2;2;4],&nbsp;[(3,&nbsp;d3)],&nbsp;(4,&nbsp;d3),&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> It then dawned on me that I had to explicitly distinguish between a communal table configuration, and individual tables that aren't communal, regardless of size. This triggered quite a refactoring. </p> <p> I defined a new type to distinguish between these two types of table layout: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;TableConfiguration&nbsp;=&nbsp;Communal&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;|&nbsp;Tables&nbsp;<span style="color:blue;">of</span>&nbsp;Table&nbsp;list</pre> </p> <p> I also had to change the existing test functions, including the <em>boutique restaurant</em> test </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;BoutiqueTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Boutique&nbsp;restaurant``&nbsp;(capacity,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;reserve&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;(Communal&nbsp;capacity)&nbsp;reservations&nbsp;(reserve&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre></pre> </p> <p> and the <em>haute cuisine</em> test </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;HauteCuisineTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Haute&nbsp;cuisine``&nbsp;(tableSeats,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;tables&nbsp;=&nbsp;List.map&nbsp;table&nbsp;tableSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;reserve&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;(Tables&nbsp;tables)&nbsp;reservations&nbsp;(reserve&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> In both cases I had to change the call to <code>canAccept</code> to pass either a <code>Communal</code> or a <code>Tables</code> value. </p> <h3 id="c3c6acc07a0f4e129fc7159247a7058a"> Delete first <a href="#c3c6acc07a0f4e129fc7159247a7058a" title="permalink">#</a> </h3> <p> I'd previously done the kata in <a href="https://www.haskell.org">Haskell</a> and was able to solve this phase of the kata using the built-in <code>deleteFirstsBy</code> function. This function doesn't exist in the F# core library, so I decided to add it. I created a new module named <code>Seq</code> and first defined a function that deletes the first element that satisfies a predicate: </p> <p> <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;bool)&nbsp;-&gt;&nbsp;seq&lt;&#39;a&gt;&nbsp;-&gt;&nbsp;seq&lt;&#39;a&gt;</span> <span style="color:blue;">let</span>&nbsp;deleteFirstBy&nbsp;pred&nbsp;(xs&nbsp;:&nbsp;_&nbsp;seq)&nbsp;=&nbsp;seq&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;found&nbsp;=&nbsp;<span style="color:blue;">false</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;e&nbsp;=&nbsp;xs.GetEnumerator&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">while</span>&nbsp;e.MoveNext&nbsp;()&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;found &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:blue;">yield</span>&nbsp;e.Current &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;pred&nbsp;e.Current &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;found&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;<span style="color:blue;">true</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">yield</span>&nbsp;e.Current &nbsp;&nbsp;&nbsp;&nbsp;}</pre> </p> <p> It moves over a sequence of elements and looks for an element that satisfies <code>pred</code>. If such an element is found, it's omitted from the output sequence. The function only deletes the first occurrence from the sequence, so any other elements that satisfy the predicate are still included. </p> <p> This function corresponds to Haskell's <code>deleteBy</code> function and can be used to implement <code>deleteFirstsBy</code>: </p> <p> <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b&nbsp;-&gt;&nbsp;bool)&nbsp;-&gt;&nbsp;seq&lt;&#39;b&gt;&nbsp;-&gt;&nbsp;seq&lt;&#39;a&gt;&nbsp;-&gt;&nbsp;seq&lt;&#39;b&gt;</span> <span style="color:blue;">let</span>&nbsp;deleteFirstsBy&nbsp;pred&nbsp;=&nbsp;Seq.fold&nbsp;(<span style="color:blue;">fun</span>&nbsp;xs&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;deleteFirstBy&nbsp;(pred&nbsp;x)&nbsp;xs)</pre> </p> <p> As <a href="https://hackage.haskell.org/package/base/docs/Data-List.html#v:deleteFirstsBy">the Haskell documentation</a> explains, the "<code>deleteFirstsBy</code> function takes a predicate and two lists and returns the first list with the first occurrence of each element of the second list removed." My F# function does the same, but works on sequences instead of linked lists. </p> <p> I could use it to find and remove tables that were already reserved. </p> <h3 id="f2c9fa5bfc6642378cf0d8edb2a637d5"> Find remaining tables <a href="#f2c9fa5bfc6642378cf0d8edb2a637d5" title="permalink">#</a> </h3> <p> I first defined a little helper function to determine whether a table can accommodate a reservation: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;Table&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;fits&nbsp;r&nbsp;t&nbsp;=&nbsp;r.Quantity&nbsp;&lt;=&nbsp;t.Seats</pre> </p> <p> The rule is simply that the table's number of <code>Seats</code> must be greater than or equal to the reservation's <code>Quantity</code>. I could use this function for the predicate for <code>Seq.deleteFirstsBy</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;config&nbsp;reservations&nbsp;({&nbsp;Quantity&nbsp;=&nbsp;q;&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;<span style="color:blue;">as</span>&nbsp;r)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;contemporaneousReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Date.Date&nbsp;=&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;config&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;capacity&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;q&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Tables&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;Seq.sort&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingTables&nbsp;=&nbsp;Seq.deleteFirstsBy&nbsp;fits&nbsp;(Seq.sort&nbsp;tables)&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.exists&nbsp;(fits&nbsp;r)&nbsp;remainingTables</pre> </p> <p> The <code>canAccept</code> function now branched on <code>Communal</code> versus <code>Tables</code> configurations. In the <code>Communal</code> configuration, it simply compared the <code>reservedSeats</code> and reservation quantity to the communal table's <code>capacity</code>. </p> <p> In the <code>Tables</code> case, the function used <code>Seq.deleteFirstsBy fits</code> to remove all the tables that are already reserved. The result is the <code>remainingTables</code>. If there exists a remaining table that <code>fits</code> the reservation, then the function accepts the reservation. </p> <p> This seemed to me an appropriate implementation of the <em>haute cuisine</em> phase of the kata. </p> <h3 id="5a4a8061a2324ceab96e22a10a5ff445"> Second seatings <a href="#5a4a8061a2324ceab96e22a10a5ff445" title="permalink">#</a> </h3> <p> Now it was time to take seating duration into account. While I could have written my test cases directly against the <a href="https://docs.microsoft.com/dotnet/api/system.timespan">TimeSpan API</a>, I didn't want to write <code>TimeSpan.FromHours 2.5</code>, <code>TimeSpan.FromDays 1.</code>, and so on. I found that it made my test cases harder to read, so I added some literal extensions: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Int32&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.hours&nbsp;=&nbsp;TimeSpan.FromHours&nbsp;(float&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.days&nbsp;=&nbsp;TimeSpan.FromDays&nbsp;(float&nbsp;x)</pre> </p> <p> This enabled me to write expressions like <code>1 .days</code> and <code>2 .hours</code>, as shown in the first test case: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;d4&nbsp;=&nbsp;DateTime&nbsp;(2023,&nbsp;10,&nbsp;22,&nbsp;18,&nbsp;0,&nbsp;0) <span style="color:blue;">type</span>&nbsp;SecondSeatingsTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;TimeSpan,&nbsp;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;(2&nbsp;.hours,&nbsp;[2;2;4],&nbsp;[(4,&nbsp;d4)],&nbsp;(3,&nbsp;d4.Add&nbsp;(2&nbsp;.hours)),&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> I used this initial parametrised test case for a new test function: </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;SecondSeatingsTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Second&nbsp;seatings``&nbsp;(dur,&nbsp;tableSeats,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;tables&nbsp;=&nbsp;List.map&nbsp;table&nbsp;tableSeats &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;reserve&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;dur&nbsp;(Tables&nbsp;tables)&nbsp;reservations&nbsp;(reserve&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> My motivation for this test case was mostly to introduce an API change to <code>canAccept</code>. I didn't want to rock the boat too much, so I picked a test case that wouldn't trigger a big change to the implementation. I prefer incremental changes. The only change is the introduction of the <code>seatingDur</code> argument: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;canAccept&nbsp;(seatingDur&nbsp;:&nbsp;TimeSpan)&nbsp;config&nbsp;reservations&nbsp;({&nbsp;Date&nbsp;=&nbsp;d&nbsp;}&nbsp;<span style="color:blue;">as</span>&nbsp;r)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;contemporaneousReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x.Date.Subtract&nbsp;seatingDur&nbsp;&lt;&nbsp;d.Date)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;config&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;capacity&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;r.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Tables&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;Seq.sort&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingTables&nbsp;=&nbsp;Seq.deleteFirstsBy&nbsp;fits&nbsp;(Seq.sort&nbsp;tables)&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.exists&nbsp;(fits&nbsp;r)&nbsp;remainingTables</pre> </p> <p> While the function already considered <code>seatingDur</code>, the way it filtered <code>reservation</code> wasn't entirely correct. It passed all tests, though. </p> <h3 id="2411805a41764ba3b166612ffa392246"> Filter reservations based on seating duration <a href="#2411805a41764ba3b166612ffa392246" title="permalink">#</a> </h3> <p> The next test case I added made me write what I consider the right implementation, but I subsequently decided to add two more test cases just for confidence. Here's all of them: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;SecondSeatingsTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;TimeSpan,&nbsp;int&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime)&nbsp;list,&nbsp;(int&nbsp;*&nbsp;DateTime),&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2&nbsp;.hours, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2;2;4], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(4,&nbsp;d4)], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(3,&nbsp;d4.Add&nbsp;(2&nbsp;.hours)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.5&nbsp;.hours, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2;4;4], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d4);(1,&nbsp;d4.AddMinutes&nbsp;15.);(2,&nbsp;d4.Subtract&nbsp;(15&nbsp;.minutes))], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(3,&nbsp;d4.AddHours&nbsp;2.), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.5&nbsp;.hours, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2;4;4], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d4);(2,&nbsp;d4.Subtract&nbsp;(15&nbsp;.minutes))], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(3,&nbsp;d4.AddHours&nbsp;2.), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.5&nbsp;.hours, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[2;4;4], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[(2,&nbsp;d4);(1,&nbsp;d4.AddMinutes&nbsp;15.);(2,&nbsp;d4.Subtract&nbsp;(15&nbsp;.minutes))], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(3,&nbsp;d4.AddHours&nbsp;2.25), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> The new test cases use some more literal extensions: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Int32&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.minutes&nbsp;=&nbsp;TimeSpan.FromMinutes&nbsp;(float&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.hours&nbsp;=&nbsp;TimeSpan.FromHours&nbsp;(float&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.days&nbsp;=&nbsp;TimeSpan.FromDays&nbsp;(float&nbsp;x) <span style="color:blue;">type</span>&nbsp;Double&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;x.hours&nbsp;=&nbsp;TimeSpan.FromHours&nbsp;x</pre> </p> <p> I added a private <code>isContemporaneous</code> function to the code base and used it to filter the reservation to pass the tests: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;isContemporaneous &nbsp;&nbsp;&nbsp;&nbsp;(seatingDur&nbsp;:&nbsp;TimeSpan) &nbsp;&nbsp;&nbsp;&nbsp;(candidate&nbsp;:&nbsp;Reservation) &nbsp;&nbsp;&nbsp;&nbsp;(existing&nbsp;:&nbsp;Reservation)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;aSeatingBefore&nbsp;=&nbsp;candidate.Date.Subtract&nbsp;seatingDur &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;aSeatingAfter&nbsp;=&nbsp;candidate.Date.Add&nbsp;seatingDur &nbsp;&nbsp;&nbsp;&nbsp;aSeatingBefore&nbsp;&lt;&nbsp;existing.Date&nbsp;&amp;&amp;&nbsp;existing.Date&nbsp;&lt;&nbsp;aSeatingAfter <span style="color:blue;">let</span>&nbsp;canAccept&nbsp;(seatingDur&nbsp;:&nbsp;TimeSpan)&nbsp;config&nbsp;reservations&nbsp;r&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;contemporaneousReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(isContemporaneous&nbsp;seatingDur&nbsp;r)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;config&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;capacity&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;r.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Tables&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;Seq.sort&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingTables&nbsp;=&nbsp;Seq.deleteFirstsBy&nbsp;fits&nbsp;(Seq.sort&nbsp;tables)&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.exists&nbsp;(fits&nbsp;r)&nbsp;remainingTables</pre> </p> <p> I could have left the functionality of <code>isContemporaneous</code> inside of <code>canAccept</code>, but I found it just hard enough to get my head around that I preferred to put it in a named helper function. Checking that a value is in a range is in itself trivial, but for some reason, figuring out the limits of the range didn't come naturally to me. </p> <p> This version of <code>canAccept</code> only considered existing reservations if they in any way overlapped with the reservation in question. It passed all tests. It also seemed to me to be a satisfactory implementation of the <em>second seatings</em> scenario. </p> <h3 id="7b7ede53fafc4a25b6bbb3fe9bbb4f09"> Alternative table configurations <a href="#7b7ede53fafc4a25b6bbb3fe9bbb4f09" title="permalink">#</a> </h3> <p> This state of the kata introduces groups of tables that can be reserved individually, or combined. To support that, I changed the definition of <code>Table</code>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Table&nbsp;=&nbsp;Discrete&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;|&nbsp;Group&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;list</pre> </p> <p> A <code>Table</code> is now either a <code>Discrete</code> table that can't be combined, or a <code>Group</code> of tables that can either be reserved individually, or combined. </p> <p> I had to change the test-specific <code>table</code> function to behave like before. </p> <p> <pre><span style="color:blue;">let</span>&nbsp;table&nbsp;s&nbsp;=&nbsp;Discrete&nbsp;s</pre> </p> <p> Before this change to the <code>Table</code> type, all tables were implicitly <code>Discrete</code> tables. </p> <p> This enabled me to add the first test case: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;AlternativeTableConfigurationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Table&nbsp;list,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) [&lt;Theory;&nbsp;ClassData(typeof&lt;AlternativeTableConfigurationTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Alternative&nbsp;table&nbsp;configurations``&nbsp;(tables,&nbsp;rs,&nbsp;r,&nbsp;expected)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;res&nbsp;i&nbsp;=&nbsp;reserve&nbsp;(i,&nbsp;d4) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservations&nbsp;=&nbsp;List.map&nbsp;res&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;canAccept&nbsp;(1&nbsp;.days)&nbsp;(Tables&nbsp;tables)&nbsp;reservations&nbsp;(res&nbsp;r) &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> Like I did when I introduced the <code>seatingDur</code> argument, I deliberately chose a test case that didn't otherwise rock the boat too much. The same was the case now, so the only other change I had to make to pass all tests was to adjust the <code>fits</code> function: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;Table&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;fits&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Discrete&nbsp;seats&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;seats &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Group&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">true</span></pre> </p> <p> It's clearly not correct to return <code>true</code> for any <code>Group</code>, but it passed all tests. </p> <h3 id="413588fbf7524177a24171c3a25f4fe7"> Accept based on sum of table group <a href="#413588fbf7524177a24171c3a25f4fe7" title="permalink">#</a> </h3> <p> I wanted to edge a little closer to correctly handling the <code>Group</code> case, so I added a test case: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;AlternativeTableConfigurationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Table&nbsp;list,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;7, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> A restaurant with this table configuration can't accept a reservation for seven people, but because <code>fits</code> returned <code>true</code> for any <code>Group</code>, <code>canAccept</code> would return <code>true</code>. Since the test expected the result to be <code>false</code>, this caused the test to fail. </p> <p> Edging closer to correct behaviour, I adjusted <code>fits</code> again: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;Table&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;fits&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Discrete&nbsp;seats&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;seats &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Group&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;List.sum&nbsp;tables</pre> </p> <p> This was still not correct, because it removed an entire group of tables when <code>fits</code> returned <code>true</code>, but it passed all tests so far. </p> <h3 id="86a27564f5b846ef8dc3afdeadd613a0"> Accept reservation by combining two tables <a href="#86a27564f5b846ef8dc3afdeadd613a0" title="permalink">#</a> </h3> <p> I added another failing test: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;AlternativeTableConfigurationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Table&nbsp;list,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;7, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2;1], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>)</pre> </p> <p> The last test case failed because the existing reservations should only have reserved one of the tables in the group, but because of the way <code>fits</code> worked, the entire group was deleted by <code>Seq.deleteFirstsBy fits</code>. This made <code>canAccept</code> reject the four-person reservation. </p> <p> To be honest, this step was difficult for me. I should probably have found out how to make a smaller step. </p> <p> I wanted a function that would compare a <code>Reservation</code> to a <code>Table</code>, but unlike <code>Fits</code> return <code>None</code> if it decided to 'use' the table, or a <code>Some</code> value if it decided that it didn't need to use the entire table. This would enable me to pick only some of the tables from a <code>Group</code>, but still return a <code>Some</code> value with the rest of tables. </p> <p> I couldn't figure out an elegant way to do this with the existing <code>Seq</code> functionality, so I started to play around with something more specific. The implementation came accidentally as I was struggling to come up with something more general. As I was experimenting, all of a sudden, all tests passed! </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;Table&nbsp;-&gt;&nbsp;Table&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;allot&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Discrete&nbsp;seats&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;seats &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;None &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;Some&nbsp;(Discrete&nbsp;seats) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Group&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Some&nbsp;(Group&nbsp;tables) <span style="color:green;">//&nbsp;seq&lt;Table&gt;&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;seq&lt;Table&gt;</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;allocate&nbsp;(tables&nbsp;:&nbsp;Table&nbsp;seq)&nbsp;r&nbsp;=&nbsp;seq&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;found&nbsp;=&nbsp;<span style="color:blue;">false</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;e&nbsp;=&nbsp;tables.GetEnumerator&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">while</span>&nbsp;e.MoveNext&nbsp;()&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;found &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;<span style="color:blue;">yield</span>&nbsp;e.Current &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;allot&nbsp;r&nbsp;e.Current&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;found&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;<span style="color:blue;">true</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">yield</span>&nbsp;t &nbsp;&nbsp;&nbsp;&nbsp;} <span style="color:blue;">let</span>&nbsp;canAccept&nbsp;(seatingDur&nbsp;:&nbsp;TimeSpan)&nbsp;config&nbsp;reservations&nbsp;r&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;contemporaneousReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(isContemporaneous&nbsp;seatingDur&nbsp;r)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;config&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;capacity&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;r.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Tables&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;Seq.sort&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingTables&nbsp;=&nbsp;Seq.fold&nbsp;allocate&nbsp;(Seq.sort&nbsp;tables)&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.exists&nbsp;(fits&nbsp;r)&nbsp;remainingTables</pre> </p> <p> I wasn't too happy with the implementation, which I found (and still find) too complicated. This was, however, the first time I've done this part of the kata (in any language), so I wasn't sure where this was going. </p> <p> The <code>allocate</code> function finds and allocates one of its input tables to a reservation. It does that by <em>not</em> yielding the first table it finds that can accommodate the reservation. Don't hurt your head too much with the code in this version, because there's plenty of cases that it incorrectly handles. It's full of bugs. Still, it passed all tests. </p> <h3 id="34ef030a71e24dfabc3bd2ff0d3bdf4c"> Reject when group has been reduced <a href="#34ef030a71e24dfabc3bd2ff0d3bdf4c" title="permalink">#</a> </h3> <p> The implementation was wrong because the <code>allot</code> function would just keep returning a <code>Group</code> without consuming it. This would imply that <code>canAccept</code> would use it more than once, which was wrong, so I added a test case: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;AlternativeTableConfigurationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Table&nbsp;list,&nbsp;int&nbsp;list,&nbsp;int,&nbsp;bool&gt;&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;7, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2;1], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Discrete&nbsp;4;&nbsp;Discrete&nbsp;1;&nbsp;Discrete&nbsp;2;&nbsp;Group&nbsp;[2;2;2]], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[3;1;2;1;4], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">false</span>)</pre> </p> <p> Given the existing reservations, this restaurant is effectively sold out that day. All the <code>Discrete</code> tables are reserved, and the last two reservations for one and four effectively consumes the <code>Group</code>. The latest test case expected <code>canAccept</code> to return <code>false</code>, but it returned <code>true</code>. Since I was following test-driven development, I expected that. </p> <h3 id="af5554a2e5b9430cb6ec5d1aaa10ec44"> Consume <a href="#af5554a2e5b9430cb6ec5d1aaa10ec44" title="permalink">#</a> </h3> <p> I needed a function that would consume from a <code>Group</code> of tables and return the remaining tables from that group; that is, the tables not consumed. I've already <a href="/2019/12/16/zone-of-ceremony">discussed this function in a different context</a>. </p> <p> <pre><span style="color:green;">//&nbsp;int&nbsp;-&gt;&nbsp;seq&lt;int&gt;&nbsp;-&gt;&nbsp;seq&lt;int&gt;</span> <span style="color:blue;">let</span>&nbsp;consume&nbsp;quantity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;go&nbsp;(acc,&nbsp;xs)&nbsp;x&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;quantity&nbsp;&lt;=&nbsp;acc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;(acc,&nbsp;Seq.append&nbsp;xs&nbsp;(Seq.singleton&nbsp;x)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;(acc&nbsp;+&nbsp;x,&nbsp;xs) &nbsp;&nbsp;&nbsp;&nbsp;Seq.fold&nbsp;go&nbsp;(0,&nbsp;Seq.empty)&nbsp;&gt;&gt;&nbsp;snd</pre> </p> <p> I put this function in my own <code>Seq</code> module. It consumes values from the left until the sum is greater than or equal to the desired <code>quantity</code>. It then returns the rest of the sequence: </p> <p> <pre>&gt; consume 1 [1;2;3];; val it : seq&lt;int&gt; = seq [2; 3] &gt; consume 2 [1;2;3];; val it : seq&lt;int&gt; = seq [3] &gt; consume 3 [1;2;3];; val it : seq&lt;int&gt; = seq [3] &gt; consume 4 [1;2;3];; val it : seq&lt;int&gt; = seq []</pre> </p> <p> The first example consumes only the leading <code>1</code>, while both the second and the third example consumes both <code>1</code> and <code>2</code> because the sum of those values is <code>3</code>, and the requested quantity is <code>2</code> and <code>3</code>, respectively. The fourth example consumes all elements because the requested quantity is <code>4</code>, and you need both <code>1</code>, <code>2</code>, and <code>3</code> before the sum is large enough. You have to pick strictly from the left, so you can't decide to just take the elements <code>1</code> and <code>3</code>. </p> <h3 id="b9e29990b0fd4a5ab5e7c8812be6b07b"> Consuming tables from a group <a href="#b9e29990b0fd4a5ab5e7c8812be6b07b" title="permalink">#</a> </h3> <p> I could now use my <code>Seq.consume</code> function to improve <code>allot</code>: </p> <p> <pre><span style="color:green;">//&nbsp;Reservation&nbsp;-&gt;&nbsp;Table&nbsp;-&gt;&nbsp;Table&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;allot&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Discrete&nbsp;seats&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;seats &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;None &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;Some&nbsp;(Discrete&nbsp;seats) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Group&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;tables&nbsp;|&gt;&nbsp;Seq.consume&nbsp;r.Quantity&nbsp;|&gt;&nbsp;Seq.toList&nbsp;|&gt;&nbsp;Group&nbsp;|&gt;&nbsp;Some</pre> </p> <p> It handles the <code>Group</code> case by consuming the reservation <code>Quantity</code> and then returning a <code>Some Group</code> with the remaining tables. </p> <p> It also turned out that sorting the reservations wasn't appropriate, mainly because it's not entirely clear how to sort a list with elements of a discriminated union. My final implementation of <code>canAccept</code> was this: </p> <p> <pre><span style="color:green;">//&nbsp;TimeSpan&nbsp;-&gt;&nbsp;TableConfiguration&nbsp;-&gt;&nbsp;seq&lt;Reservation&gt;&nbsp;-&gt;&nbsp;Reservation&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;canAccept&nbsp;seatingDur&nbsp;config&nbsp;reservations&nbsp;r&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;contemporaneousReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.filter&nbsp;(isContemporaneous&nbsp;seatingDur&nbsp;r)&nbsp;reservations &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;config&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;capacity&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reservedSeats&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservedSeats&nbsp;+&nbsp;r.Quantity&nbsp;&lt;=&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Tables&nbsp;tables&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingTables&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.fold&nbsp;allocate&nbsp;(Seq.ofList&nbsp;tables)&nbsp;contemporaneousReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Seq.exists&nbsp;(fits&nbsp;r)&nbsp;remainingTables</pre> </p> <p> Nothing much has changed - only that neither reservations nor tables are now sorted. It passes all tests. </p> <h3 id="fcba96687bac4cb3b35cbaef1b8232db"> Retrospective <a href="#fcba96687bac4cb3b35cbaef1b8232db" title="permalink">#</a> </h3> <p> I must admit that I ran out of steam towards the end. It's possible that there's some edge cases I didn't think of. I'd probably feel more confident of the final implementation if I'd used property-based testing instead of <a href="https://twitter.com/tastapod/status/1157996947581652993">Example-Guided Development</a>. </p> <p> I also took some unfortunate turns along the way. Early in the kata, I could easily implement <code>canAccept</code> with functionality from the <code>Seq</code> module. This meant that the function could take a <code>seq&lt;Reservation&gt;</code> as an input argument. I'm always inclined to follow <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a> and be liberal with what I accept. I thought that being able to accept any <code>seq&lt;Reservation&gt;</code> was a good design decision. It might have been if I'd been publishing a reusable library, but it made things more awkward. </p> <p> I'm also not sure that I chose to model the table layouts in the best way. For example, I currently can't handle a scenario with both bar seating and individual tables. I think I should have made <code>Communal</code> a case of <code>Table</code>. This would have enabled me to model layouts with several communal tables combined with discrete tables, and even groups of tables. </p> <p> In general, my solution seems too complicated, but I don't see an easy fix. Often, if I work some more with the problem, insight arrives. It usually arrives when you least need it, so I thought it better to let the problem rest for a while. I can always return to it when I feel that I have a fresh approach. </p> <h3 id="d19eccbe58924d8a96a1acab89f9bbaa"> Summary <a href="#d19eccbe58924d8a96a1acab89f9bbaa" title="permalink">#</a> </h3> <p> This article walks you through my first F# attempt at the <em>Maître d' kata</em>. The repository is available on GitHub. </p> <p> I'm not entirely happy with how it turned out, but I don't consider it an utter failure either. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="61ab9d61eb6346d39b4fcf94d094a6ef"> <div class="comment-author">Ghillie Dhu</div> <div class="comment-content"> <p> You could leverage library functions and avoid mutability like so: <pre><span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;bool)&nbsp;-&gt;&nbsp;seq&lt;&#39;a&gt;&nbsp;-&gt;&nbsp;seq&lt;&#39;a&gt;</span> <span style="color:blue;">let</span>&nbsp;deleteFirstBy&nbsp;pred&nbsp;(xs&nbsp;:&nbsp;_&nbsp;seq)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;Seq.tryFindIndex pred xs&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-></span>&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;n&nbsp;<span style="color:blue;">-></span>&nbsp;Seq.concat&nbsp;(Seq.take&nbsp;(n-1)&nbsp;xs)&nbsp;(Seq.skip&nbsp;n&nbsp;xs)</pre> </p> </div> <div class="comment-date">2020-04-28 19:02 UTC</div> </div> <div class="comment" id="6fe45d2631bf410285b5403b1da737df"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Ghillie, thank you for writing. It looks like you're on the right track, but I think you have to write the function like this: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;deleteFirstBy&nbsp;pred&nbsp;(xs&nbsp;:&nbsp;_&nbsp;seq)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;Seq.tryFindIndex&nbsp;pred&nbsp;xs&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;xs &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;n&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Seq.append&nbsp;(Seq.take&nbsp;(n-1)&nbsp;xs)&nbsp;(Seq.skip&nbsp;n&nbsp;xs)</pre> </p> </div> <div class="comment-date">2020-04-28 19:21 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Unit bias against collections https://blog.ploeh.dk/2020/04/20/unit-bias-against-collections 2020-04-20T06:27:00+00:00 Mark Seemann <div id="post"> <p> <em>How do you get the value out of a collection? Mu. Which value?</em> </p> <p> The other day I was looking for documentation on how to write automated tests with a self-hosted ASP.NET Core 3 web application. I've <a href="/outside-in-tdd">done this numerous times with previous versions of the framework</a>, but ASP.NET Core 3 is new, so I wanted to learn how I'm supposed to do it this year. I found <a href="https://docs.microsoft.com/aspnet/core/test/integration-tests">official documentation</a> that helped me figure it out. </p> <p> One of the code examples in that article displays a motif that I keep encountering. It displays behaviour close enough to <em>unit bias</em> that I consider it reasonable to use that label. Unit bias is the cognitive tendency to consider a <em>unit</em> of something the preferred amount. Our brains don't like fractions, and they don't like multiples either. </p> <h3 id="c9468b526bcb444a99680d8a6a5deb29"> Unit bias in action <a href="#c9468b526bcb444a99680d8a6a5deb29" title="permalink">#</a> </h3> <p> The sample code in the ASP.NET Core documentation differs in the type of dependency it looks for, but is otherwise identical to this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">SingleOrDefault</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>.ServiceType&nbsp;<span style="font-weight:bold;color:#74531f;">==</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">IReservationsRepository</span>)); <span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;!=&nbsp;<span style="color:blue;">null</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">Remove</span>(<span style="font-weight:bold;color:#1f377f;">descriptor</span>); }</pre> </p> <p> The goal is to enable an automated integration test to <a href="/2019/04/01/an-example-of-state-based-testing-in-c">run against a Fake database</a> instead of an actual relational database. My production <a href="/2011/07/28/CompositionRoot">Composition Root</a> registers an implementation of <code>IReservationsRepository</code> that communicates with an actual database. In an automated integration test, I'd like to unregister the existing dependency and replace it with <a href="http://xunitpatterns.com/Fake%20Object.html">a Fake</a>. Here's the code in context: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">RestaurantApiFactory</span>&nbsp;:&nbsp;<span style="color:#2b91af;">WebApplicationFactory</span>&lt;<span style="color:#2b91af;">Startup</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">protected</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="font-weight:bold;color:#74531f;">ConfigureWebHost</span>(<span style="color:#2b91af;">IWebHostBuilder</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">builder</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">builder</span>&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(<span style="font-weight:bold;color:#1f377f;">builder</span>)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">builder</span>.<span style="font-weight:bold;color:#74531f;">ConfigureServices</span>(<span style="font-weight:bold;color:#1f377f;">services</span>&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">SingleOrDefault</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>.ServiceType&nbsp;<span style="font-weight:bold;color:#74531f;">==</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">IReservationsRepository</span>)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;!=&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">Remove</span>(<span style="font-weight:bold;color:#1f377f;">descriptor</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">AddSingleton</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> It works as intended, so what's the problem? </p> <h3 id="d84651bde63944e9bb04873f51a58085"> How do I get the value out of my collection? <a href="#d84651bde63944e9bb04873f51a58085" title="permalink">#</a> </h3> <p> The problem is that it's fragile. What happens if there's more than one registration of <code>IReservationsRepository</code>? </p> <p> This happens: </p> <p> <pre>System.InvalidOperationException : Sequence contains more than one matching element</pre> </p> <p> This is a completely avoidable error, stemming from unit bias. </p> <p> A large proportion of programmers I meet seem to be fundamentally uncomfortable with thinking in multitudes. They subconsciously prefer thinking in procedures and algorithms that work on a single object. The programmer who wrote the above call to <a href="https://docs.microsoft.com/dotnet/api/system.linq.enumerable.singleordefault">SingleOrDefault</a> exhibits behaviour putting him or her in that category. </p> <p> This is nothing but a specific instantiation of a more general programmer bias: <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">How do I get the value out of the monad?</a> </p> <p> As usual, the answer is <a href="https://en.wikipedia.org/wiki/Mu_(negative)">mu</a>. You don't. The question borders on the nonsensical. <em>How do I get the value out of my collection?</em> Which value? The first? The last? Some arbitrary value at an unknown position? Which value do you want if the collection is empty? Which value do you want if there's more than one that fits a predicate? </p> <p> If you can answer such questions, you <em>can</em> get 'the' value out of a collection, but often, you can't. In the current example, the code doesn't handle multiple <code>IReservationsRepository</code> registrations. </p> <p> It easily could, though. </p> <h3 id="568f351c446f4e33a158eccdfbb461e6"> Inject the behaviour into the collection <a href="#568f351c446f4e33a158eccdfbb461e6" title="permalink">#</a> </h3> <p> The best answer to the question of <em>how to get the value out of the monad</em> (in this case, <em>the collection</em>) is that you don't. Instead, you inject the desired behaviour into it. </p> <p> In this case, the desired behaviour is to remove a <code>descriptor</code>. The monad in question is the collection of <code>services</code>. What does that mean in practice? </p> <p> A first attempt might be something like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptors</span>&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span> &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">Where</span>(<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>.ServiceType&nbsp;<span style="font-weight:bold;color:#74531f;">==</span>&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">IReservationsRepository</span>)); <span style="font-weight:bold;color:#8f08c4;">foreach</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">in</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptors</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">Remove</span>(<span style="font-weight:bold;color:#1f377f;">descriptor</span>);</pre> </p> <p> Unfortunately, this doesn't quite work: </p> <p> <pre>System.InvalidOperationException : Collection was modified; enumeration operation may not execute.</pre> </p> <p> This happens because <code>descriptors</code> is a lazily evaluated <a href="https://en.wikipedia.org/wiki/Iterator_pattern">Iterator</a> over <code>services</code>, and you're not allowed to remove elements from a collection while you enumerate it. It could lead to bugs if you could. </p> <p> That problem is easily solved. Just copy the selected <code>descriptors</code> to an array or list: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptors</span>&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span> &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">Where</span>(<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>.ServiceType&nbsp;<span style="font-weight:bold;color:#74531f;">==</span>&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">IReservationsRepository</span>)) &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">ToList</span>(); <span style="font-weight:bold;color:#8f08c4;">foreach</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptor</span>&nbsp;<span style="font-weight:bold;color:#8f08c4;">in</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">descriptors</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">Remove</span>(<span style="font-weight:bold;color:#1f377f;">descriptor</span>);</pre> </p> <p> This achieves the desired outcome regardless of the number of matches to the predicate. This is a more robust solution, and it requires the same amount of code. </p> <p> You can stop there, since the code now works, but if you truly want to <em>inject the behaviour into the collection</em>, you're not quite done yet. </p> <p> But you're close. All you have to do is this: </p> <p> <pre><span style="font-weight:bold;color:#1f377f;">services</span> &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">Where</span>(<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">d</span>.ServiceType&nbsp;<span style="font-weight:bold;color:#74531f;">==</span>&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">IReservationsRepository</span>)) &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">ToList</span>() &nbsp;&nbsp;&nbsp;&nbsp;.<span style="font-weight:bold;color:#74531f;">ForEach</span>(<span style="font-weight:bold;color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">services</span>.<span style="font-weight:bold;color:#74531f;">Remove</span>(<span style="font-weight:bold;color:#1f377f;">d</span>));</pre> </p> <p> Notice how this statement never produces an output. Instead, you 'inject' the call to <code>services.Remove</code> into the list, using the <a href="https://docs.microsoft.com/dotnet/api/system.collections.generic.list-1.foreach">ForEach</a> method, which then mutates the <code>services</code> collection. </p> <p> Whether you prefer the version that uses the <code>foreach</code> keyword or the version that uses <code>List&lt;T&gt;.ForEach</code> doesn't matter. What matters is that you don't use the <a href="https://en.wikipedia.org/wiki/Partial_function">partial</a> <code>SingleOrDefault</code> function. </p> <h3 id="cd80281860ed4b8299278cdaa2e65709"> Conclusion <a href="#cd80281860ed4b8299278cdaa2e65709" title="permalink">#</a> </h3> <p> It's a common code smell when programmers try to extract a single value from a collection. Sometimes it's appropriate, but there are several edge cases you should be prepared to address. What should happen if the collection is empty? What should happen if the collection contains many elements? What should happen if the collection is infinite? (I didn't address that in this article.) </p> <p> You can often both simplify your code and make it more robust by staying 'in' the collection, so to speak. Let the desired behaviour apply to all appropriate elements of the collection. </p> <p> Don't be biased against collections. </p> </div> <hr> <div id="comments"> <h2 id="comments-header"> Comments </h2> <div class="comment" id="e6675033a3a9dc8a21c64650dff91b8432a9a151"> <div class="comment-author">Julius H</div> <div class="comment-content"> <p>I concur that often the first element of a collection is picked without thinking. Anecdotally, I experienced a system that would not work if set up freshly because in some places there was no consideration for empty collections. (The testing environment always contained some data)</p> <p> Yet I would reverse the last change (towards <code>.ForEach</code>). For one, because (to my biased eye) it <i>looks</i> side effect free but isn't. And then it does add value compared to a forech loop, also both solutions are needlessy inefficient. If you want to avoid the foreach, go for the <code>RemoveAll()</code> method (also present on List&lt;T&gt;): </p> <pre> services.<span style="font-weight:bold;color:#74531f;">RemoveAll</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;(); </pre> </p> </div> <div class="comment-date">2020-04-29 8:52 UTC</div> </div> <div class="comment" id="99baee4e923b471891eef04b499b64a1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Julius, thank you for writing. Yes, I agree that in C# it's more <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> to use <code>foreach</code>. </p> <p> How would using <code>RemoveAll</code> work? Isn't that going to remove the entries from the <code>List</code> instead of from <code>services</code>? </p> </div> <div class="comment-date">2020-04-29 10:49 UTC</div> </div> <div class="comment" id="17b7d3a21cff5b6585d6d2fa0f63f1539f9bce3e"> <div class="comment-author">Julius H</div> <div class="comment-content"> <p> Hi Mark,<br />As you&quot;re calling <a href="https://docs.microsoft.com/en-us/dotnet/api/microsoft.extensions.dependencyinjection.extensions.servicecollectiondescriptorextensions.removeall?view=dotnet-plat-ext-3.1"><code>IServiceCollection.RemoveAll()</code></a>, it will remove it from the collection. I tried it, and to me it seems to be working. (As long as you are not copying the services into a list beforehand) </p> <p> But to your main point, I remember when I wrote <code>.Single()</code> once, and years later I got a bug report because of it. I see two approaches there: Fail as fast and hard as possible or check just as much as needed for the moment. Considering TDD in the former approach, one would need to write a lot of test code for scenarios, that should never happen to verify the exceptions happen. Still, in the latter approach, a subtle bug could stay in the system for quite some time... What do you prefer? </p> </div> <div class="comment-date">2020-05-01 18:07 UTC</div> </div> <div class="comment" id="3eed1b53957e4abf8b92c83c15e63ec6"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Julius, thank you for writing. It turns out that <code>RemoveAll</code> are <a href="https://docs.microsoft.com/dotnet/api/microsoft.extensions.dependencyinjection.extensions.servicecollectiondescriptorextensions.removeall">a couple of extension methods on <code>IServiceCollection</code></a>. One has to import <code>Microsoft.Extensions.DependencyInjection.Extensions</code> with a <code>using</code> directive before one can use them. I didn't know about these methods, but I agree that they seem to do their job. Thank you for the tip. </p> <p> As for your question, my preference is for robustness. In my experience, there's rarely such a thing as a scenario that never happens. If the code allows something to happen, it'll likely happen sooner or later. Code changes, so even if you've analysed that some combination of input isn't possible today, a colleague may change the situation tomorrow. It pays to write that extra unit test or two to make sure that encapsulation isn't broken. </p> <p> This is also one of the reasons I'm fond of <a href="/property-based-testing-intro">property-based testing</a>. You automatically get coverage of all sorts of boundary conditions you'd normally not think about testing. </p> </div> <div class="comment-date">2020-05-02 9:12 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Curb code rot with thresholds https://blog.ploeh.dk/2020/04/13/curb-code-rot-with-thresholds 2020-04-13T08:43:00+00:00 Mark Seemann <div id="post"> <p> <em>Code bases deteriorate unless you actively prevent it. Institute some limits that encourage developers to clean up.</em> </p> <p> From time to time I manage to draw the ire of people, with articles such as <a href="/2019/11/04/the-80-24-rule">The 80/24 rule</a> or <a href="/2019/12/09/put-cyclomatic-complexity-to-good-use">Put cyclomatic complexity to good use</a>. I can understand why. These articles suggest specific constraints to which people should consider consenting. <em>Don't write code wider than 80 characters. Don't write code with a <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a> higher than 7</em>. </p> <p> It makes people uncomfortable. </p> <h3 id="6c5e4b0817f742bf9ecc0a9050ecb7b5"> Sophistication <a href="#6c5e4b0817f742bf9ecc0a9050ecb7b5" title="permalink">#</a> </h3> <p> I hope that regular readers understand that I'm a more sophisticated thinker than some of my texts may suggest. I deliberately simplify my points. </p> <p> I do this to make the text more readable. I also aspire to present sufficient arguments, and enough context, that a charitable reader will understand that everything I write should be taken as food for thought rather than gospel. </p> <p> Consider a sentence like the above: <em>I deliberately simplify my points</em>. That sentence, in itself, is an example of deliberate simplification. In reality, I don't <em>always</em> simplify my points. Perhaps sometimes I simplify, but it's not <em>deliberate</em>. I could have written: <em>I often deliberately simplify some of my points</em>. Notice the extra <a href="https://en.wikipedia.org/wiki/Hedge_(linguistics)">hedge words</a>. Imagine an entire text written like that. It would be less readable. </p> <p> I could hedge my words when I write articles, but I don't. I believe that a text that states its points as clearly as possible is easier to understand for any careful reader. I also believe that hedging my language will not prevent casual readers from misunderstanding what I had in mind. </p> <h3 id="a5da32d0580d4a0f8c19069c57c2a335"> Archetypes <a href="#a5da32d0580d4a0f8c19069c57c2a335" title="permalink">#</a> </h3> <p> Why do I suggest hard limits on line width, cyclomatic complexity, and so on? </p> <p> In light of the above, realise that the limits I offer are suggestions. A number like 80 characters isn't a hard limit. It's a representation of an idea; a token. The same is true for <a href="https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two">the magic number seven, plus or minus two</a>. That too, represents an idea - the idea that human short-term memory is limited, and that this impacts our ability to read and understand code. </p> <p> The number seven serves as an archetype. It's a proxy for a more complex idea. It's a simplification that, hopefully, makes it easier to follow the plot. </p> <p> <em>Each method should have a maximum cyclomatic complexity of seven.</em> That's easier to understand than <em>each method should have a maximum cyclomatic complexity small enough that it fits within the cognitive limits of the human brain's short-term memory</em>. </p> <p> I've noticed that a subset of the developer population is quite literal-minded. If I declare: <em>don't write code wider than 80 characters</em> they're happy if they agree, and infuriated if they don't. </p> <p> If you've been paying attention, you now understand that this isn't about the number <em>80</em>, or <em>24</em>, or <em>7</em>. It's about instituting useful quantitative guidance. The actual number is less important. </p> <p> I have reasons to prefer those specific values. I've already motivated them in previous articles. I'm not, though, obdurately attached to those particular numbers. I'd rather work with a team that has agreed to a 120-character maximum width than with a team that follows no policy. </p> <h3 id="8b7d0a456f6f4bdca9ea2814a67144fc"> How code rots <a href="#8b7d0a456f6f4bdca9ea2814a67144fc" title="permalink">#</a> </h3> <p> No-one deliberately decides to write legacy code. Code bases gradually deteriorate. </p> <p> Here's another deliberate simplification: code gradually becomes more complicated because each change seems small, and no-one pays attention to the overall quality. It doesn't happen overnight, but one day you realise that you've developed a legacy code base. When that happens, it's too late to do anything about it. </p> <p> <img src="/content/binary/gradual-increase-of-complexity.png" alt="Line chart showing increasing complexity as time passes."> </p> <p> At the beginning, a method has low complexity, but as you fix defects and add features, the complexity increases. If you don't pay attention to cyclomatic complexity, you pass <em>7</em> without noticing it. You pass <em>10</em> without noticing it. You pass <em>15</em> and <em>20</em> without noticing it. </p> <p> One day you discover that you have a problem - not because you finally look at a metric, but because the code has now become so complicated that everyone notices. Alas, now it's too late to do anything about it. </p> <p> <a href="https://en.wikipedia.org/wiki/Software_rot">Code rot</a> sets in a little at a time; it works like <a href="https://en.wikipedia.org/wiki/Boiling_frog">boiling the proverbial frog</a>. </p> <h3 id="4fd818d9459548a7863f38ebdd6c40f8"> Thresholds <a href="#4fd818d9459548a7863f38ebdd6c40f8" title="permalink">#</a> </h3> <p> Agreeing on a threshold can help curb code rot. Institute a rule and monitor a metric. For example, you could agree to keep an eye on cyclomatic complexity. If it exceeds <em>7</em>, you reject the change. </p> <p> <img src="/content/binary/development-of-complexity-guarded-by-threshold.png" alt="Line chart showing how complexity is curbed by a threshold of 7."> </p> <p> Such rules work because they can be used to counteract gradual decay. It's not the specific value <em>7</em> that contributes to better <a href="/2019/03/04/code-quality-is-not-software-quality">code quality</a>; it's the automatic activation of a rule based on a threshold. If you decide that the threshold should be <em>10</em> instead, that'll also make a difference. </p> <p> Notice that the above diagram suggests that exceeding the threshold is still possible. Rules are in the way if you must rigidly obey them. Situations arise where breaking a rule is the best response. Once you've responded to the situation, however, find a way to bring the offending code back in line. Once a threshold is exceeded, you don't get any further warnings, and there's a risk that that particular code will gradually decay. </p> <h3 id="431b3a30bb6448b6b34c06a169b6fefb"> What you measure is what you get <a href="#431b3a30bb6448b6b34c06a169b6fefb" title="permalink">#</a> </h3> <p> You could automate the process. Imagine running cyclomatic complexity analysis as part of a Continuous Integration build and rejecting changes that exceed a threshold. This is, in a way, a deliberate attempt to hack the management effect where you get what you measure. With emphasis on a metric like cyclomatic complexity, developers will pay attention to it. </p> <p> Be aware, however, of <a href="https://en.wikipedia.org/wiki/Goodhart%27s_law">Goodhart's law</a> and <a href="https://en.wikipedia.org/wiki/Unintended_consequences">the law of unintended consequences</a>. Just as <a href="/2015/11/16/code-coverage-is-a-useless-target-measure">code coverage is a useless target measure</a>, you have to be careful when you institute hard rules. </p> <p> I've had success with introducing threshold rules because they increase awareness. It can help a technical leader shift emphasis to the qualities that he or she wishes to improve. Once the team's mindset has changed, the rule itself becomes redundant. </p> <p> I'm reminded of <a href="https://en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition">the Dreyfus model of skill acquisition</a>. Rules make great training wheels. Once you become proficient, the rules are no longer required. They may even be in your way. When that happens, get rid of them. </p> <h3 id="97ad7bcd44644a05b78c75e82a3c4ad7"> Conclusion <a href="#97ad7bcd44644a05b78c75e82a3c4ad7" title="permalink">#</a> </h3> <p> Code deteriorates gradually, when you aren't looking. Instituting rules that make you pay attention can combat code rot. Using thresholds to activate your attention can be an effective countermeasure. The specific value of the threshold is less important. </p> <p> In this article, I've mostly used cyclomatic complexity as an example of a metric where a threshold could be useful. Another example is line width; don't exceed 80 characters. Or line height: methods shouldn't exceed 24 lines of code. Those are examples. If you agree that keeping an eye on a metric would be useful, but you disagree with the threshold I suggest, pick a value that suits you better. </p> <p> It's not the specific threshold value that improves your code; paying attention does. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Repeatable execution in C# https://blog.ploeh.dk/2020/04/06/repeatable-execution-in-c 2020-04-06T07:46:00+00:00 Mark Seemann <div id="post"> <p> <em>A C# example of Goldilogs.</em> </p> <p> This article is part of <a href="/2020/03/23/repeatable-execution">a series of articles about repeatable execution</a>. The introductory article argued that if you've logged the impure actions that a system made, you have enough information to reproduce what happened. The <a href="/2020/03/30/repeatable-execution-in-haskell">previous article</a> verified that for the example scenario, the impure actions were limited to reading the current time and interacting with the application database. </p> <p> This article shows how to implement equivalent functionality in C#. You should be able to extrapolate from this to other object-oriented programming languages. </p> <p> The code is <a href="https://github.com/ploeh/reservation-api-slice-csharp">available on GitHub</a>. </p> <h3 id="338d728ddae0410dbb7fefccd6781f6f"> Impure actions <a href="#338d728ddae0410dbb7fefccd6781f6f" title="permalink">#</a> </h3> <p> In the previous article I <a href="/2017/07/10/pure-interactions">modelled impure actions as free monads</a>. In C#, it'd be more <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> to <a href="/2017/01/27/from-dependency-injection-to-dependency-rejection">use Dependency Injection</a>. Model each impure interaction as an interface. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IClock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>(); }</pre> </p> <p> The demo code demonstrates a single feature of a REST API and it only requires a single method on this interface to work. Following the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a> <blockquote> <p> "clients [...] own the abstract interfaces" </p> <footer><cite><a href="http://amzn.to/19W4JHk">Agile Principles, Patterns, and Practices</a>, chapter 11</cite></footer> </blockquote> This interface only defines a single method, because that's all the client code requires. </p> <p> Likewise, the client code only needs two methods to interact with the database: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="font-weight:bold;color:#74531f;">ReadReservations</span>(<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">date</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="font-weight:bold;color:#74531f;">Create</span>(<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>); }</pre> </p> <p> In the Haskell example code base, I also implemented <code>GET</code> for <code>/reservations</code>, but I forgot to do that here. There's only two methods on the interface: one to query the database, and one to create a new row. </p> <h3 id="63f39b6964f74a9c847913f4e1b5359c"> Receive a reservation <a href="#63f39b6964f74a9c847913f4e1b5359c" title="permalink">#</a> </h3> <p> The central feature of the service is to receive and handle an HTTP POST request, as described in the introductory article. When a document arrives it triggers a series of non-trivial work: <ol> <li>The service validates the input data. Among other things, it checks that the reservation is in the future. It uses <code>GetCurrentDateTime</code> for this.</li> <li>It queries the database for existing reservations. It uses <code>ReadReservations</code> for this.</li> <li>It uses complex business logic to determine whether to accept the reservation. This essentially implements the <a href="/2020/01/27/the-maitre-d-kata">Ma&icirc;tre d' kata</a>.</li> <li>If it accepts the reservation, it stores it. It uses <code>Create</code> for this.</li> </ol> These steps manifest as this Controller method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;<span style="font-weight:bold;color:#74531f;">Post</span>(<span style="color:#2b91af;">ReservationDto</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">dto</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(!<span style="color:#2b91af;">DateTime</span>.<span style="color:#74531f;">TryParse</span>(<span style="font-weight:bold;color:#1f377f;">dto</span>.Date,&nbsp;<span style="color:blue;">out</span>&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:blue;">_</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#74531f;">BadRequest</span>(<span style="color:#a31515;">$&quot;Invalid&nbsp;date:&nbsp;</span>{<span style="font-weight:bold;color:#1f377f;">dto</span>.Date}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>&nbsp;=&nbsp;<span style="color:#2b91af;">Mapper</span>.<span style="color:#74531f;">Map</span>(<span style="font-weight:bold;color:#1f377f;">dto</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">reservation</span>.Date&nbsp;<span style="font-weight:bold;color:#74531f;">&lt;</span>&nbsp;Clock.<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#74531f;">BadRequest</span>(<span style="color:#a31515;">$&quot;Invalid&nbsp;date:&nbsp;</span>{<span style="font-weight:bold;color:#1f377f;">reservation</span>.Date}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservations</span>&nbsp;=&nbsp;Repository.<span style="font-weight:bold;color:#74531f;">ReadReservations</span>(<span style="font-weight:bold;color:#1f377f;">reservation</span>.Date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">accepted</span>&nbsp;=&nbsp;maîtreD.<span style="font-weight:bold;color:#74531f;">CanAccept</span>(<span style="font-weight:bold;color:#1f377f;">reservations</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(!<span style="font-weight:bold;color:#1f377f;">accepted</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#74531f;">StatusCode</span>(<span style="color:#2b91af;">StatusCodes</span>.Status500InternalServerError,&nbsp;<span style="color:#a31515;">&quot;Couldn&#39;t&nbsp;accept.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Repository.<span style="font-weight:bold;color:#74531f;">Create</span>(<span style="font-weight:bold;color:#1f377f;">reservation</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#74531f;">Ok</span>(); }</pre> </p> <p> <code>Clock</code> and <code>Repository</code> are injected dependencies, and <code>maîtreD</code> is an object that implements the decision logic as the <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a> <code>CanAccept</code> function. </p> <h3 id="c7042ff0e8c246cc8a9f12e656a4bf54"> Composition <a href="#c7042ff0e8c246cc8a9f12e656a4bf54" title="permalink">#</a> </h3> <p> The <code>Post</code> method is defined on a class called <code>ReservationsController</code> with these dependencies: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">seatingDuration</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IReadOnlyCollection</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">tables</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">repository</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IClock</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">clock</span>)</pre> </p> <p> The <code>seatingDuration</code> and <code>tables</code> arguments are <a href="/2012/07/02/PrimitiveDependencies">primitive dependencies</a> used to configure the <code>maîtreD</code> object. I could also have injected <code>maîtreD</code> as a <a href="/2012/08/31/ConcreteDependencies">concrete dependency</a>, but I decided against that for no particular reason. </p> <p> There's no logging dependency, but the system still logs. Like in the previous example, logging is a cross-cutting concern and exclusively addressed through composition: </p> <p> <pre><span style="font-weight:bold;color:#8f08c4;">if</span>&nbsp;(<span style="font-weight:bold;color:#1f377f;">controllerType</span>&nbsp;<span style="font-weight:bold;color:#74531f;">==</span>&nbsp;<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">ReservationsController</span>)) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">l</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ScopedLog</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FileLog</span>(LogFile)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">controller</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SeatingDuration, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tables, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LogReservationsRepository</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>(ConnectionString), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">l</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LogClock</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">l</span>)); &nbsp;&nbsp;&nbsp;&nbsp;Logs.<span style="font-weight:bold;color:#74531f;">AddOrUpdate</span>(<span style="font-weight:bold;color:#1f377f;">controller</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">l</span>,&nbsp;(<span style="font-weight:bold;color:#1f377f;">_</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">x</span>)&nbsp;=&gt;&nbsp;<span style="font-weight:bold;color:#1f377f;">x</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">controller</span>; }</pre> </p> <p> Each dependency is wrapped by a logger. We'll return to that in a minute, but consider first the actual implementations. </p> <h3 id="40cf79001f6f4982ba694efe324cb2b1"> Using the system clock <a href="#40cf79001f6f4982ba694efe324cb2b1" title="permalink">#</a> </h3> <p> Using the system clock is easy: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IClock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This implementation of <code>IClock</code> simply delegates to <code>DateTime.Now</code>. Again, no logging service is injected. </p> <h3 id="392b67a393cc4deaad9b19901ea4c4c4"> Using the database <a href="#392b67a393cc4deaad9b19901ea4c4c4" title="permalink">#</a> </h3> <p> Using the database isn't much harder. I don't find that <a href="https://en.wikipedia.org/wiki/Object-relational_mapping">ORMs</a> offer any benefits, so I prefer to implement database functionality using basic database APIs: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="font-weight:bold;color:#74531f;">Create</span>(<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">conn</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlConnection</span>(ConnectionString)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlCommand</span>(createReservationSql,&nbsp;<span style="font-weight:bold;color:#1f377f;">conn</span>)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.Parameters.<span style="font-weight:bold;color:#74531f;">Add</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Guid&quot;</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>.Id)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.Parameters.<span style="font-weight:bold;color:#74531f;">Add</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Date&quot;</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>.Date)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.Parameters.<span style="font-weight:bold;color:#74531f;">Add</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Name&quot;</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>.Name)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.Parameters.<span style="font-weight:bold;color:#74531f;">Add</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Email&quot;</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>.Email)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.Parameters.<span style="font-weight:bold;color:#74531f;">Add</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlParameter</span>(<span style="color:#a31515;">&quot;@Quantity&quot;</span>,&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">conn</span>.<span style="font-weight:bold;color:#74531f;">Open</span>(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#1f377f;">cmd</span>.<span style="font-weight:bold;color:#74531f;">ExecuteNonQuery</span>(); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;createReservationSql&nbsp;=&nbsp;<span style="color:maroon;">@&quot; &nbsp;&nbsp;&nbsp;&nbsp;INSERT&nbsp;INTO&nbsp;[dbo].[Reservations]&nbsp;([Guid],&nbsp;[Date],&nbsp;[Name],&nbsp;[Email],&nbsp;[Quantity]) &nbsp;&nbsp;&nbsp;&nbsp;OUTPUT&nbsp;INSERTED.Id &nbsp;&nbsp;&nbsp;&nbsp;VALUES&nbsp;(@Guid,&nbsp;@Date,&nbsp;@Name,&nbsp;@Email,&nbsp;@Quantity)&quot;</span>;</pre> </p> <p> The above code snippet implements the <code>Create</code> method of the <code>IReservationsRepository</code> interface. Please refer to the Git repository for the full code if you need more details. </p> <p> If you prefer to implement your database functionality with an ORM, or in another way, you can do that. It doesn't change the architecture of the system. No logging service is required to interact with the database. </p> <h3 id="e70ddcfa03c04444931f0b3737e09101"> Compose with logging <a href="#e70ddcfa03c04444931f0b3737e09101" title="permalink">#</a> </h3> <p> As the above composition code snippet suggests, logging is implemented with <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorators</a>. The ultimate implementation of <code>IClock</code> is <code>SystemClock</code>, but the <a href="/2011/07/28/CompositionRoot">Composition Root</a> decorates it with <code>LogClock</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LogClock</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IClock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">LogClock</span>(<span style="color:#2b91af;">IClock</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">inner</span>,&nbsp;<span style="color:#2b91af;">ScopedLog</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">log</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Inner&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">inner</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Log&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">log</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IClock</span>&nbsp;Inner&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ScopedLog</span>&nbsp;Log&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">currentDateTime</span>&nbsp;=&nbsp;Inner.<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Log.<span style="font-weight:bold;color:#74531f;">Observe</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Interaction</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Operation&nbsp;=&nbsp;<span style="color:blue;">nameof</span>(<span style="font-weight:bold;color:#74531f;">GetCurrentDateTime</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Output&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">currentDateTime</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">currentDateTime</span>; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> <code>ScopedLog</code> is a Concrete Dependency that, among other members, affords the <code>Observe</code> method. Notice that <code>LogClock</code> implements <code>IClock</code> by decorating another polymorphic <code>IClock</code> instance. It delegates functionality to <code>inner</code>, logs the <code>currentDateTime</code> and returns it. </p> <p> The <code>LogReservationsRepository</code> class implements the same pattern: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LogReservationsRepository</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IReservationsRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">LogReservationsRepository</span>(<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">inner</span>,&nbsp;<span style="color:#2b91af;">ScopedLog</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">log</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Inner&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">inner</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Log&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">log</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;Inner&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ScopedLog</span>&nbsp;Log&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="font-weight:bold;color:#74531f;">Create</span>(<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Log.<span style="font-weight:bold;color:#74531f;">Observe</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Interaction</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Operation&nbsp;=&nbsp;<span style="color:blue;">nameof</span>(<span style="font-weight:bold;color:#74531f;">Create</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Input&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;<span style="font-weight:bold;color:#1f377f;">reservation</span>&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Inner.<span style="font-weight:bold;color:#74531f;">Create</span>(<span style="font-weight:bold;color:#1f377f;">reservation</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="font-weight:bold;color:#74531f;">ReadReservations</span>(<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">date</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservations</span>&nbsp;=&nbsp;Inner.<span style="font-weight:bold;color:#74531f;">ReadReservations</span>(<span style="font-weight:bold;color:#1f377f;">date</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Log.<span style="font-weight:bold;color:#74531f;">Observe</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Interaction</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Operation&nbsp;=&nbsp;<span style="color:blue;">nameof</span>(<span style="font-weight:bold;color:#74531f;">ReadReservations</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Input&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;<span style="font-weight:bold;color:#1f377f;">date</span>&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Output&nbsp;=&nbsp;<span style="font-weight:bold;color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="font-weight:bold;color:#8f08c4;">return</span>&nbsp;<span style="font-weight:bold;color:#1f377f;">reservations</span>; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This architecture not only implements the desired functionality, but also <em>Goldilogs</em>: not too little, not too much, but just what you need. Notice that I didn't have to change any of my Domain Model or HTTP-specific code to enable logging. This cross-cutting concern is enabled entirely via composition. </p> <h3 id="59b912d918c9451e97e09b8274d7fb87"> Repeatability <a href="#59b912d918c9451e97e09b8274d7fb87" title="permalink">#</a> </h3> <p> An HTTP request like this: </p> <p> <pre>POST /reservations/ HTTP/1.1 Content-Type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;7bc3fc93-a777-4138-8630-a805e7246335&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-03-20&nbsp;18:45:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kozue Kaburagi&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;ninjette@example.net&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 }</pre> </p> <p> produces a log entry like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;entry&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-01-02T09:50:34.2678703+01:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;operation&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Post&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;input&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;dto&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;7bc3fc93-a777-4138-8630-a805e7246335&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-03-20&nbsp;18:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;ninjette@example.net&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kozue Kaburagi&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;output&quot;</span>:&nbsp;<span style="color:blue;">null</span> &nbsp;&nbsp;}, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;interactions&quot;</span>:&nbsp