ploeh blog 2023-02-02T19:08:12+00:00 Mark Seemann danish software design https://blog.ploeh.dk Built-in alternatives to applicative assertions https://blog.ploeh.dk/2023/01/30/built-in-alternatives-to-applicative-assertions 2023-01-30T08:08:00+00:00 Mark Seemann <div id="post"> <p> <em>Why make things so complicated?</em> </p> <p> Several readers reacted to my small article series on <a href="/2022/11/07/applicative-assertions">applicative assertions</a>, pointing out that error-collecting assertions are already supported in more than one unit-testing framework. </p> <blockquote> <p> "In the Java world this seems similar to the result gained by Soft Assertions in AssertJ. <a href="https://assertj.github.io/doc/#assertj-core-soft-assertions">https://assertj.github.io/doc/#assertj-c...</a> if you’re after a target for functionality (without the adventures through monad land)" </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1597190184134590464">Josh McK</a></cite></footer> </blockquote> <p> While I'm not familiar with the details of Java unit-testing frameworks, the situation is similar in .NET, it turns out. </p> <blockquote> <p> "Did you know there is Assert.Multiple in NUnit and now also in xUnit .Net? It seems to have quite an overlap with what you're doing here. </p> <p> "For a quick overview, I found this blogpost helpful: <a href="https://www.thomasbogholm.net/2021/11/25/xunit-2-4-2-pre-multiple-asserts-in-one-test/">https://www.thomasbogholm.net/2021/11/25/xunit-2-4-2-pre-multiple-asserts-in-one-test/</a>" </p> <footer><cite><a href="https://twitter.com/DoCh_Dev/status/1597158737357459456">DoCh_Dev</a></cite></footer> </blockquote> <p> I'm not surprised to learn that something like this exists, but let's take a quick look. </p> <h3 id="d7c0c78093084c08aa22ccaa7b86cb8a"> NUnit Assert.Multiple <a href="#d7c0c78093084c08aa22ccaa7b86cb8a" title="permalink">#</a> </h3> <p> Let's begin with <a href="https://nunit.org/">NUnit</a>, as this seems to be the first .NET unit-testing framework to support error-collecting assertions. As a beginning, the <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">documentation example</a> works as it's supposed to: </p> <p> <pre>[Test] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ComplexNumberTest() { &nbsp;&nbsp;&nbsp;&nbsp;ComplexNumber&nbsp;result&nbsp;=&nbsp;SomeCalculation(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple(()&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.AreEqual(5.2,&nbsp;result.RealPart,&nbsp;<span style="color:#a31515;">&quot;Real&nbsp;part&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.AreEqual(3.9,&nbsp;result.ImaginaryPart,&nbsp;<span style="color:#a31515;">&quot;Imaginary&nbsp;part&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> When you run the test, it fails (as expected) with this error message: </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures or warnings in test: &nbsp;&nbsp;&nbsp;&nbsp;1) Real part &nbsp;&nbsp;&nbsp;&nbsp;Expected: 5.2000000000000002d &nbsp;&nbsp;&nbsp;&nbsp;But was: 5.0999999999999996d &nbsp;&nbsp;&nbsp;&nbsp;2) Imaginary part &nbsp;&nbsp;&nbsp;&nbsp;Expected: 3.8999999999999999d &nbsp;&nbsp;&nbsp;&nbsp;But was: 4.0d</pre> </p> <p> That seems to work well enough, but how does it actually work? I'm not interested in reading the NUnit source code - after all, the concept of <a href="/encapsulation-and-solid">encapsulation</a> is that one should be able to make use of the capabilities of an object without knowing all implementation details. Instead, I'll guess: Perhaps <code>Assert.Multiple</code> executes the code block in a <code>try/catch</code> block and collects the various exceptions thrown by the nested assertions. </p> <p> Does it catch all exception types, or only a subset? </p> <p> Let's try with the kind of composed assertion that I <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previously investigated</a>: </p> <p> <pre>[Test] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple(()&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.That(getResp.StatusCode,&nbsp;Is.EqualTo(HttpStatusCode.NotFound)); &nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> This test fails (again, as expected). What's the error message? </p> <p> <pre>Message: &nbsp;&nbsp;System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;&nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> (I've wrapped the result over multiple lines for readability. The <code>↩</code> symbol indicates where I've wrapped the text. I'll do that again later in this article.) </p> <p> Notice that I'm using <a href="/2020/09/28/ensuresuccessstatuscode-as-an-assertion">EnsureSuccessStatusCode as an assertion</a>. This seems to spoil the behaviour of <code>Assert.Multiple</code>. It only reports the first status code error, but not the second one. </p> <p> I admit that I don't fully understand what's going on here. In fact, I <em>have</em> taken a cursory glance at the relevant NUnit source code without being enlightened. </p> <p> One hypothesis might be that NUnit assertions throw special <code>Exception</code> sub-types that <code>Assert.Multiple</code> catch. In order to test that, I wrote a few more tests in <a href="https://fsharp.org/">F#</a> with <a href="http://www.swensensoftware.com/unquote/">Unquote</a>, assuming that, since Unquote hardly throws NUnit exceptions, the behaviour might be similar to above. </p> <p> <pre>[&lt;Test&gt;] <span style="color:blue;">let</span>&nbsp;Test4&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;y&nbsp;=&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;z&nbsp;=&nbsp;3 &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple&nbsp;(<span style="color:blue;">fun</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=!&nbsp;y &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;y&nbsp;=!&nbsp;z)</pre> </p> <p> The <code>=!</code> operator is an Unquote operator that I usually read as <em>must equal</em>. How does that error message look? </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures or warnings in test: &nbsp;&nbsp;&nbsp;&nbsp;1) &nbsp;&nbsp;1 = 2 &nbsp;&nbsp;false &nbsp;&nbsp;&nbsp;&nbsp;2) &nbsp;&nbsp;2 = 3 &nbsp;&nbsp;false</pre> </p> <p> Somehow, <code>Assert.Multiple</code> understands Unquote error messages, but not <code>HttpRequestException</code>. As I wrote, I don't fully understand why it behaves this way. To a degree, I'm intellectually curious enough that I'd like to know. On the other hand, from a maintainability perspective, as a user of NUnit, I shouldn't have to understand such details. </p> <h3 id="1f28e534b0f94e93bb12cbb951fa663f"> xUnit.net Assert.Multiple <a href="#1f28e534b0f94e93bb12cbb951fa663f" title="permalink">#</a> </h3> <p> How fares the <a href="https://xunit.net/">xUnit.net</a> port of <code>Assert.Multiple</code>? </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;()&nbsp;=&gt;&nbsp;deleteResp.EnsureSuccessStatusCode(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;()&nbsp;=&gt;&nbsp;Assert.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode)); }</pre> </p> <p> The API is, you'll notice, not quite identical. Where the NUnit <code>Assert.Multiple</code> method takes a single delegate as input, the xUnit.net method takes an array of actions. The difference is not only at the level of API; the behaviour is different, too: </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures were encountered: &nbsp;&nbsp;---- System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request). &nbsp;&nbsp;---- Assert.Equal() Failure &nbsp;&nbsp;Expected: NotFound &nbsp;&nbsp;Actual: OK</pre> </p> <p> This error message reports both problems, as we'd like it to do. </p> <p> I also tried writing equivalent tests in F#, with and without Unquote, and they behave consistently with this result. </p> <p> If I had to use something like <code>Assert.Multiple</code>, I'd trust the xUnit.net variant more than NUnit's implementation. </p> <h3 id="c51fb932618c4464a2e9be05869005d4"> Assertion scopes <a href="#c51fb932618c4464a2e9be05869005d4" title="permalink">#</a> </h3> <p> Apparently, <a href="https://fluentassertions.com/">Fluent Assertions</a> offers yet another alternative. </p> <blockquote> <p> "Hey @ploeh, been reading your applicative assertion series. I recently discovered Assertion Scopes, so I'm wondering what is your take on them since it seems to me they are solving this problem in C# already. <a href="https://fluentassertions.com/introduction#assertion-scopes">https://fluentassertions.com/introduction#assertion-scopes</a></a>" </p> <footer><cite><a href="https://twitter.com/JernejGoricki/status/1597704973839904768">Jernej Gorički</a></cite></footer> </blockquote> <p> The linked documentation contains this example: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;DocExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">new</span>&nbsp;AssertionScope()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.Should().Be(10); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Actual&quot;</span>.Should().Be(<span style="color:#a31515;">&quot;Expected&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> It fails in the expected manner: </p> <p> <pre>Message: &nbsp;&nbsp;Expected value to be 10, but found 5 (difference of -5). &nbsp;&nbsp;Expected string to be "Expected" with a length of 8, but "Actual" has a length of 6,↩ &nbsp;&nbsp;&nbsp;&nbsp;differs near "Act" (index 0).</pre> </p> <p> How does it fare when subjected to the <code>EnsureSuccessStatusCode</code> test? </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">new</span>&nbsp;AssertionScope()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getResp.StatusCode.Should().Be(HttpStatusCode.NotFound); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> That test produces this error output: </p> <p> <pre>Message: &nbsp;&nbsp;System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;&nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> Again, <code>EnsureSuccessStatusCode</code> prevents further assertions from being evaluated. I can't say that I'm that surprised. </p> <h3 id="2f325809027243889d703e927a115efc"> Implicit or explicit <a href="#2f325809027243889d703e927a115efc" title="permalink">#</a> </h3> <p> You might protest that using <code>EnsureSuccessStatusCode</code> and treating the resulting <code>HttpRequestException</code> as an assertion is unfair and unrealistic. Possibly. As usual, such considerations are subject to a multitude of considerations, and there's no one-size-fits-all answer. </p> <p> My intent with this article isn't to attack or belittle the APIs I've examined. Rather, I wanted to explore their boundaries by stress-testing them. That's one way to gain a better understanding. Being aware of an API's limitations and quirks can prevent subtle bugs. </p> <p> Even if you'd <em>never</em> use <code>EnsureSuccessStatusCode</code> as an assertion, perhaps you or a colleague might inadvertently do something to the same effect. </p> <p> I'm not surprised that both NUnit's <code>Assert.Multiple</code> and Fluent Assertions' <code>AssertionScope</code> behaves in a less consistent manner than xUnit.net's <code>Assert.Multiple</code>. The clue is in the API. </p> <p> The xUnit.net API looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Multiple</span>(<span style="color:blue;">params</span>&nbsp;Action[]&nbsp;<span style="color:#1f377f;">checks</span>)</pre> </p> <p> Notice that each assertion is explicitly a separate action. This enables the implementation to isolate it and treat it independently of other actions. </p> <p> Neither the NUnit nor the Fluent Assertions API is that explicit. Instead, you can write arbitrary code inside the 'scope' of multiple assertions. For <code>AssertionScope</code>, the notion of a 'scope' is plain to see. For the NUnit API it's more implicit, but the scope is effectively the extent of the method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Multiple</span>(TestDelegate&nbsp;<span style="color:#1f377f;">testDelegate</span>)</pre> </p> <p> That <code>testDelegate</code> can have as many (nested, even) assertions as you'd like, so the <code>Multiple</code> implementation needs to somehow demarcate when it begins and when it ends. </p> <p> The <code>testDelegate</code> can be implemented in a different file, or even in a different library, and it has no way to communicate or coordinate with its surrounding scope. This reminds me of an Ambient Context, an idiom that <a href="/2019/01/21/some-thoughts-on-anti-patterns">Steven van Deursen convinced me was an anti-pattern</a>. The surrounding context changes the behaviour of the code block it surrounds, and it's quite implicit. </p> <blockquote> <p> Explicit is better than implicit. </p> <footer><cite>Tim Peters, <a href="https://peps.python.org/pep-0020/">The Zen of Python</a></cite></footer> </blockquote> <p> The xUnit.net API, at least, looks a bit saner. Still, this kind of API is quirky enough that it reminds me of <a href="https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule">Greenspun's tenth rule</a>; that these APIs are ad-hoc, informally-specified, bug-ridden, slow implementations of half of <a href="/2018/10/01/applicative-functors">applicative functors</a>. </p> <h3 id="95b723f05a7f4b4c88b5e716f5f82989"> Conclusion <a href="#95b723f05a7f4b4c88b5e716f5f82989" title="permalink">#</a> </h3> <p> Not surprisingly, popular unit-testing and assertion libraries come with facilities to compose assertions. Also, not surprisingly, these APIs are crude and require you to learn their implementation details. </p> <p> Would I use them if I had to? I probably would. As <a href="https://www.infoq.com/presentations/Simple-Made-Easy/">Rich Hickey put it</a>, they're already <em>at hand</em>. That makes them easy, but not necessarily simple. APIs that compel you to learn their internal implementation details aren't simple. </p> <p> <a href="/2017/10/04/from-design-patterns-to-category-theory">Universal abstractions</a>, on the other hand, you only have to learn one time. Once you understand what an applicative functor is, you know what to expect from it, and which capabilities it has. </p> <p> In languages with good support for applicative functors, I would favour an assertion API based on that abstraction, if given a choice. At the moment, though, that's not much of an option. Even <a href="https://hackage.haskell.org/package/HUnit/docs/Test-HUnit-Base.html#t:Assertion">HUnit assertions</a> are based on side effects. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="e3269279066146f985c8405f6d3ad286"> <div class="comment-author">Joker_vD</div> <div class="comment-content"> <p> Just a reminder: in .NET, method's execution <b>cannot</b> be resumed after an exception is thrown, there is just simply no way to do this, at all. Which means that NUnit's Assert.Multiple absolutely cannot work the way you guess it probably does, by running the delegate and resuming its execution after it throws an exception until the delegate returns. </p> <p> How could it work then? Well, considering that documentation to almost every Assert's method has "Returns without throwing an exception when inside a multiple assert block" line in it, I would assume that Assert.Multiple sets a global flag which makes actual assertions to store the failures in some global hidden context instead on throwing them, then runs the delegate and after it finishes or throws, collects and clears all those failures from the context and resets the global flag. </p> <p> Cursory inspection of NUnit's source code supports this idea, except that apparently it's not just a boolean flag but a "depth" counter; and assertions report the failures <a href="https://github.com/nunit/nunit/blob/62059054137de84b711353765d474779db95f731/src/NUnitFramework/framework/Assert.cs#L371">just the way I've speculated</a>. I personally hate such side-channels but you have to admit, they allow for some nifty, seemingly impossible magical tricks (a.k.a. "spooky action at the distance"). </p> <p> Also, why do you assume that Unquote would not throw NUnit's assertions? It literally has "Unquote integrates configuration-free with all exception-based unit testing frameworks including xUnit.net, NUnit, MbUnit, Fuchu, and MSTest" in its README, and indeed, if you look at <a href="https://github.com/SwensenSoftware/unquote/blob/78b071043c42372f3693a07e5562520046873ebc/src/Unquote/Assertions.fs">its source code</a>, you'll see that at runtime it tries to locate any testing framework it's aware of and use its assertions. More funny party tricks, this time with reflection! </p> <p> I understand that after working in more pure/functional programming environments one does start to slowly forget about those terrible things, but: those horrorterrors <i>still</i> exist, and people <i>keep making</i> more of them. Now, if you can, have a good night :) </p> </div> <div class="comment-date">2023-01-31 03:00 UTC</div> </div> <div class="comment" id="e0e7c5b258d54c30b87f157e8746150d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Joker_vD, thank you for explaining those details. I admit that I hadn't thought too deeply about implementation details, for the reasons I briefly mentioned in the post. </p> <blockquote> <p> "I understand that after working in more pure/functional programming environments one does start to slowly forget about those terrible things" </p> </blockquote> <p> Yes, that summarises my current thinking well, I'm afraid. </p> </div> <div class="comment-date">2023-01-30 6:49 UTC</div> </div> <div class="comment" id="821541be129a4ea7976ab33f71d3637a"> <div class="comment-author"><a href="https://github.com/MaxKot">Max Kiselev</a></div> <div class="comment-content"> <p> NUnit has <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/classic-assertions/Assert.DoesNotThrow.html">Assert.DoesNotThrow</a> and Fluent Assertions has <a href="https://fluentassertions.com/exceptions/">.Should().NotThrow()</a>. I did not check Fluent Assertions, but NUnit does gather failures of Assert.DoesNotThrow inside Assert.Multiple into a multi-error report. One might argue that asserting that a delegate should not throw is another application of the "explicit is better than implicit" philosophy. Here's what Fluent Assertions has to say on that matter: </p> <blockquote> <p> "We know that a unit test will fail anyhow if an exception was thrown, but this syntax returns a clearer description of the exception that was thrown and fits better to the AAA syntax." </p> </blockquote> <p> As a side note, you might also want to take a look on NUnits Assert.That syntax. It allows to construct complex conditions tested against a single actual value: </p> <p> <pre style="font-family:Consolas;font-size:13px;color:black;background:white;"><span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;3; Assert.That&nbsp;(actual,&nbsp;Is.GreaterThan&nbsp;(0).And.LessThanOrEqualTo&nbsp;(2).And.Matches&nbsp;(Has.Property&nbsp;(<span style="color:#a31515;">&quot;P&quot;</span>).EqualTo&nbsp;(<span style="color:#a31515;">&quot;a&quot;</span>)));</pre> </p> <p> A failure is then reported like this: </p> <p> <pre>Expected: greater than 0 and less than or equal to 2 and property P equal to "a" But was: 3</pre> </p> </div> <div class="comment-date">2023-01-31 18:35 UTC</div> </div> <div class="comment" id="815b4f0e18284dccb3ce38dbb476eb4d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Max, thank you for writing. I have to admit that I never understood the point of <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/assertion-models/constraint.html">NUnit's constraint model</a>, but your example clearly illustrates how it may be useful. It enables you to compose assertions. </p> <p> It's interesting to try to understand the underlying reason for that. I took a cursory glance at that <code>IResolveConstraint</code> API, and as far as I can tell, it may form a <a href="/2017/10/06/monoids">monoid</a> (I'm not entirely sure about the <code>ConstraintStatus</code> enum, but even so, it may be 'close enough' to be composable). </p> <p> I can see how that may be useful when making assertions against complex objects (i.e. object composed from other objects). </p> <p> In xUnit.net you'd typically address that problem with custom <a href="https://learn.microsoft.com/dotnet/api/system.collections.generic.iequalitycomparer-1">IEqualityComparers</a>. This is more verbose, but also strikes me as more reusable. One disadvantage of that approach, however, is that when tests fail, the assertion message is typically useless. </p> <p> This is the reason I favour Unquote: Instead of inventing a Boolean algebra(?) from scratch, it uses the existing language and still gives you good error messages. Alas, that only works in F#. </p> <p> In general, though, I'm inclined to think that all of these APIs address symptoms rather than solve real problems. Granted, they're useful whenever you need to make assertions against values that you don't control, but for your own APIs, <a href="/2021/05/03/structural-equality-for-better-tests">a simpler solution is to model values as immutable data with structural equality</a>. </p> <p> Another question is whether aiming for clear assertion messages is optimising for the right concern. At least with TDD, <a href="/2022/12/12/when-do-tests-fail">I don't think that it is</a>. </p> </div> <div class="comment-date">2023-02-02 7:53 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Agilean https://blog.ploeh.dk/2023/01/23/agilean 2023-01-23T07:55:00+00:00 Mark Seemann <div id="post"> <p> <em>There are other agile methodologies than scrum.</em> </p> <p> More than twenty years after <a href="https://agilemanifesto.org/">the Agile Manifesto</a> it looks as though there's only one kind of agile process left: <a href="https://en.wikipedia.org/wiki/Scrum_(software_development)">Scrum</a>. </p> <p> I recently held a workshop and as a side remark I mentioned that I don't consider scrum the best development process. This surprised some attendees, who politely inquired about my reasoning. </p> <h3 id="8837c4f67d694f93ad0b708cc1739705"> My experience with scrum <a href="#8837c4f67d694f93ad0b708cc1739705" title="permalink">#</a> </h3> <p> The first nine years I worked as a professional programmer, the companies I worked in used various <a href="https://en.wikipedia.org/wiki/Waterfall_model">waterfall</a> processes. When I joined the Microsoft Dynamics Mobile team in 2008 they were already using scrum. That was my first exposure to it, and I liked it. Looking back on it today, we weren't particular dogmatic about the process, being more interested in getting things done. </p> <p> One telling fact is that we took turns being Scrum Master. Every sprint we'd rotate that role. </p> <p> We did test-driven development, and had two-week sprints. This being a Microsoft development organisation, we had a dedicated build master, tech writers, specialised testers, and security reviews. </p> <p> I liked it. It's easily one of the most professional software organisations I've worked in. I think it was a good place to work for many reasons. Scrum may have been a contributing factor, but hardly the only reason. </p> <p> I have no issues with scrum as we practised it then. I recall later attending a presentation by <a href="https://en.wikipedia.org/wiki/Mike_Cohn">Mike Cohn</a> where he outlined four quadrants of team maturity. You'd start with scrum, but use retrospectives to evaluate what worked and what didn't. Then you'd adjust. A mature, self-organising team would arrive at its own process, perhaps initiated with scrum, but now having little resemblance with it. </p> <p> I like scrum when viewed like that. When it becomes rigid and empty ceremony, I don't. If all you do is daily stand-ups, sprints, and backlogs, you may be doing scrum, but probably not agile. </p> <h3 id="1bb8e521e78341768ca7f5942f3ace4a"> Continuous deployment <a href="#1bb8e521e78341768ca7f5942f3ace4a" title="permalink">#</a> </h3> <p> After Microsoft I joined a startup so small that formal process was unnecessary. Around that time I also became interested in <a href="https://en.wikipedia.org/wiki/Lean_software_development">lean software development</a>. In the beginning, I learned a lot from <a href="https://www.linkedin.com/in/martin-jul-39a12/">Martin Jul</a> who seemed to use the now-defunct <a href="https://ative.dk/">Ative</a> blog as a public notepad as he was reading works of <a href="https://en.wikipedia.org/wiki/W._Edwards_Deming">Deming</a>. I suppose, if you want a more canonical introduction to the topic, that you might start with one of <a href="http://www.poppendieck.com/">the Poppendiecks'</a> books, but since I've only read <a href="/ref/implementing-lean">Implementing Lean Software Development</a>, that's the only one I can recommend. </p> <p> Around 2014 I returned to a regular customer. The team had, in my absence, been busy implementing <a href="https://en.wikipedia.org/wiki/Continuous_deployment">continuous deployment</a>. Instead of artificial periods like 'sprints' we had a <a href="https://en.wikipedia.org/wiki/Kanban_board">kanban board</a> to keep track of our work. We used a variation of <a href="https://en.wikipedia.org/wiki/Feature_toggle">feature flags</a> and marked features as done when they were complete and in production. </p> <p> Why wait until <em>next</em> Friday if the feature is <em>done, done</em> on a Wednesday? Why wait until the <em>next</em> Monday to identify what to work on next, if you're ready to take on new work on a Thursday? Why not move towards <em>one-piece flow?</em> </p> <p> An effective self-organising team typically already knows what it's doing. Much process is introduced in order to give external stakeholders visibility into what a team is doing. </p> <p> I found, in that organisation, that continuous deployment eliminated most of that need. At one time I asked a stakeholder what he thought of the feature I'd deployed a week before - a feature that <em>he had requested</em>. He replied that he hadn't had time to look at it yet. </p> <p> The usual inquires about status (<em>Is it done yet? When is it done?</em>) were gone. The team moved faster than the stakeholders could keep up. That also gave us <a href="/2022/09/19/when-to-refactor">enough slack to keep the code base in good order</a>. We also used test-driven development throughout (TDD). </p> <p> TDD with continuous deployment and a kanban board strikes me as congenial with the ideas of lean software development, but that's not all. </p> <h3 id="9f5e725164f54d32b5d56a28d3b1619e"> Stop-the-line issues <a href="#9f5e725164f54d32b5d56a28d3b1619e" title="permalink">#</a> </h3> <p> An <a href="https://en.wikipedia.org/wiki/Andon_(manufacturing)">andon cord</a> is a central concept in <a href="https://en.wikipedia.org/wiki/Lean_manufacturing">lean manufactoring</a>. If a worker (or anyone, really) discovers a problem during production, he or she pulls the andon cord and <em>stops the production line</em>. Then everyone investigates and determines what to do about the problem. Errors are not allowed to accumulate. </p> <p> I think that I've internalised this notion to such a degree that I only recently connected it to lean software development. </p> <p> In <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>, I recommend turning compiler warnings into errors at the beginning of a code base. Don't allow warnings to pile up. Do the same with static code analysis and linters. </p> <p> When discussing software engineering with developers, I'm beginning to realise that this runs even deeper. </p> <ul> <li>Turn warnings into errors. Don't allow warnings to accumulate.</li> <li>The correct number of unhandled exceptions in production is zero. If you observe an unhandled exception in your production logs, fix it. Don't let them accumulate.</li> <li>The correct number of known bugs is zero. Don't let bugs accumulate.</li> </ul> <p> If you're used to working on a code base with hundreds of known bugs, and frequent exceptions in production, this may sound unrealistic. If you deal with issues as soon as they arise, however, this is not only possible - it's faster. </p> <p> In lean software development, bugs are stop-the-line issues. When something unexpected happens, you stop what you're doing and make fixing the problem the top priority. You build quality in. </p> <p> This has been my modus operandi for years, but I only recently connected the dots to realise that this is a typical lean practice. I may have picked it up from there. Or perhaps it's just common sense. </p> <h3 id="e2513b8e32b148cbab8dbd6b75c67348"> Conclusion <a href="#e2513b8e32b148cbab8dbd6b75c67348" title="permalink">#</a> </h3> <p> When Agile was new and exciting, there were <a href="https://en.wikipedia.org/wiki/Extreme_programming">extreme programming</a> and scrum, and possibly some lesser known techniques. Lean was around the corner, but didn't come to my attention, at least, until around 2010. Then it seems to have faded away again. </p> <p> Today, agile looks synonymous with scrum, but I find lean software development more efficient. Why divide work into artificial time periods when you can release continuously? Why <em>plan</em> bug fixing when it's more efficient to stop the line and deal with the problem as it arises? </p> <p> That may sound counter-intuitive, but it works because it prevents technical debt from accumulating. </p> <p> Lean software development is, in my experience, a better agile methodology than scrum. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. In the long run https://blog.ploeh.dk/2023/01/16/in-the-long-run 2023-01-16T08:28:00+00:00 Mark Seemann <div id="post"> <p> <em>Software design decisions should be time-aware.</em> </p> <p> A common criticism of modern capitalism is that maximising <a href="https://en.wikipedia.org/wiki/Shareholder_value">shareholder value</a> leads to various detrimental outcomes, both societal, but possibly also for the maximising organisation itself. One major problem is when company leadership is incentivised to optimise stock market price for the next quarter, or other short terms. When considering only the short term, decision makers may (rationally) decide to sacrifice long-term benefits for short-term gains. </p> <p> We often see similar behaviour in democracies. Politicians tend to optimise within a time frame that coincides with the election period. Getting re-elected is more important than good policy in the next period. </p> <p> These observations are crude generalisations. Some democratic politicians and CEOs take longer views. Inherent in the context, however, is an incentive to short-term thinking. </p> <p> This, it strikes me, is frequently the case in software development. </p> <p> Particularly in the context of <a href="https://en.wikipedia.org/wiki/Scrum_(software_development)">scrum</a> there's a focus on delivering at the end of every sprint. I've observed developers and other stakeholders together engage in short-term thinking in order to meet those arbitrary and fictitious deadlines. </p> <p> Even when deadlines are more remote than two weeks, project members rarely think beyond some perceived end date. As I describe in <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>, a <em>project</em> is rarely is good way to organise software development work. Projects end. Successful software doesn't. </p> <p> Regardless of the specific circumstances, a too myopic focus on near-term goals gives you an incentive to cut corners. To not care about code quality. </p> <h3 id="5cd528d178504f2ba849019e5d6d425e"> ...we're all dead <a href="#5cd528d178504f2ba849019e5d6d425e" title="permalink">#</a> </h3> <p> As <a href="https://en.wikipedia.org/wiki/John_Maynard_Keynes">Keynes</a> once quipped: </p> <blockquote> <p> "In the long run we are all dead." </p> <footer><cite>John Maynard Keynes</cite></footer> </blockquote> <p> Clearly, while you can be too short-sighted, you can also take too long a view. Sometimes deadlines matter, and software not used makes no-one happy. </p> <p> Working software remains the ultimate test of value, but as I've tried to express many times before, this does <em>not</em> imply that anything else is worthless. </p> <p> You can't measure code quality. <a href="/2019/03/04/code-quality-is-not-software-quality">Code quality isn't software quality</a>. Low code quality <a href="https://martinfowler.com/articles/is-quality-worth-cost.html">slows you down</a>, and that, eventually, costs you money, blood, sweat, and tears. </p> <p> This is, however, not difficult to predict. All it takes is a slightly wider time horizon. Consider the impact of your decisions past the next deadline. </p> <h3 id="b0f0f05bd00142379c2d713adde6eb77"> Conclusion <a href="#b0f0f05bd00142379c2d713adde6eb77" title="permalink">#</a> </h3> <p> Don't be too short-sighted, but don't forget the immediate value of what you do. Your <a href="/2019/03/18/the-programmer-as-decision-maker">decisions</a> matter. The impact is not always immediate. Consider what consequences short-term optimisations may have in a longer perspective. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The IO monad https://blog.ploeh.dk/2023/01/09/the-io-monad 2023-01-09T07:39:00+00:00 Mark Seemann <div id="post"> <p> <em>The IO container forms a monad. An article for object-oriented programmers.</em> </p> <p> This article is an instalment in <a href="/2022/03/28/monads">an article series about monads</a>. A previous article described <a href="/2020/06/22/the-io-functor">the IO functor</a>. As is the case with many (but not all) <a href="/2018/03/22/functors">functors</a>, this one also forms a monad. </p> <h3 id="bee075f5bf474b099146cbc7c96f4888"> SelectMany <a href="#bee075f5bf474b099146cbc7c96f4888" title="permalink">#</a> </h3> <p> A monad must define either a <em>bind</em> or <em>join</em> function. In C#, monadic bind is called <code>SelectMany</code>. In a recent article, I gave an example of <a href="/2020/06/15/io-container-in-a-parallel-c-universe">what <em>IO</em> might look like in C#</a>. Notice that it already comes with a <code>SelectMany</code> function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;IO&lt;TResult&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(Func&lt;T,&nbsp;IO&lt;TResult&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>)</pre> </p> <p> Unlike other monads, the IO implementation is considered a black box, but if you're interested in a prototypical implementation, I already posted <a href="/2020/07/13/implementation-of-the-c-io-container">a sketch</a> in 2020. </p> <h3 id="1ba6a4e6da014fc3a511e71bdba4f0a9"> Query syntax <a href="#1ba6a4e6da014fc3a511e71bdba4f0a9" title="permalink">#</a> </h3> <p> I have also, already, demonstrated <a href="/2020/06/29/syntactic-sugar-for-io">syntactic sugar for IO</a>. In that article, however, I used an implementation of the required <code>SelectMany</code> overload that is more explicit than it has to be. The <a href="/2022/03/28/monads">monad introduction</a> makes the prediction that you can always implement that overload in the same way, and yet here I didn't. </p> <p> That's an oversight on my part. You can implement it like this instead: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;TResult&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">U</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;IO&lt;U&gt;&gt;&nbsp;<span style="color:#1f377f;">k</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;U,&nbsp;TResult&gt;&nbsp;<span style="color:#1f377f;">s</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;k(x).Select(<span style="color:#1f377f;">y</span>&nbsp;=&gt;&nbsp;s(x,&nbsp;y))); }</pre> </p> <p> Indeed, the conjecture from the introduction still holds. </p> <h3 id="f234f9e9d3724a30b8465d176a0d98a6"> Join <a href="#f234f9e9d3724a30b8465d176a0d98a6" title="permalink">#</a> </h3> <p> In <a href="/2022/03/28/monads">the introduction</a> you learned that if you have a <code>Flatten</code> or <code>Join</code> function, you can implement <code>SelectMany</code>, and the other way around. Since we've already defined <code>SelectMany</code> for <code>IO&lt;T&gt;</code>, we can use that to implement <code>Join</code>. In this article I use the name <code>Join</code> rather than <code>Flatten</code>. This is an arbitrary choice that doesn't impact behaviour. Perhaps you find it confusing that I'm inconsistent, but I do it in order to demonstrate that the behaviour is the same even if the name is different. </p> <p> The concept of a monad is universal, but the names used to describe its components differ from language to language. What C# calls <code>SelectMany</code>, Scala calls <code>flatMap</code>, and what <a href="https://www.haskell.org/">Haskell</a> calls <code>join</code>, other languages may call <code>Flatten</code>. </p> <p> You can always implement <code>Join</code> by using <code>SelectMany</code> with the identity function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#74531f;">Join</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;IO&lt;IO&lt;T&gt;&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x); }</pre> </p> <p> In C# the identity function is <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatically</a> given as the lambda expression <code><span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x</code> since C# doesn't come with a built-in identity function. </p> <h3 id="c57bcf1e74144046bc7191280d78519b"> Return <a href="#c57bcf1e74144046bc7191280d78519b" title="permalink">#</a> </h3> <p> Apart from monadic bind, a monad must also define a way to put a normal value into the monad. Conceptually, I call this function <em>return</em> (because that's the name that Haskell uses). In <a href="/2020/06/22/the-io-functor">the IO functor</a> article, I wrote that the <code>IO&lt;T&gt;</code> constructor corresponds to <em>return</em>. That's not strictly true, though, since the constructor takes a <code>Func&lt;T&gt;</code> and not a <code>T</code>. </p> <p> This issue is, however, trivially addressed: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#74531f;">Return</span>&lt;<span style="color:#2b91af;">T</span>&gt;(T&nbsp;<span style="color:#1f377f;">x</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;IO&lt;T&gt;(()&nbsp;=&gt;&nbsp;x); }</pre> </p> <p> Take the value <code>x</code> and wrap it in a lazily-evaluated function. </p> <h3 id="e0548aa855224081a5c1c3ec80f23951"> Laws <a href="#e0548aa855224081a5c1c3ec80f23951" title="permalink">#</a> </h3> <p> While <a href="/2020/07/06/referential-transparency-of-io">IO values are referentially transparent</a> you can't compare them. You also can't 'run' them by other means than running a program. This makes it hard to talk meaningfully about the <a href="/2022/04/11/monad-laws">monad laws</a>. </p> <p> For example, the left identity law is: </p> <p> <pre>return &gt;=&gt; h ≡ h</pre> </p> <p> Note the implied equality. The composition of <code>return</code> and <code>h</code> should be equal to <code>h</code>, for some reasonable definition of equality. How do we define that? </p> <p> Somehow we must imagine that two alternative compositions would produce the same observable effects <a href="https://en.wikipedia.org/wiki/Ceteris_paribus">ceteris paribus</a>. If you somehow imagine that you have two parallel universes, one with one composition (say <code>return &gt;=&gt; h</code>) and one with another (<code>h</code>), if all else in those two universes were equal, then you would observe no difference in behaviour. </p> <p> That may be useful as a thought experiment, but isn't particularly practical. Unfortunately, due to side effects, things <em>do</em> change when non-deterministic behaviour and side effects are involved. As a simple example, consider an IO action that gets the current time and prints it to the console. That involves both non-determinism and a side effect. </p> <p> In Haskell, that's a straightforward composition of two <code>IO</code> actions: </p> <p> <pre>&gt; h () = getCurrentTime &gt;>= print</pre> </p> <p> How do we compare two compositions? By running them? </p> <p> <pre>&gt; return () &gt;&gt;= h 2022-06-25 16:47:30.6540847 UTC &gt; h () 2022-06-25 16:47:37.5281265 UTC</pre> </p> <p> The outputs are not the same, because time goes by. Can we thereby conclude that the monad laws don't hold for IO? Not quite. </p> <p> The IO Container is referentially transparent, but evaluation isn't. Thus, we have to pretend that two alternatives will lead to the same evaluation behaviour, all things being equal. </p> <p> This property seems to hold for both the identity and associativity laws. Whether or not you compose with <em>return</em>, or in which evaluation order you compose actions, it doesn't affect the outcome. </p> <p> For completeness sake, the <a href="/2020/07/13/implementation-of-the-c-io-container">C# implementation sketch</a> is just a wrapper over a <code>Func&lt;T&gt;</code>. We can also think of such a function as a function from <a href="/2018/01/15/unit-isomorphisms">unit</a> to <code>T</code> - in pseudo-C# <code>() =&gt; T</code>. That's a function; in other words: <a href="/2022/11/14/the-reader-monad">The Reader monad</a>. We already know that the Reader monad obeys the monad laws, so the C# implementation, at least, should be okay. </p> <h3 id="74154d1a95a44a91a8d534fbfda59d8d"> Conclusion <a href="#74154d1a95a44a91a8d534fbfda59d8d" title="permalink">#</a> </h3> <p> IO forms a monad, among other abstractions. This is what enables Haskell programmers to compose an arbitrary number of impure actions with monadic bind without ever having to force evaluation. In C# <a href="/2020/06/15/io-container-in-a-parallel-c-universe">it might have looked the same</a>, except that it doesn't. </p> <p> <strong>Next:</strong> <a href="/2018/01/08/software-design-isomorphisms">Software design isomorphisms</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Adding NuGet packages when offline https://blog.ploeh.dk/2023/01/02/adding-nuget-packages-when-offline 2023-01-02T05:41:00+00:00 Mark Seemann <div id="post"> <p> <em>A fairly trivial technical detective story.</em> </p> <p> I was recently in an air plane, writing code, when I realised that I needed to add a couple of NuGet packages to my code base. I was on one of those less-travelled flights in Europe, on board an <a href="https://en.wikipedia.org/wiki/Embraer_E-Jet_family">Embraer E190</a>, and as is usually the case on those 1½-hour flights, there was no WiFi. </p> <p> Adding a NuGet package typically requires that you're online so that the tools can query the relevant NuGet repository. You'll need to download the package, so if you're offline, you're just out of luck, right? </p> <p> Fortunately, I'd previously used the packages I needed in other projects, on the same laptop. While <a href="/2014/01/29/nuget-package-restore-considered-harmful">I'm no fan of package restore</a>, I know that the local NuGet tools cache packages somewhere on the local machine. </p> <p> So, perhaps I could entice the tools to reuse a cached package... </p> <p> First, I simply tried adding a package that I needed: </p> <p> <pre>$ dotnet add package unquote Determining projects to restore... Writing C:\Users\mark\AppData\Local\Temp\tmpF3C.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET. info : Adding PackageReference for package 'unquote' into project '[redacted]'. error: Unable to load the service index for source https://api.nuget.org/v3/index.json. error: No such host is known. (api.nuget.org:443) error: No such host is known.</pre> </p> <p> Fine plan, but no success. </p> <p> Clearly the <code>dotnet</code> tool was trying to access <code>api.nuget.org</code>, which, obviously, couldn't be reached because my laptop was in flight mode. It occurred to me, though, that the reason that the tool was querying <code>api.nuget.org</code> was that it wanted to see which version of the package was the most recent. After all, I hadn't specified a version. </p> <p> What if I were to specify a version? Would the tool use the cached version of the package? </p> <p> That seemed worth a try, but which versions did I already have on my laptop? </p> <p> I don't go around remembering which version numbers I've used of various NuGet packages, but I expected the NuGet tooling to have that information available, somewhere. </p> <p> But where? Keep in mind that I was offline, so couldn't easily look this up. </p> <p> On the other hand, I knew that these days, most Windows applications keep data of that kind somewhere in <code>AppData</code>, so I started spelunking around there, looking for something that might be promising. </p> <p> After looking around a bit, I found a subdirectory named <code>AppData\Local\NuGet\v3-cache</code>. This directory contained a handful of subdirectories obviously named with GUIDs. Each of these contained a multitude of <code>.dat</code> files. The names of those files, however, looked promising: </p> <p> <pre>list_antlr_index.dat list_autofac.dat list_autofac.extensions.dependencyinjection.dat list_autofixture.automoq.dat list_autofixture.automoq_index.dat list_autofixture.automoq_range_2.0.0-3.6.7.dat list_autofixture.automoq_range_3.30.3-3.50.5.dat list_autofixture.automoq_range_3.50.6-4.17.0.dat list_autofixture.automoq_range_3.6.8-3.30.2.dat list_autofixture.dat ...</pre> </p> <p> and so on. </p> <p> These names were clearly(?) named <code>list_[package-name].dat</code> or <code>list_[package-name]_index.dat</code>, so I started looking around for one named after the package I was looking for (<a href="https://www.nuget.org/packages/Unquote/">Unquote</a>). </p> <p> Often, both files are present, which was also the case for Unquote. </p> <p> <pre>$ ls list_unquote* -l -rw-r--r-- 1 mark 197609 348 Oct 1 18:38 list_unquote.dat -rw-r--r-- 1 mark 197609 42167 Sep 23 21:29 list_unquote_index.dat</pre> </p> <p> As you can tell, <code>list_unquote_index.dat</code> is much larger than <code>list_unquote.dat</code>. Since I didn't know what the format of these files were, I decided to look at the smallest one first. It had this content: </p> <p> <pre>{ "versions": [ "1.3.0", "2.0.0", "2.0.1", "2.0.2", "2.0.3", "2.1.0", "2.1.1", "2.2.0", "2.2.1", "2.2.2", "3.0.0", "3.1.0", "3.1.1", "3.1.2", "3.2.0", "4.0.0", "5.0.0", "6.0.0-rc.1", "6.0.0-rc.2", "6.0.0-rc.3", "6.0.0", "6.1.0" ] }</pre> </p> <p> A list of versions. Sterling. It looked as though version 6.1.0 was the most recent one on my machine, so I tried to add that one to my code base: </p> <p> <pre>$ dotnet add package unquote --version 6.1.0 Determining projects to restore... Writing C:\Users\mark\AppData\Local\Temp\tmp815D.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET. info : Adding PackageReference for package 'unquote' into project '[redacted]'. info : Restoring packages for [redacted]... info : Package 'unquote' is compatible with all the specified frameworks in project '[redacted]'. info : PackageReference for package 'unquote' version '6.1.0' added to file '[redacted]'. info : Generating MSBuild file [redacted]. info : Writing assets file to disk. Path: [redacted] log : Restored [redacted] (in 397 ms).</pre> </p> <p> Jolly good! That worked. </p> <p> This way I managed to install all the NuGet packages I needed. This was fortunate, because I had so little time to transfer to my connecting flight that I never got to open the laptop before I was airborne again - in another E190 without WiFi, and another session of offline programming. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="7387859cda784fdf8612ddb34666d49b"> <div class="comment-author"><a href="https://github.com/asherber">Aaron Sherber</a></div> <div class="comment-content"> <p> A postscript to your detective story might note that the primary NuGet cache lives at <code>%userprofile%\.nuget\packages</code> on Windows and <code>~/.nuget/packages</code> on Mac and Linux. The folder names there are much easier to decipher than the folders and files in the http cache. </p> </div> <div class="comment-date">2023-01-02 13:46 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Functors as invariant functors https://blog.ploeh.dk/2022/12/26/functors-as-invariant-functors 2022-12-26T13:05:00+00:00 Mark Seemann <div id="post"> <p> <em>A most likely useless set of invariant functors that nonetheless exist.</em> </p> <p> This article is part of <a href="/2022/08/01/invariant-functors">a series of articles about invariant functors</a>. An invariant functor is a <a href="/2018/03/22/functors">functor</a> that is neither covariant nor contravariant. See the series introduction for more details. </p> <p> It turns out that all <a href="/2018/03/22/functors">functors</a> are also invariant functors. </p> <p> Is this useful? Let me be honest and say that if it is, I'm not aware of it. Thus, if you're interested in practical applications, you can stop reading here. This article contains nothing of practical use - as far as I can tell. </p> <h3 id="d748d07afe5342e9847e858849053911"> Because it's there <a href="#d748d07afe5342e9847e858849053911" title="permalink">#</a> </h3> <p> Why describe something of no practical use? </p> <p> Why do some people climb <a href="https://en.wikipedia.org/wiki/Mount_Everest">Mount Everest</a>? <em>Because it's there</em>, or for other irrational reasons. Which is fine. I've no personal goals that involve climbing mountains, but <a href="/2020/10/12/subjectivity">I happily engage in other irrational and subjective activities</a>. </p> <p> One of them, apparently, is to write articles of software constructs of no practical use, <em>because it's there</em>. </p> <p> All functors are also invariant functors, even if that's of no practical use. That's just the way it is. This article explains how, and shows a few (useless) examples. </p> <p> I'll start with a few <a href="https://www.haskell.org/">Haskell</a> examples and then move on to showing the equivalent examples in C#. If you're unfamiliar with Haskell, you can skip that section. </p> <h3 id="268965f94deb48d79335afea6e9b85e4"> Haskell package <a href="#268965f94deb48d79335afea6e9b85e4" title="permalink">#</a> </h3> <p> For Haskell you can find an existing definition and implementations in the <a href="https://hackage.haskell.org/package/invariant">invariant</a> package. It already makes most common functors <code>Invariant</code> instances, including <code>[]</code> (list), <code>Maybe</code>, and <code>Either</code>. Here's an example of using <code>invmap</code> with a small list: </p> <p> <pre>ghci&gt; invmap secondsToNominalDiffTime nominalDiffTimeToSeconds [0.1, 60] [0.1s,60s]</pre> </p> <p> Here I'm using the <a href="https://hackage.haskell.org/package/time">time</a> package to convert fixed-point decimals into <code>NominalDiffTime</code> values. </p> <p> How is this different from normal functor mapping with <code>fmap</code>? In observable behaviour, it's not: </p> <p> <pre>ghci&gt; fmap secondsToNominalDiffTime [0.1, 60] [0.1s,60s]</pre> </p> <p> When invariantly mapping a functor, only the covariant mapping function <code>a -&gt; b</code> is used. Here, that's <code>secondsToNominalDiffTime</code>. The contravariant mapping function <code>b -&gt; a</code> (<code>nominalDiffTimeToSeconds</code>) is simply ignored. </p> <p> While the <em>invariant</em> package already defines certain common functors as <code>Invariant</code> instances, every <code>Functor</code> instance can be converted to an <code>Invariant</code> instance. There are two ways to do that: <code>invmapFunctor</code> and <code>WrappedFunctor</code>. </p> <p> In order to demonstrate, we need a custom <code>Functor</code> instance. This one should do: </p> <p> <pre>data Pair a = Pair (a, a) deriving (Eq, Show, Functor)</pre> </p> <p> If you just want to perform an ad-hoc invariant mapping, you can use <code>invmapFunctor</code>: </p> <p> <pre>ghci&gt; invmapFunctor secondsToNominalDiffTime nominalDiffTimeToSeconds $ Pair (0.1, 60) Pair (0.1s,60s)</pre> </p> <p> I can't think of any reason to do this, but it's possible. </p> <p> <code>WrappedFunctor</code> is perhaps marginally more relevant. If you run into a function that takes an <code>Invariant</code> argument, you can convert any <code>Functor</code> to an <code>Invariant</code> instance by wrapping it in <code>WrappedFunctor</code>: </p> <p> <pre>ghci&gt; invmap secondsToNominalDiffTime nominalDiffTimeToSeconds $ WrapFunctor $ Pair (0.1, 60) WrapFunctor {unwrapFunctor = Pair (0.1s,60s)}</pre> </p> <p> A realistic, useful example still escapes me, but there it is. </p> <h3 id="2578032e937f4a68997f10b0e2412ffb"> Pair as an invariant functor in C# <a href="#2578032e937f4a68997f10b0e2412ffb" title="permalink">#</a> </h3> <p> What would the above Haskell example look like in C#? First, we're going to need a <code>Pair</code> data structure: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Pair</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Pair</span>(T&nbsp;x,&nbsp;T&nbsp;y) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;X&nbsp;=&nbsp;x; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Y&nbsp;=&nbsp;y; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;X&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;Y&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;follow...</span></pre> </p> <p> Making <code>Pair&lt;T&gt;</code> a functor is so easy that Haskell can do it automatically with the <code>DeriveFunctor</code> extension. In C# you must explicitly write the function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Pair&lt;T1&gt;&nbsp;Select&lt;<span style="color:#2b91af;">T1</span>&gt;(Func&lt;T,&nbsp;T1&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;T1&gt;(selector(X),&nbsp;selector(Y)); }</pre> </p> <p> An example equivalent to the above <code>fmap</code> example might be this, here expressed as a unit test: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;FunctorExample() { &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;/&nbsp;10, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;*&nbsp;60); &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;TimeSpan&gt;&nbsp;actual&nbsp;=&nbsp;sut.Select(ticks&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(ticks)); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;TimeSpan&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(.1), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(60)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;actual); }</pre> </p> <p> You can trivially make <code>Pair&lt;T&gt;</code> an invariant functor by giving it a function equivalent to <code>invmap</code>. As I outlined in <a href="/2022/08/01/invariant-functors">the introduction</a> it's possible to add an <code>InvMap</code> method to the class, but it might be more <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> to instead add a <code>Select</code> overload: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Pair&lt;T1&gt;&nbsp;Select&lt;<span style="color:#2b91af;">T1</span>&gt;(Func&lt;T,&nbsp;T1&gt;&nbsp;tToT1,&nbsp;Func&lt;T1,&nbsp;T&gt;&nbsp;t1ToT) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Select(tToT1); }</pre> </p> <p> Notice that this overload simply ignores the <code>t1ToT</code> argument and delegates to the normal <code>Select</code> overload. That's consistent with the Haskell package. This unit test shows an examples: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;InvariantFunctorExample() { &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;/&nbsp;10, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;*&nbsp;60); &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;TimeSpan&gt;&nbsp;actual&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sut.Select(ticks&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(ticks),&nbsp;ts&nbsp;=&gt;&nbsp;ts.Ticks); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;TimeSpan&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(.1), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(60)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;actual); }</pre> </p> <p> I can't think of a reason to do this in C#. In Haskell, at least, you have enough power of abstraction to describe something as simply an <code>Invariant</code> functor, and then let client code decide whether to use <code>Maybe</code>, <code>[]</code>, <code>Endo</code>, or a custom type like <code>Pair</code>. You can't do that in C#, so the abstraction is even less useful here. </p> <h3 id="8dbd7bd0f9c64fa5a0ac7c397617521b"> Conclusion <a href="#8dbd7bd0f9c64fa5a0ac7c397617521b" title="permalink">#</a> </h3> <p> All functors are invariant functors. You simply use the normal functor mapping function (<code>fmap</code> in Haskell, <code>map</code> in many other languages, <code>Select</code> in C#). This enables you to add an invariant mapping (<code>invmap</code>) that only uses the covariant argument (<code>a -&gt; b</code>) and ignores the contravariant argument (<code>b -&gt; a</code>). </p> <p> Invariant functors are, however, not particularly useful, so neither is this result. Still, it's there, so deserves a mention. The situation is similar for the next article. </p> <p> <strong>Next:</strong> Contravariant functors as invariant functors. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Error-accumulating composable assertions in C# https://blog.ploeh.dk/2022/12/19/error-accumulating-composable-assertions-in-c 2022-12-19T08:39:00+00:00 Mark Seemann <div id="post"> <p> <em>Perhaps the list monoid is all you need for non-short-circuiting assertions.</em> </p> <p> This article is the second instalment in a small articles series about <a href="/2022/11/07/applicative-assertions">applicative assertions</a>. It explores a way to compose assertions in such a way that failure messages accumulate rather than short-circuit. It assumes that you've read the <a href="/2022/11/07/applicative-assertions">article series introduction</a> and the <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previous article</a>. </p> <p> Unsurprisingly, the previous article showed that you can use an <a href="/2018/10/01/applicative-functors">applicative functor</a> to create composable assertions that don't short-circuit. It also concluded that, in C# at least, the API is awkward. </p> <p> This article explores a simpler API. </p> <h3 id="dc146cd5b792473db7f7e64cac3c4607"> A clue left by the proof of concept <a href="#dc146cd5b792473db7f7e64cac3c4607" title="permalink">#</a> </h3> <p> The previous article's proof of concept left a clue suggesting a simpler API. Consider, again, how the rather horrible <code>RunAssertions</code> method decides whether or not to throw an exception: </p> <p> <pre><span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty); <span style="color:blue;">if</span>&nbsp;(!<span style="color:blue;">string</span>.IsNullOrEmpty(errors)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(errors);</pre> </p> <p> Even though <code><span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;</code> is a <a href="https://en.wikipedia.org/wiki/Tagged_union">sum type</a>, the <code>RunAssertions</code> method declines to take advantage of that. Instead, it reduces <code>composition</code> to a simple type: A <code>string</code>. It then decides to throw an exception if the <code>errors</code> value is not null or empty. </p> <p> This suggests that using a sum type may not be necessary to distinguish between the success and the failure case. Rather, an empty error string is all it takes to indicate success. </p> <h3 id="801897c3ffc0455e8e7a7ac7531f2be3"> Non-empty errors <a href="#801897c3ffc0455e8e7a7ac7531f2be3" title="permalink">#</a> </h3> <p> The proof-of-concept assertion type is currently defined as <code>Validated</code> with a particular combination of type arguments: <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. Consider, again, this <code>Match</code> expression: </p> <p> <pre><span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty);</pre> </p> <p> Does an empty string unambiguously indicate success? Or is it possible to arrive at an empty string even if <code>composition</code> actually represents a failure case? </p> <p> You can arrive at an empty string from a failure case if the collection of error messages is empty. Consider the type argument that takes the place of the <code>F</code> generic type: <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. A collection of this type can be empty, which would also cause the above <code>Match</code> to produce an empty string. </p> <p> Even so, the proof-of-concept works in practice. The reason it works is that failure cases will never have empty assertion messages. We know this because (in the proof-of-concept code) only two functions produce assertions, and they each populate the error message collection with a string. You may want to revisit the <code>AssertTrue</code> and <code>AssertEqual</code> functions in the <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previous article</a> to convince yourself that this is true. </p> <p> This is a good example of knowledge that 'we' as developers know, but the code currently doesn't capture. Having to deal with such knowledge taxes your working memory, so why not <a href="/encapsulation-and-solid">encapsulate</a> such information in the type itself? </p> <p> How do you encapsulate the knowledge that a collection is never empty? Introduce a <code>NotEmptyCollection</code> collection. I'll reuse the class from the article <a href="/2017/12/11/semigroups-accumulate">Semigroups accumulate</a> and add a <code>Concat</code> instance method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;NotEmptyCollection&lt;T&gt;&nbsp;Concat(NotEmptyCollection&lt;T&gt;&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;T&gt;(Head,&nbsp;Tail.Concat(other).ToArray()); }</pre> </p> <p> Since the two assertion-producing functions both supply an error message in the failure case, it's trivial to change them to return <code>Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code> - just change the types used: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;condition, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;(message)); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertEqual&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;expected, &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Equals(expected,&nbsp;actual) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>)); }</pre> </p> <p> This change guarantees that the <code>RunAssertions</code> method only produces an empty <code>errors</code> string in success cases. </p> <h3 id="e5097dc5d94f4a1480eeaf483ac08336"> Error collection isomorphism <a href="#e5097dc5d94f4a1480eeaf483ac08336" title="permalink">#</a> </h3> <p> Assertions are still defined by the <code>Validated</code> sum type, but the <em>success</em> case carries no information: <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code>, and the <em>failure</em> case is always guaranteed to contain at least one error message. </p> <p> This suggests that a simpler representation is possible: One that uses a normal collection of errors, and where an empty collection indicates an absence of errors: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>()&nbsp;:&nbsp;<span style="color:blue;">this</span>(Array.Empty&lt;T&gt;()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>(T&nbsp;error)&nbsp;:&nbsp;<span style="color:blue;">this</span>(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;error&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>(IReadOnlyCollection&lt;T&gt;&nbsp;errors) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Errors&nbsp;=&nbsp;errors; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Asserted&lt;T&gt;&nbsp;And(Asserted&lt;T&gt;&nbsp;other) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(other)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;(Errors.Concat(other.Errors).ToList()); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;T&gt;&nbsp;Errors&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} }</pre> </p> <p> The <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> class is scarcely more than a glorified wrapper around a normal collection, but it's isomorphic to <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code>, which the following two functions prove: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;T&gt;&nbsp;FromValidated&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;&nbsp;v) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;v.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;failures&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;(failures), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;()); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;&nbsp;ToValidated&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;Asserted&lt;T&gt;&nbsp;a) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(a.Errors.Any()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;errors&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;T&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;a.Errors.First(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;a.Errors.Skip(1).ToArray()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Validated.Fail&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;(errors); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Validated.Succeed&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;(Unit.Value); }</pre> </p> <p> You can translate back and forth between <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code> and <code>Asserted&lt;T&gt;</code> without loss of information. </p> <p> A collection, however, gives rise to a <a href="/2017/10/06/monoids">monoid</a>, which suggests a much simpler way to compose assertions than using an applicative functor. </p> <h3 id="0ccfc89ad23f4b95932637723ff3c29a"> Asserted truth <a href="#0ccfc89ad23f4b95932637723ff3c29a" title="permalink">#</a> </h3> <p> You can now rewrite the assertion-producing functions to return <code>Asserted&lt;<span style="color:blue;">string</span>&gt;</code> instead of <code>Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;True(<span style="color:blue;">bool</span>&nbsp;condition,&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition&nbsp;?&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;()&nbsp;:&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(message); }</pre> </p> <p> This <code>Asserted.True</code> function returns no error messages when <code>condition</code> is <code>true</code>, but a collection with the single element <code>message</code> when it's <code>false</code>. </p> <p> You can use it in a unit test like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Asserted.True( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> You'll see how <code>assertResponse</code> composes with another assertion later in this article. The example continues from <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">the previous article</a>. It's the same test from the same code base. </p> <h3 id="519d7e0d757f4dceb6a2833c09d019d8"> Asserted equality <a href="#519d7e0d757f4dceb6a2833c09d019d8" title="permalink">#</a> </h3> <p> You can also rewrite the other assertion-producing function in the same way: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;Equal(<span style="color:blue;">object</span>&nbsp;expected,&nbsp;<span style="color:blue;">object</span>&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Equals(expected,&nbsp;actual)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>); }</pre> </p> <p> Again, when the assertion passes, it returns no errors; otherwise, it returns a collection with a single error message. </p> <p> Using it may look like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode);</pre> </p> <p> At this point, each of the assertions are objects that represent a verification step. By themselves, they neither pass nor fail the test. You have to execute them to reach a verdict. </p> <h3 id="29825aed26124471803324863ea7d897"> Evaluating assertions <a href="#29825aed26124471803324863ea7d897" title="permalink">#</a> </h3> <p> The above code listing of the <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> class already shows how to combine two <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> objects into one. The <code>And</code> instance method is a binary operation that, together with the parameterless constructor, makes <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> a <a href="/2017/10/06/monoids">monoid</a>. </p> <p> Once you've combined all assertions into a single <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> object, you need to <code>Run</code> it to produce a test outcome: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Run(<span style="color:blue;">this</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;assertions) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(assertions?.Errors.Any()&nbsp;??&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;messages&nbsp;=&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;assertions.Errors); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(messages); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> If there are no errors, <code>Run</code> does nothing; otherwise it combines all the error messages together and throws an exception. As was also the case in the previous article, I've allowed myself a few proof-of-concept shortcuts. <a href="https://learn.microsoft.com/dotnet/standard/design-guidelines/using-standard-exception-types">The framework design guidelines admonishes against throwing System.Exception</a>. It might be more appropriate to introduce a new <code>Exception</code> type that also allows enumerating the error messages. </p> <p> The entire <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">assertion phase</a> of the test looks like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Asserted.True( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); <span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); assertResponse.And(assertState).Run();</pre> </p> <p> You can see the entire test in the previous article. Notice how the two assertion objects are first combined into one with the <code>And</code> binary operation. The result is a single <code>Asserted&lt;<span style="color:blue;">string</span>&gt;</code> object on which you can call <code>Run</code>. </p> <p> Like the previous proof of concept, this assertion passes and fails in the same way. It's possible to compose assertions and collect error messages, instead of short-circuiting on the first failure, even without an applicative functor. </p> <h3 id="f3d9baf1ad564c05be99b32f0b4a1017"> Method chaining <a href="#f3d9baf1ad564c05be99b32f0b4a1017" title="permalink">#</a> </h3> <p> If you don't like to come up with variable names just to make assertions, it's also possible to use the <code>Asserted</code> API's <a href="https://martinfowler.com/bliki/FluentInterface.html">fluent interface</a>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); Asserted &nbsp;&nbsp;&nbsp;&nbsp;.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;.And(Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode)) &nbsp;&nbsp;&nbsp;&nbsp;.Run();</pre> </p> <p> This isn't necessarily better, but it's an option. </p> <h3 id="ecabe91c2b9949edbcb4cfed5213ec72"> Conclusion <a href="#ecabe91c2b9949edbcb4cfed5213ec72" title="permalink">#</a> </h3> <p> While it's possible to design non-short-circuiting composable assertions using an applicative functor, it looks as though a simpler solution might solve the same problem. Collect error messages. If none were collected, interpret that as a success. </p> <p> As I wrote in the <a href="/2022/11/07/applicative-assertions">introduction article</a>, however, this may not be the last word. Some assertions return values that can be used for other assertions. That's a scenario that I have not yet investigated in this light, and it may change the conclusion. If so, I'll add more articles to this small article series. As I'm writing this, though, I have no such plans. </p> <p> Did I just, in a roundabout way, write that <em>more research is needed?</em> </p> <p> <strong>Next:</strong> <a href="/2023/01/30/built-in-alternatives-to-applicative-assertions">Built-in alternatives to applicative assertions</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="2aae9a7f4c1145aa92b487925f4b46ba"> <div class="comment-author"><a href="https://ptupitsyn.github.io/">Pavel Tupitsyn</a></div> <div class="comment-content"> <p> I think NUnit's <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">Assert.Multiple</a> is worth mentioning in this series. It does not require any complicated APIs, just wrap your existing test with multiple asserts into a delegate. </p> </div> <div class="comment-date">2022-12-20 08:02 UTC</div> </div> <div class="comment" id="5b34eb2d997f417bac27738ee7cbb16d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Pavel, thank you for writing. I'm aware of both that API and similar ones for other testing frameworks. As is usually the case, there are trade-offs to consider. I'm currently working on some material that may turn into another article about that. </p> </div> <div class="comment-date">2022-12-21 20:17 UTC</div> </div> <div class="comment" id="f244b81042414b98be67a937cf652648"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> A new article is now available: <a href="/2023/01/30/built-in-alternatives-to-applicative-assertions">Built-in alternatives to applicative assertions</a>. </p> </div> <div class="comment-date">2023-01-30 12:31 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. When do tests fail? https://blog.ploeh.dk/2022/12/12/when-do-tests-fail 2022-12-12T08:33:00+00:00 Mark Seemann <div id="post"> <p> <em>Optimise for the common scenario.</em> </p> <p> Unit tests occasionally fail. When does that happen? How often? What triggers it? What information is important when tests fail? </p> <p> Regularly I encounter the viewpoint that it should be easy to understand the purpose of a test <em>when it fails</em>. Some people consider test names important, a topic that <a href="/2022/06/13/some-thoughts-on-naming-tests">I've previously discussed</a>. Recently I discussed the <a href="http://xunitpatterns.com/Assertion%20Roulette.html">Assertion Roulette</a> test smell on Twitter, and again I learned some surprising things about what people value in unit tests. </p> <h3 id="6673d3ef3f374f849fe7fe38a7b96985"> The importance of clear assertion messages <a href="#6673d3ef3f374f849fe7fe38a7b96985" title="permalink">#</a> </h3> <p> The Assertion Roulette test smell is often simplified to degeneracy, but it really describes situations where it may be a problem if you can't tell which of several assertions actually caused a test to fail. </p> <p> <a href="https://www.joshka.net">Josh McKinney</a> gave a more detailed example than Gerard Meszaros does in <a href="/ref/xunit-patterns">the book</a>: </p> <blockquote> <p> "Background. In a legacy product, we saw some tests start failing intermittently. They weren’t just flakey, but also failed without providing enough info to fix. One of things which caused time to fix to increase was multiple ways of a single test to fail." </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572528796125003777">Josh McK</a></cite></footer> </blockquote> <p> He goes on: </p> <blockquote> <p> "I.e. if you fix the first assertion and you know there still could be flakiness, or long cycle times to see the failure. Multiple assertions makes any test problem worse. In an ideal state, they are fine, but every assertion doubles the amount of failures a test catches." </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572529894361534464">Josh McK</a></cite></footer> </blockquote> <p> and concludes: </p> <blockquote> <p> "the other main way (unrelated) was things like: </p> <p> assertTrue(someListResult.isRmpty()) </p> <p> Which tells you what failed, but nothing about how. </p> <p> But the following is worse. You must run the test twice to fix: </p> <p> assertFalse(someList.isEmpty());<br/> assertEqual(expected, list.get(0));" </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572534297403469824">Josh McK</a></cite></footer> </blockquote> <p> The final point is due to the short-circuiting nature of most assertion libraries. That, however, is <a href="/2022/11/07/applicative-assertions">a solvable problem</a>. </p> <p> I find the above a compelling example of why Assertion Roulette may be problematic. </p> <p> It did give me pause, though. How common is this scenario? </p> <h3 id="5ae99f4515444bd6ba483beffb819557"> Out of the blue <a href="#5ae99f4515444bd6ba483beffb819557" title="permalink">#</a> </h3> <p> The situation described by Josh McKinney comes with more than a single warning flag. I hope that it's okay to point some of them out. I didn't get the impression from my interaction with Josh McKinney that he considered the situation ideal in any way. </p> <p> First, of course, there's the lack of information about the problem. Here, that's a real problem. As I understand it, it makes it harder to reproduce the problem in a development environment. </p> <p> Next, there's long cycle times, which I interpret as significant time may pass from when you attempt a fix until you can actually observe whether or not it worked. Josh McKinney doesn't say how long, but I wouldn't surprised if it was measured in days. At least, if the cycle time is measured in days, I can see how this is a problem. </p> <p> Finally, there's the observation that "some tests start failing intermittently". This was the remark that caught my attention. How often does that happen? </p> <p> Tests shouldn't do that. Tests should be deterministic. If they're not, you should work to <a href="https://martinfowler.com/articles/nonDeterminism.html">eradicate non-determinism in tests</a>. </p> <p> I'll be the first to admit that that I also write non-deterministic tests. Not by design, but because I make mistakes. I've written many <a href="http://xunitpatterns.com/Erratic%20Test.html">Erratic Tests</a> in my career, and I've documented a few of them here: </p> <ul> <li><a href="/2021/01/11/waiting-to-happen">Waiting to happen</a></li> <li><a href="/2022/05/23/waiting-to-never-happen">Waiting to never happen</a></li> <li><a href="/2020/10/05/fortunately-i-dont-squash-my-commits">Fortunately, I don't squash my commits</a></li> <li><a href="/2016/01/18/make-pre-conditions-explicit-in-property-based-tests">Make pre-conditions explicit in Property-Based Tests</a></li> </ul> <p> While it <em>can</em> happen, it shouldn't be the norm. When it nonetheless happens, eradicating that source of non-determinism should be top priority. Pull the <a href="https://en.wikipedia.org/wiki/Andon_(manufacturing)">andon cord</a>. </p> <h3 id="5fd251665441473ca3048a9137f9bc2b"> When tests fail <a href="#5fd251665441473ca3048a9137f9bc2b" title="permalink">#</a> </h3> <p> Ideally, tests should rarely fail. As examined above, you may have Erratic Tests in your test suite, and if you do, these tests will occasionally (or often) fail. As Martin Fowler writes, this is a problem and you should do something about it. He also outlines strategies for it. </p> <p> Once you've eradicated non-determinism in unit tests, then when do tests fail? </p> <p> I can think of a couple of situations. </p> <p> Tests routinely fail as part of the <a href="/2019/10/21/a-red-green-refactor-checklist">red-green-refactor cycle</a>. This is by design. If no test is failing in the <em>red</em> phase, you probably made a mistake (which also regularly <a href="/2019/10/14/tautological-assertion">happens to me</a>), or you may not really be doing test-driven development (TDD). </p> <p> Another situation that may cause a test to fail is if you changed some code and triggered a regression test. </p> <p> In both cases, tests don't just fail <a href="https://amzn.to/3SPdHAO">out of the blue</a>. They fail as an immediate consequence of something you did. </p> <h3 id="aaeebfa1a96b47398219817bc3327a9c"> Optimise for the common scenario <a href="#aaeebfa1a96b47398219817bc3327a9c" title="permalink">#</a> </h3> <p> In both cases you're (hopefully) in a tight feedback loop. If you're in a tight feedback loop, then how important is the assertion message really? How important is the test name? </p> <p> You work on the code base, make some changes, run the tests. If one or more tests fail, it's correlated to the change you just made. You should have a good idea of what went wrong. Are code forensics and elaborate documentation really necessary to understand a test that failed because you just did something a few minutes before? </p> <p> The reason I don't care much about test names or whether there's one or more assertion in a unit test is exactly that: When tests fail, it's usually because of something I just did. I don't need diagnostics tools to find the root cause. The root cause is the change that I just made. </p> <p> That's my common scenario, and I try to optimise my processes for the common scenarios. </p> <h3 id="21b16c9b2fb649ec859e53b0a1ab431a"> Fast feedback <a href="#21b16c9b2fb649ec859e53b0a1ab431a" title="permalink">#</a> </h3> <p> There's an implied way of working that affects such attitudes. Since I learned about TDD in 2003 I've always relished the fast feedback I get from a test suite. Since I tried continuous deployment around 2014, I consider it central to <a href="/ref/modern-software-engineering">modern software engineering</a> (and <a href="/ref/accelerate">Accelerate</a> strongly suggests so, too). </p> <p> The modus operandi I outline above is one of fast feedback. If you're sitting on a feature branch for weeks before integrating into master, or if you can only deploy two times a year, this influences what works and what doesn't. </p> <p> Both <em>Modern Software Engineering</em> and <em>Accelerate</em> make a strong case that short feedback cycles are pivotal for successful software development organisations. </p> <p> I also understand that that's not the reality for everyone. When faced with long cycle times, a multitude of Erratic Tests, a legacy code base, and so on, other things become important. In those circumstances, tests may fail for different reasons. </p> <p> When you work with TDD, continuous integration (CI), and continuous deployment (CD), then when do tests fail? They fail because you made them fail, only minutes earlier. Fix your code and move forward. </p> <h3 id="53cc6ce19bad4bdea3f40fd752f6338d"> Conclusion <a href="#53cc6ce19bad4bdea3f40fd752f6338d" title="permalink">#</a> </h3> <p> When discussing test names and assertion messages, I've been surprised by the emphasis some people put on what I consider to be of secondary importance. I think the explanation is that circumstances differ. </p> <p> With TDD and CI/CD you mostly look at a unit test when you write it, or if some regression test fails because you changed some code (perhaps in response to a test you just wrote). Your test suite may have hundreds or thousands of tests. Most of these pass every time you run the test suite. That's the normal state of affairs. </p> <p> In other circumstances, you may have Erratic Tests that fail unpredictably. You should make it a priority to stop that, but as part of that process, you may need good assertion messages and good test names. </p> <p> Different circumstances call for different reactions, so what works well in one situation may be a liability in other situations. I hope that this article has shed a little light on the forces you may want to consider. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. GitHub Copilot preliminary experience report https://blog.ploeh.dk/2022/12/05/github-copilot-preliminary-experience-report 2022-12-05T08:37:00+00:00 Mark Seemann <div id="post"> <p> <em>Based on a few months of use.</em> </p> <p> I've been evaluating <a href="https://github.com/features/copilot">GitHub Copilot</a> since August 2022. Perhaps it's time to collect my thoughts so far. </p> <p> In short, it's surprisingly good, but also gets a lot of things wrong. It does seem helpful to the experienced programmer, but I don't see it replacing all programmers yet. </p> <h3 id="19d8478cc2d5429f9c62138574a30a45"> Not only for boilerplate code <a href="#19d8478cc2d5429f9c62138574a30a45" title="permalink">#</a> </h3> <p> I was initially doubtful. I'd seen some demos where Copilot created fifteen to twenty lines of code to, say, make a REST API request. These examples mostly struck me as auto-generation of something that ought to be a proper abstraction: A method in a reusable library. </p> <p> In general <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">I don't think that typing is a productivity bottleneck</a>, and I'm <a href="/2013/02/04/BewareofProductivityTools">sceptical of productivity tools</a>, and particularly code generators. The more code a code base contains, the more code there is to read. Accelerating code production doesn't strike me as a goal in itself. </p> <p> On the other hand, I'm past fifty and aware that in my age, one often tends to resist new ideas. Perhaps the above justifications are nothing but <a href="https://en.wikipedia.org/wiki/Motivated_reasoning">motivated reasoning</a>. I decided to install the plug-ins and go about my business to see what would happen. </p> <p> It has surprised and delighted me more than once. It's not only for boilerplate code. Here's a recent example. </p> <p> I was doing an exercise in <a href="https://fsharp.org">F#</a> and wrote this type definition: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Instruction&nbsp;=&nbsp;Enqueue&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;|&nbsp;Dequeue</pre> </p> <p> I wanted to write a property-based test with this type, and since I was using <a href="https://hedgehog.qa/">Hedgehog</a> I needed a generator for the type. While <a href="https://fscheck.github.io/FsCheck/">FsCheck</a> can automatically generate values of custom <a href="https://learn.microsoft.com/dotnet/fsharp/language-reference/discriminated-unions">discriminated unions</a>, Hedgehog has a more <a href="https://peps.python.org/pep-0020/">explicit-is-better-than-implicit</a> design philosophy. In other words, if there's a way to make Hedgehog automatically generate values of a type like the above <code>Instruction</code>, it has so far escaped me. </p> <p> So I started typing <code><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;genInstruction&nbsp;=</code> after which Copilot suggested the following: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;genInstruction&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Gen.frequency &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[&nbsp;1,&nbsp;Gen.constant&nbsp;Dequeue &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1,&nbsp;Gen.int32&nbsp;(Range.constant&nbsp;0&nbsp;10_000)&nbsp;|&gt;&nbsp;Gen.map&nbsp;Enqueue&nbsp;]</pre> </p> <p> I was, frankly, stunned. While I do understand that Copilot doesn't actually understand the code it suggests, Hedgehog is sufficiently esoteric that I didn't expect Copilot to have enough training data to enable it to make a useful contribution in this niche. I was wrong. I've repeatedly seen Copilot make suggestions to my F# and <a href="https://www.haskell.org/">Haskell</a> code. It's not just for C#, <a href="https://www.javascript.com">JavaScript</a>, or <a href="https://www.python.org/">python</a> code. </p> <p> The above suggestion was, to be clear, absolutely appropriate and compiled right away. The only detail I decided to change was the <code>Range</code>, which I decided to change to <code>Range.linear</code>. That's not, however, a significant change. </p> <p> Perhaps you're not impressed by three lines of auto-generated code. How much of a productivity improvement is that? Quite a bit, in my case. </p> <p> It wouldn't have taken me long to type those three lines of code, but as I already mentioned, <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">typing isn't a bottleneck</a>. On the other hand, looking up an unfamiliar API can take some time. <a href="/ref/programmers-brain">The Programmer's Brain</a> discusses this kind of problem and suggests exercises to address it. Does Copilot offer a shortcut? </p> <p> While I couldn't remember the details of Hedgehog's API, once I saw the suggestion, I recognised <code>Gen.frequency</code>, so I understood it as an appropriate code suggestion. The productivity gain, if there is one, may come from saving you the effort of looking up unfamiliar APIs, rather than saving you some keystrokes. </p> <p> In this example, I already knew of the <code>Gen.frequency</code> function - I just couldn't recall the exact name and type. This enabled me to evaluate Copilot's suggestion and deem it correct. If I hadn't known that API already, how could I have known whether to trust Copilot? </p> <h3 id="424f6a4928d945d7bcb13f2de8f3a98e"> Detectably wrong suggestions <a href="#424f6a4928d945d7bcb13f2de8f3a98e" title="permalink">#</a> </h3> <p> As amazing as Copilot can be, it's hardly faultless. It makes many erroneous suggestions. Sometimes the suggestion is obviously wrong. If you accept it, it doesn't compile. Sometimes, the compilation error is only a little edit from being correct, but at least in such situations you'll be explicitly aware that the suggestion couldn't be used verbatim. </p> <p> Other suggestions are wrong, but less conspicuously so. Here's an example. </p> <p> I was recently subjecting the code base that accompanies <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a> to the <a href="https://en.wikipedia.org/wiki/Mutation_testing">mutation testing</a> tool <a href="https://stryker-mutator.io/">Stryker</a>. Since it did point out a few possible mutations, I decided to add a few tests. One was of a wrapper class called <code>TimeOfDay</code>. Because of static code analysis rules, it came with conversions to and from <a href="https://learn.microsoft.com/dotnet/api/system.timespan">TimeSpan</a>, but these methods weren't covered by any tests. </p> <p> In order to remedy that situation, I started writing an FsCheck property and came as far as: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>)</pre> </p> <p> At that point Copilot suggested the following, which I accepted: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">timeOfDay</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TimeOfDay(timeSpan); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;(TimeSpan)timeOfDay; &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(timeSpan,&nbsp;actual); }</pre> </p> <p> Looks good, doesn't it? Again, I was impressed. It compiled, and it even looks as though Copilot had picked up one of my naming conventions: <a href="/2020/11/30/name-by-role">naming variables by role</a>, in this case <code>actual</code>. </p> <p> While I tend to be on guard, I immediately ran the test suite instead of thinking it through. It failed. Keep in mind that this is a <a href="https://en.wikipedia.org/wiki/Characterization_test">characterisation test</a>, so it was supposed to pass. </p> <p> The <code>TimeOfDay</code> constructor reveals why: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TimeOfDay</span>(TimeSpan&nbsp;<span style="color:#1f377f;">durationSinceMidnight</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(durationSinceMidnight&nbsp;&lt;&nbsp;TimeSpan.Zero&nbsp;|| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromHours(24)&nbsp;&lt;&nbsp;durationSinceMidnight) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentOutOfRangeException( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nameof(durationSinceMidnight), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Please&nbsp;supply&nbsp;a&nbsp;TimeSpan&nbsp;between&nbsp;0&nbsp;and&nbsp;24&nbsp;hours.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.durationSinceMidnight&nbsp;=&nbsp;durationSinceMidnight; }</pre> </p> <p> While FsCheck knows how to generate <code>TimeSpan</code> values, it'll generate arbitrary durations, including negative values and spans much longer than 24 hours. That explains why the test fails. </p> <p> Granted, this is hardly a searing indictment against Copilot. After all, I could have made this mistake myself. </p> <p> Still, that prompted me to look for more issues with the code that Copilot had suggested. Another problem with the code is that it tests the wrong API. The suggested test tries to round-trip via the <code>TimeOfDay</code> class' explicit cast operators, which were already covered by tests. Well, I might eventually have discovered that, too. Keep in mind that I was adding this test to improve the code base's Stryker score. After running the tool again, I would probably eventually have discovered that the score didn't improve. It takes Stryker around 25 minutes to test this code base, though, so it wouldn't have been rapid feedback. </p> <p> Since, however, I examined the code with a critical eye, I noticed this by myself. This would clearly require changing the test code as well. </p> <p> In the end, I wrote this test: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">expected</span>&nbsp;=&nbsp;ScaleToTimeOfDay(timeSpan); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">sut</span>&nbsp;=&nbsp;TimeOfDay.ToTimeOfDay(expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;TimeOfDay.ToTimeSpan(sut); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TimeSpan&nbsp;<span style="color:#74531f;">ScaleToTimeOfDay</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Convert&nbsp;an&nbsp;arbitrary&nbsp;TimeSpan&nbsp;to&nbsp;a&nbsp;24-hour&nbsp;TimeSpan.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;The&nbsp;data&nbsp;structure&nbsp;that&nbsp;underlies&nbsp;TimeSpan&nbsp;is&nbsp;a&nbsp;64-bit&nbsp;integer,</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;so&nbsp;first&nbsp;we&nbsp;need&nbsp;to&nbsp;identify&nbsp;the&nbsp;range&nbsp;of&nbsp;possible&nbsp;TimeSpan</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;values.&nbsp;It&nbsp;might&nbsp;be&nbsp;easier&nbsp;to&nbsp;understand&nbsp;to&nbsp;calculate</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;TimeSpan.MaxValue&nbsp;-&nbsp;TimeSpan.MinValue,&nbsp;but&nbsp;that&nbsp;underflows.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Instead,&nbsp;the&nbsp;number&nbsp;of&nbsp;possible&nbsp;64-bit&nbsp;integer&nbsp;values&nbsp;is&nbsp;the&nbsp;same</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;as&nbsp;the&nbsp;number&nbsp;of&nbsp;possible&nbsp;unsigned&nbsp;64-bit&nbsp;integer&nbsp;values.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">range</span>&nbsp;=&nbsp;<span style="color:blue;">ulong</span>.MaxValue; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">domain</span>&nbsp;=&nbsp;TimeSpan.FromHours(24).Ticks; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">scale</span>&nbsp;=&nbsp;(<span style="color:blue;">ulong</span>)domain&nbsp;/&nbsp;range; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">expected</span>&nbsp;=&nbsp;timeSpan&nbsp;*&nbsp;scale; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;expected; }</pre> </p> <p> In this case, Copilot didn't improve my productivity. It may actually have slowed me down a bit. </p> <p> This time, it wasn't too difficult to spot issues with the suggested code. What if the problems are more subtle? </p> <h3 id="02cb23fc57394a1c963a2f4ebe75ec48"> Errors that are difficult to detect <a href="#02cb23fc57394a1c963a2f4ebe75ec48" title="permalink">#</a> </h3> <p> How do bugs appear? We write them, thinking that our code is going to do one thing, while the compiler decides otherwise. Even when we actively engage with the code, we may be looking at a bug and still fail to see it. </p> <p> Why should we trust Copilot to produce bug-free code? </p> <p> Here's another example. I was going through the <a href="https://codingdojo.org/kata/Range/">Range kata</a> in F# and made my <code>Endpoint</code> discriminated union a <a href="/2018/03/22/functors">functor</a>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Endpoint&lt;&#39;a&gt;&nbsp;=&nbsp;Open&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;a&nbsp;|&nbsp;Closed&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;a <span style="color:blue;">module</span>&nbsp;Endpoint&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;map&nbsp;f&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Open&nbsp;&nbsp;&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Open&nbsp;&nbsp;&nbsp;(f&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Closed&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Closed&nbsp;(f&nbsp;x)</pre> </p> <p> Making a type a functor is such a well-understood process that <a href="https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/derive-functor">Haskell can automate it</a>. I knew how to write the above <code>map</code> function as well, so I wrote it without testing it. </p> <p> After I'd written the <code>map</code> function, I decided to add tests of the two functor laws. Not because I doubted the implementation, but because I might need that as example code later. </p> <p> Copilot suggested this test after I named it: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Xunit <span style="color:blue;">open</span>&nbsp;Hedgehog [&lt;Fact&gt;] <span style="color:blue;">let</span>&nbsp;``First&nbsp;functor&nbsp;law``&nbsp;()&nbsp;=&nbsp;Property.check&nbsp;&lt;|&nbsp;property&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&nbsp;=&nbsp;Gen.int32&nbsp;(Range.linearBounded&nbsp;()) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;f&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;+&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;g&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;*&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Endpoint.map&nbsp;f&nbsp;(Endpoint.map&nbsp;g&nbsp;(Open&nbsp;x)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;Endpoint.map&nbsp;(f&nbsp;&gt;&gt;&nbsp;g)&nbsp;(Open&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual)&nbsp;}</pre> </p> <p> It compiles. It also fails. </p> <p> What's wrong with it? </p> <p> If you know the functor laws by heart, you'll be able to spot that this is clearly not the first functor law. On the other hand, it looks convincingly like the second functor law. Should I just change the name and move on? </p> <p> I can't, though, since the test fails. Could there be a bug in my <code>map</code> function, after all? </p> <p> No, there's an error in the test. I invite you to spot it. </p> <p> In terms of keystrokes, it's easy to fix the problem: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Xunit <span style="color:blue;">open</span>&nbsp;Hedgehog [&lt;Fact&gt;] <span style="color:blue;">let</span>&nbsp;``First&nbsp;functor&nbsp;law``&nbsp;()&nbsp;=&nbsp;Property.check&nbsp;&lt;|&nbsp;property&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&nbsp;=&nbsp;Gen.int32&nbsp;(Range.linearBounded&nbsp;()) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;f&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;+&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;g&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;*&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Endpoint.map&nbsp;f&nbsp;(Endpoint.map&nbsp;g&nbsp;(Open&nbsp;x)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;Endpoint.map&nbsp;(f&nbsp;&lt;&lt;&nbsp;g)&nbsp;(Open&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual)&nbsp;}</pre> </p> <p> Spot the edit. I bet it'll take you longer to find it than it took me to type it. </p> <p> The test now passes, but for one who has spent less time worrying over functor laws than I have, troubleshooting this could have taken a long time. </p> <p> These almost-right suggestions from Copilot both worry me and give me hope. </p> <h3 id="7c90d2bd03054906a5f2505baaf78a31"> Copilot for experienced programmers <a href="#7c90d2bd03054906a5f2505baaf78a31" title="permalink">#</a> </h3> <p> When a new technology like Copilot appears, it's natural to speculate on the consequences. <em>Does this mean that programmers will lose their jobs?</em> </p> <p> This is just a preliminary evaluation after a few months, so I could be wrong, but I think we programmers are safe. If you're experienced, you'll be able to tell most of Copilot's hits from its misses. Perhaps you'll get a productivity improvement out of, but it could also slow you down. </p> <p> The tool is likely to improve over time, so I'm hopeful that this could become a net productivity gain. Still, with this high an error rate, I'm not too worried yet. </p> <p> <a href="/ref/pragmatic-programmer">The Pragmatic Programmer</a> describes a programming style named <em>Programming by Coincidence</em>. People who develop software this way have only a partial understanding of the code they write. </p> <blockquote> <p> "Fred doesn't know why the code is failing because <em>he didn't know why it worked in the first place.</em>" </p> <footer><cite>Andy Hunt and Dave Thomas, <a href="/ref/pragmatic-programmer">The Pragmatic Programmer</a></cite></footer> </blockquote> <p> I've encountered my fair share of these people. When editing code, they make small adjustments and do cursory manual testing until 'it looks like it works'. If they have to start a new feature or are otherwise faced with a metaphorical blank page, they'll copy some code from somewhere else and use that as a starting point. </p> <p> You'd think that Copilot could enhance the productivity of such people, but I'm not sure. It might actually slow them down. These people don't fully understand the code they themselves 'write', so why should we expect them to understand the code that Copilot suggests? </p> <p> If faced with a Copilot suggestion that 'almost works', will they be able to spot if it's a genuinely good suggestion, or whether it's off, like I've described above? If the Copilot code doesn't work, how much time will they waste thrashing? </p> <h3 id="f3e17cbf9dbc41f19ae46974d2f28a90"> Conclusion <a href="#f3e17cbf9dbc41f19ae46974d2f28a90" title="permalink">#</a> </h3> <p> GitHub Copilot has the potential to be a revolutionary technology, but it's not, yet. So far, I'm not too worried. It's an assistant, like a pairing partner, but it's up to you to evaluate whether the code that Copilot suggests is useful, correct, and safe. How can you do that unless you already know what you're doing? </p> <p> If you don't have the qualifications to evaluate the suggested code, I fail to see how it's going to help you. Granted, it does have potential to help you move on in less time that you would otherwise have spent. In this article, I showed one example where I would have had to spend significant time looking up API documentation. Instead, Copilot suggested the correct code to use. </p> <p> Pulling in the other direction are the many <a href="https://en.wikipedia.org/wiki/False_positives_and_false_negatives">false positives</a>. Copilot makes many suggestions, and many of them are poor. The ones that are recognisably bad are unlikely to slow you down. I'm more concerned with those that are subtly wrong. They have the potential to waste much time. </p> <p> Which of these forces are strongest? The potential for wasting time is infinite, while the maximum productivity gain you can achieve is 100 percent. That's an asymmetric distribution. There's a long tail of time wasters, but there's no equivalent long tail of improvement. </p> <p> I'm not, however, trying to be pessimistic. I expect to keep Copilot around for the time being. It could very well be here to stay. Used correctly, it seems useful. </p> <p> Is it going to replace programmers? Hardly. Rather, it may enable poor developers to make such a mess of things that you need even more good programmers to subsequently fix things. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An initial proof of concept of applicative assertions in C# https://blog.ploeh.dk/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c 2022-11-28T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Worthwhile? Not obviously.</em> </p> <p> This article is the first instalment in a small articles series about <a href="/2022/11/07/applicative-assertions">applicative assertions</a>. It explores a way to compose assertions in such a way that failure messages accumulate rather than short-circuit. It assumes that you've read the <a href="/2022/11/07/applicative-assertions">article series introduction</a>. </p> <p> Assertions are typically based on throwing exceptions. As soon as one assertion fails, an exception is thrown and no further assertions are evaluated. This is normal short-circuiting behaviour of exceptions. In some cases, however, it'd be useful to keep evaluating other assertions and collect error messages. </p> <p> This article series explores <a href="https://twitter.com/lucasdicioccio/status/1572264819109003265">an intriguing idea</a> to address such issues: Use an <a href="/2018/10/01/applicative-functors">applicative functor</a> to collect multiple assertion messages. I started experimenting with the idea to see where it would lead. The article series serves as a report of what I found. It is neither a recommendation nor a caution. I still find the idea interesting, but I'm not sure whether the complexity is warranted. </p> <h3 id="877c6f5ee5b34ecf990d1126b8139720"> Example scenario <a href="#877c6f5ee5b34ecf990d1126b8139720" title="permalink">#</a> </h3> <p> A realistic example is often illustrative, although there's a risk that the realism carries with it some noise that detracts from the core of the matter. I'll reuse an example that I've <a href="https://stackoverflow.blog/2022/11/03/multiple-assertions-per-test-are-fine/">already discussed and explained in greater detail</a>. The code is from the code base that accompanies my book <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>. </p> <p> This test has two independent assertions: </p> <p> <pre>[Theory] [InlineData(884,&nbsp;18,&nbsp;47,&nbsp;<span style="color:#a31515;">&quot;c@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Nick&nbsp;Klimenko&quot;</span>,&nbsp;2)] [InlineData(902,&nbsp;18,&nbsp;50,&nbsp;<span style="color:#a31515;">&quot;emot@example.gov&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Emma&nbsp;Otting&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;DeleteReservation( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days,&nbsp;<span style="color:blue;">int</span>&nbsp;hours,&nbsp;<span style="color:blue;">int</span>&nbsp;minutes,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;LegacyApi(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;DateTime.Today.AddDays(days).At(hours,&nbsp;minutes) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToIso8601DateTimeString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;Create.ReservationDto(at,&nbsp;email,&nbsp;name,&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;postResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.PostReservation(dto); &nbsp;&nbsp;&nbsp;&nbsp;Uri&nbsp;address&nbsp;=&nbsp;FindReservationAddress(postResp); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().DeleteAsync(address); &nbsp;&nbsp;&nbsp;&nbsp;Assert.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); }</pre> </p> <p> The test exercises the REST API to first create a reservation, then delete it, and finally check that the reservation no longer exists. Two independent postconditions must be true for the test to pass: </p> <ul> <li>The <code>DELETE</code> request must result in a status code that indicates success.</li> <li>The resource must no longer exist.</li> </ul> <p> It's conceivable that a bug might fail one of these without invalidating the other. </p> <p> As the test is currently written, it uses <a href="https://xunit.net/">xUnit.net</a>'s standard assertion library. If the <code>Assert.True</code> verification fails, the <code>Assert.Equal</code> statement isn't evaluated. </p> <h3 id="cd132aaee5484068a890cc4b7995bed3"> Assertions as validations <a href="#cd132aaee5484068a890cc4b7995bed3" title="permalink">#</a> </h3> <p> Is it possible to evaluate the <code>Assert.Equal</code> postcondition even if the first assertion fails? You could use a <code>try/catch</code> block, but is there a more composable and elegant option? How about an applicative functor? </p> <p> Since I was interested in exploring this question as a proof of concept, I decided to reuse the machinery that I'd already put in place for the article <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">An applicative reservation validation example in C#</a>: The <code>Validated</code> class and its associated functions. In a sense, you can think of an assertion as a validation of a postcondition. </p> <p> This is not a resemblance I intend to carry too far. What I learn by experimenting with <code>Validated</code> I can apply to a more appropriately-named class like <code>Asserted</code>. </p> <p> Neither of the two above assertions return a value; they are one-stop assertions. If they succeed, they return nothing; if they fail, they produce an error. </p> <p> It's possible to model this kind of behaviour with <code>Validated</code>. You can model a collection of errors with, well, a collection. To keep the proof of concept simple, I decided to use a collection of strings: <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. To model 'nothing' I had to add a <a href="/2018/01/15/unit-isomorphisms">unit</a> type: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Unit</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">Unit</span>()&nbsp;{&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Unit&nbsp;Value&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Unit(); }</pre> </p> <p> This enabled me to define assertions as <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code> values: Either a collection of error messages, or nothing. </p> <h3 id="5ac18fb532c04876b00f7de080905a16"> Asserting truth <a href="#5ac18fb532c04876b00f7de080905a16" title="permalink">#</a> </h3> <p> Instead of xUnit.net's <code>Assert.True</code>, you can now define an equivalent function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;condition, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;message&nbsp;}); }</pre> </p> <p> It simply returns a <code>Success</code> value containing nothing when <code>condition</code> is <code>true</code>, and otherwise a <code>Failure</code> value containing the error <code>message</code>. </p> <p> You can use it like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Validated.AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> Later in the article you'll see how this assertion combines with another assertion. </p> <h3 id="adc98673e4734918ac983a5bbb520e15"> Asserting equality <a href="#adc98673e4734918ac983a5bbb520e15" title="permalink">#</a> </h3> <p> Instead of xUnit.net's <code>Assert.Equal</code>, you can also define a function that works the same way but returns a <code>Validated</code> value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertEqual&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;expected, &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Equals(expected,&nbsp;actual) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>&nbsp;}); }</pre> </p> <p> The <code>AssertEqual</code> function first uses <a href="https://learn.microsoft.com/dotnet/api/system.object.equals">Equals</a> to compare <code>expected</code> with <code>actual</code>. If the result is <code>true</code>, the function returns a <code>Success</code> value containing nothing; otherwise, it returns a <code>Failure</code> value containing a failure message. Since this is only a proof of concept, the failure message is useful, but minimal. </p> <p> Notice that this function returns a value of the same type (<code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>) as <code>AssertTrue</code>. </p> <p> You can use the function like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Validated.AssertEqual(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode);</pre> </p> <p> Again, you'll see how to combine this assertion with the above <code>assertResponse</code> value later in this article. </p> <h3 id="1d29ecff5ea64911918168cb985ed6b5"> Evaluating assertions <a href="#1d29ecff5ea64911918168cb985ed6b5" title="permalink">#</a> </h3> <p> The <code>DeleteReservation</code> test only has two independent assertions, so in my proof of concept, all I needed to do was to figure out a way to combine two applicative assertions into one, and then evaluate it. This rather horrible method does that: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;RunAssertions( &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;assertion1, &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;assertion2) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;f&nbsp;=&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Func&lt;Unit,&nbsp;Unit,&nbsp;Unit&gt;&gt;((_,&nbsp;__)&nbsp;=&gt;&nbsp;Unit.Value); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;combine&nbsp;=&nbsp;(x,&nbsp;y)&nbsp;=&gt;&nbsp;x.Concat(y).ToArray(); &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;composition&nbsp;=&nbsp;f &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(assertion1,&nbsp;combine) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(assertion2,&nbsp;combine); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:blue;">string</span>.IsNullOrEmpty(errors)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(errors);</pre> </p> <p> C# doesn't have good language features for applicative functors the same way that <a href="https://fsharp.org/">F#</a> and <a href="https://www.haskell.org/">Haskell</a> do, and although you can use various tricks to make the programming experience better that what is on display here, I was still doing a proof of concept. If it turns out that this approach is useful and warranted, we can introduce some of the facilities to make the API more palatable. For now, though, we're dealing with all the rough edges. </p> <p> The way that applicative functors work, you typically use a 'lifted' function to combine two (or more) 'lifted' values. Here, 'lifted' means 'being inside the <code>Validated</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers/">container</a>'. </p> <p> Each of the assertions that I want to combine has the same type: <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. Notice that the <code>S</code> (<em>success</em>) generic type argument is <code>Unit</code> in both cases. While it seems redundant, formally I needed a 'lifted' function to combine two <code>Unit</code> values into a single value. This single value can (in principle) have any type I'd like it to have, but since you can't extract any information out of a <code>Unit</code> value, it makes sense to use the <a href="/2018/01/15/unit-isomorphisms#4657dbd4b5fc4abda6e8dee2cac67ea9">monoidal nature of unit</a> to combine two into one. </p> <p> Basically, you just ignore the <code>Unit</code> input values because they carry no information. Also, they're all the same value anyway, since the type is a <a href="https://en.wikipedia.org/wiki/Singleton_pattern">Singleton</a>. In its 'naked' form, the function might be implemented like this: <code>(_,&nbsp;__)&nbsp;=&gt;&nbsp;Unit.Value</code>. Due to the <a href="/2019/12/16/zone-of-ceremony">ceremony</a> required by the combination of C# and applicative functors, however, this <a href="/2017/10/06/monoids">monoidal</a> binary operation has to be 'lifted' to a <code>Validated</code> value. That's the <code>f</code> value in the <code>RunAssertions</code> function body. </p> <p> The <code>Validated.Apply</code> function requires as an argument a function that combines the generic <code>F</code> (<em>failure</em>) values into one, in order to deal with the case where there's multiple failures. In this case <code>F</code> is <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. Since declarations of <code>Func</code> values in C# requires explicit type declaration, that's a bit of a mouthful, but the <code>combine</code> function just concatenates two collections into one. </p> <p> The <code>RunAssertions</code> method can now <code>Apply</code> both <code>assertion1</code> and <code>assertion2</code> to <code>f</code>, which produces a combined <code>Validated</code> value, <code>composition</code>. It then matches on the combined value to produce a <code>string</code> value. If there are no assertion messages, the result is the empty string; otherwise, the function combines the assertion messages with a <code>NewLine</code> between each. Again, this is proof-of-concept code. A more robust and flexible API (if warranted) might keep the errors around as a collection of strongly typed <a href="https://martinfowler.com/bliki/ValueObject.html">Value Objects</a>. </p> <p> Finally, if the resulting <code>errors</code> string is not null or empty, the <code>RunAssertions</code> method throws an exception with the combined error message(s). Here I once more invoked my proof-of-concept privilege to throw an <a href="https://learn.microsoft.com/dotnet/api/system.exception">Exception</a>, even though <a href="https://learn.microsoft.com/dotnet/standard/design-guidelines/using-standard-exception-types">the framework design guidelines admonishes against doing so</a>. </p> <p> Ultimately, then, the <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">assert phase</a> of the test looks like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Validated.AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); <span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Validated.AssertEqual(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); Validated.RunAssertions(assertResponse,&nbsp;assertState);</pre> </p> <p> The rest of the test hasn't changed. </p> <h3 id="42e68063b7a3472e960031fde5e3f40d"> Outcomes <a href="#42e68063b7a3472e960031fde5e3f40d" title="permalink">#</a> </h3> <p> Running the test with the applicative assertions passes, as expected. In order to verify that it works as it's supposed to, I tried to sabotage the System Under Test (SUT) in various ways. First, I made the <code>Delete</code> method that handles <code>DELETE</code> requests a <a href="https://en.wikipedia.org/wiki/NOP_(code)">no-op</a>, while still returning <code>200 OK</code>. As you'd expect, the result is a test failure with this message: </p> <p> <pre>Message: System.Exception : Expected NotFound, but got OK.</pre> </p> <p> This is the assertion that verifies that <code>getResp.StatusCode</code> is <code>404 Not Found</code>. It fails because the sabotaged <code>Delete</code> method doesn't delete the reservation. </p> <p> Then I further sabotaged the SUT to also return an incorrect status code (<code>400 Bad Request</code>), which produced this failure message: </p> <p> <pre>Message: System.Exception : Actual status code: BadRequest. Expected NotFound, but got OK.</pre> </p> <p> Notice that the message contains information about both failure conditions. </p> <p> Finally, I re-enabled the correct behaviour (deleting the reservation from the data store) while still returning <code>400 Bad Request</code>: </p> <p> <pre>Message: System.Exception : Actual status code: BadRequest.</pre> </p> <p> As desired, the assertions collect all relevant failure messages. </p> <h3 id="4a2c496da999408ea6c6cddd6a4f33d4"> Conclusion <a href="#4a2c496da999408ea6c6cddd6a4f33d4" title="permalink">#</a> </h3> <p> Not surprisingly, it's possible to design a composable assertion API that collects multiple failure messages using an applicative functor. Anyone who knows how <a href="/2018/11/05/applicative-validation">applicative validation</a> works would have been able to predict that outcome. That's not what the above proof of concept was about. What I wanted to see was rather how it would play out in a realistic scenario, and whether using an applicative functor is warranted. </p> <p> Applicative functors don't gel well with C#, so unsurprisingly the API is awkward. It's likely possible to smooth much of the friction, but without good language support and syntactic sugar, it's unlikely to become <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C#. </p> <p> Rather than taking the edge off the unwieldy API, the implementation of <code>RunAssertions</code> suggests another alternative. </p> <p> <strong>Next:</strong> <a href="/2022/12/19/error-accumulating-composable-assertions-in-c">Error-accumulating composable assertions in C#</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Decouple to delete https://blog.ploeh.dk/2022/11/21/decouple-to-delete 2022-11-21T08:46:00+00:00 Mark Seemann <div id="post"> <p> <em>Don't try to predict the future.</em> </p> <p> Do you know why it's called <a href="https://en.wikipedia.org/wiki/Spaghetti_code">spaghetti code</a>? It's a palatable metaphor. You may start with a single spaghetto, but usually, as you wind your fork around it, the whole dish follows along. Unless you're careful, eating spaghetti can be a mess. </p> <p> <img src="/content/binary/spaghetti-le-calandre.jpg" alt="A small spaghetti serving."> </p> <p> Spaghetti code is tangled and everything is directly or transitively connected to everything else. As you try to edit the code, every change you make affects other code. Fix one thing and another thing breaks, cascading through the code base. </p> <p> I was recently <a href="https://www.goodreads.com/review/show/4913194780">reading Clean Architecture</a>, and as <a href="https://en.wikipedia.org/wiki/Robert_C._Martin">Robert C. Martin</a> was explaining the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a> for the umpteenth time, my brain made a new connection. To be clear: Connecting (coupling) code is bad, but connecting ideas is good. </p> <h3 id="51ca1981115f493bb24ac9338419fe91"> What a tangled web we weave <a href="#51ca1981115f493bb24ac9338419fe91" title="permalink">#</a> </h3> <p> It's impractical to write code that depends on nothing else. Most code will call other code, which again calls other code. It behoves us, though, to be careful that the web of dependencies don't get too tangled. </p> <p> Imagine a code base where the dependency graph looks like this: </p> <p> <img src="/content/binary/tangled-dependency-graph.png" alt="A connected graph."> </p> <p> Think of each node as a unit of code; a class or a module. While a dependency graph is a <a href="https://en.wikipedia.org/wiki/Directed_graph">directed graph</a>, I didn't indicate the directions. Imagine that most edges point both ways, so that the nodes are interdependent. In other ways, the graph has <a href="https://en.wikipedia.org/wiki/Cycle_(graph_theory)">cycles</a>. This is <a href="http://evelinag.com/blog/2014/06-09-comparing-dependency-networks/">not uncommon in C# code</a>. </p> <p> Pick any node in such a graph, and chances are that other nodes depend on it. This makes it hard to make changes to the code in that node, because a change may affect the code that depends on it. As you try to fix the depending code, that change, too, ripples through the network. </p> <p> This already explains why tight coupling is problematic. </p> <h3 id="61e4982dad794d0085dd7240508f73b7"> It is difficult to make predictions, especially about the future <a href="#61e4982dad794d0085dd7240508f73b7" title="permalink">#</a> </h3> <p> When you write source code, you might be tempted to try to take into account future needs and requirements. There may be a historical explanation for that tendency. </p> <blockquote> <p> "That is, once it was a sign of failure to change product code. You should have gotten it right the first time." </p> <footer><cite><a href="https://twitter.com/marick/status/1566564277573525507">Brian Marick</a></cite></footer> </blockquote> <p> In the days of punchcards, you had to schedule time to use a computer. If you made a mistake in your program, you typically didn't have time to fix it during your timeslot. A mistake could easily cost you days as you scrambled to schedule a new time. Not surprisingly, emphasis was on correctness. </p> <p> With this mindset, it's natural to attempt to future-proof code. </p> <h3 id="0f12f362efaf461bbbdf9bddd2e01361"> YAGNI <a href="#0f12f362efaf461bbbdf9bddd2e01361" title="permalink">#</a> </h3> <p> With interactive development environments you can get rapid feedback. If you make a mistake, change the code and observe the outcome. Don't add code because you think that you might need it later. <a href="https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it">You probably will not</a>. </p> <p> While you should avoid <a href="https://wiki.c2.com/?SpeculativeGenerality">speculative generality</a>, that alone is no guarantee of clean code. Unless you're careful, you can easily make a mess by tightly coupling different parts of your code base. </p> <p> How do produce a code base that is as easy to change as possible? </p> <h3 id="89bda4d0fd774c7c969c30e1562aa11b"> Write code that is easy to delete <a href="#89bda4d0fd774c7c969c30e1562aa11b" title="permalink">#</a> </h3> <p> Write code that is easy to change. The ultimate change you can make is to delete code. After that, you can write something else that better does what you need. </p> <blockquote> <p> "A system where you can delete parts without rewriting others is often called loosely coupled" </p> <footer><cite><a href="https://programmingisterrible.com/post/139222674273/how-to-write-disposable-code-in-large-systems">tef</a></cite></footer> </blockquote> <p> I don't mean that you should always delete code in order to make changes, but often, looking at extremes can provide insights into less extreme cases. </p> <p> When you have a tangled web as shown above, most of the code is coupled to other parts. If you delete a node, then you break something else. You'd think that deleting code is the easiest thing in the world, but it's not. </p> <p> What if, on the other hand, you have smaller clusters of nodes that are independent? </p> <p> <img src="/content/binary/less-coupled-dependency-graph.png" alt="A disconnected graph with small islands of connected graphs."> </p> <p> If your dependency graph looks like this, you can at least delete each of the 'islands' without impacting the other sub-graphs. </p> <p> <img src="/content/binary/dependency-graph-without-deleted-subgraph.png" alt="The graph from the previous figure, less one sub-graph."> </p> <p> <a href="https://programmingisterrible.com/post/139222674273/how-to-write-disposable-code-in-large-systems">Writing code that is easy to delete</a> may be a good idea, but even <em>that</em> is easier said that done. Loose coupling is, once more, key to good architecture. </p> <h3 id="88c387fb34bc48b382d1eefdc3ee6367"> Add something better <a href="#88c387fb34bc48b382d1eefdc3ee6367" title="permalink">#</a> </h3> <p> Once you've deleted a cluster of code, you have the opportunity to add something that is even less coupled than the island you deleted. </p> <p> <img src="/content/binary/dependency-graph-with-new-subgraphs.png" alt="The graph from the previous figure, with new small graphs added."> </p> <p> If you add new code that is less coupled than the code you deleted, it's even easier to delete again. </p> <h3 id="8a7a8d9547eb4d3881a1cced603f8422"> Conclusion <a href="#8a7a8d9547eb4d3881a1cced603f8422" title="permalink">#</a> </h3> <p> Coupling is a key factor in code organisation. Tightly coupled code is difficult to change. Loosely coupled code is easier to change. As a thought experiment, consider how difficult it would be to delete a particular piece of code. The easier it is to delete the code, the less coupled it is. </p> <p> Deleting a small piece of code to add new code in its stead is the ultimate change. You can often get by with a less radical edit, but if all else fails, delete part of your code base and start over. The less coupled the code is, the easier it is to change. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The Reader monad https://blog.ploeh.dk/2022/11/14/the-reader-monad 2022-11-14T06:50:00+00:00 Mark Seemann <div id="post"> <p> <em>Normal functions form monads. An article for object-oriented programmers.</em> </p> <p> This article is an instalment in <a href="/2022/03/28/monads">an article series about monads</a>. A previous article described <a href="/2021/08/30/the-reader-functor">the Reader functor</a>. As is the case with many (but not all) <a href="/2018/03/22/functors">functors</a>, Readers also form monads. </p> <p> This article continues where the Reader functor article stopped. It uses the same code base. </p> <h3 id="d54d83f22d854e94853271e1a559a1d8"> Flatten <a href="#d54d83f22d854e94853271e1a559a1d8" title="permalink">#</a> </h3> <p> A monad must define either a <em>bind</em> or <em>join</em> function, although you can use other names for both of these functions. <code>Flatten</code> is in my opinion a more intuitive name than <code>join</code>, since a monad is really just a functor that you can flatten. Flattening is relevant if you have a nested functor; in this case a Reader within a Reader. You can flatten such a nested Reader with a <code>Flatten</code> function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IReader&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#74531f;">Flatten</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IReader&lt;R,&nbsp;IReader&lt;R,&nbsp;A&gt;&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;FlattenReader&lt;R,&nbsp;A&gt;(source); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">FlattenReader</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>&gt;&nbsp;:&nbsp;IReader&lt;R,&nbsp;A&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;IReader&lt;R,&nbsp;IReader&lt;R,&nbsp;A&gt;&gt;&nbsp;source; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">FlattenReader</span>(IReader&lt;R,&nbsp;IReader&lt;R,&nbsp;A&gt;&gt;&nbsp;<span style="color:#1f377f;">source</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.source&nbsp;=&nbsp;source; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;A&nbsp;<span style="color:#74531f;">Run</span>(R&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IReader&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">newReader</span>&nbsp;=&nbsp;source.Run(environment); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;newReader.Run(environment); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Since the <code>source</code> Reader is nested, calling its <code>Run</code> method once returns a <code>newReader</code>. You can <code>Run</code> that <code>newReader</code> one more time to get an <code>A</code> value to return. </p> <p> You could easily chain the two calls to <code>Run</code> together, one after the other. That would make the code terser, but here I chose to do it in two explicit steps in order to show what's going on. </p> <p> Like the previous article about <a href="/2022/06/20/the-state-monad">the State monad</a>, a lot of <a href="/2019/12/16/zone-of-ceremony">ceremony</a> is required because this variation of the Reader monad is defined with an interface. You could also define the Reader monad on a 'raw' function of the type <code>Func&lt;R, A&gt;</code>, in which case <code>Flatten</code> would be simpler: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#74531f;">Flatten</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>&gt;(<span style="color:blue;">this</span>&nbsp;Func&lt;R,&nbsp;Func&lt;R,&nbsp;A&gt;&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#1f377f;">environment</span>&nbsp;=&gt;&nbsp;source(environment)(environment); }</pre> </p> <p> In this variation <code>source</code> is a function, so you can call it with <code>environment</code>, which returns another function that you can again call with <code>environment</code>. This produces an <code>A</code> value for the function to return. </p> <h3 id="e2f72c66681d45949a23a7e574ae5ae7"> SelectMany <a href="#e2f72c66681d45949a23a7e574ae5ae7" title="permalink">#</a> </h3> <p> When you have <code>Flatten</code> you can always define <code>SelectMany</code> (<em>monadic bind</em>) like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IReader&lt;R,&nbsp;B&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>,&nbsp;<span style="color:#2b91af;">B</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IReader&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;A,&nbsp;IReader&lt;R,&nbsp;B&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.Select(selector).Flatten(); }</pre> </p> <p> First use functor-based mapping. Since the <code>selector</code> returns a Reader, this mapping produces a Reader within a Reader. That's exactly the situation that <code>Flatten</code> addresses. </p> <p> The above <code>SelectMany</code> example works with the <code>IReader&lt;R,&nbsp;A&gt;</code> interface, but the 'raw' function version has the exact same implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;R,&nbsp;B&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>,&nbsp;<span style="color:#2b91af;">B</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Func&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;A,&nbsp;Func&lt;R,&nbsp;B&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.Select(selector).Flatten(); }</pre> </p> <p> Only the method declaration differs. </p> <h3 id="8c5f94f7395040a7bcfdb2561c8e3ed3"> Query syntax <a href="#8c5f94f7395040a7bcfdb2561c8e3ed3" title="permalink">#</a> </h3> <p> Monads also enable query syntax in C# (just like they enable other kinds of syntactic sugar in languages like <a href="https://fsharp.org/">F#</a> and <a href="https://www.haskell.org">Haskell</a>). As outlined in the <a href="/2022/03/28/monads">monad introduction</a>, however, you must add a special <code>SelectMany</code> overload: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IReader&lt;R,&nbsp;T1&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">U</span>,&nbsp;<span style="color:#2b91af;">T1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IReader&lt;R,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;IReader&lt;R,&nbsp;U&gt;&gt;&nbsp;<span style="color:#1f377f;">k</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;U,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">s</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;k(x).Select(<span style="color:#1f377f;">y</span>&nbsp;=&gt;&nbsp;s(x,&nbsp;y))); }</pre> </p> <p> As already predicted in the monad introduction, this boilerplate overload is always implemented in the same way. Only the signature changes. With it, you could write an expression like this nonsense: </p> <p> <pre>IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">bool</span>&gt;&nbsp;<span style="color:#1f377f;">r</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;dur&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:blue;">new</span>&nbsp;MinutesReader() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;b&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Thingy(dur) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;b;</pre> </p> <p> Where <code>MinutesReader</code> was already shown in the article <a href="/2021/10/04/reader-as-a-contravariant-functor">Reader as a contravariant functor</a>. I couldn't come up with a good name for another reader, so I went with <a href="https://dannorth.net">Dan North</a>'s naming convention that if you don't yet know what to call a class, method, or function, don't <em>pretend</em> that you know. Be explicit that you don't know. </p> <p> Here it is, for the sake of completion: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Thingy</span>&nbsp;:&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">bool</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;TimeSpan&nbsp;timeSpan; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Thingy</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.timeSpan&nbsp;=&nbsp;timeSpan; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="color:#74531f;">Run</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(timeSpan.Ticks&nbsp;*&nbsp;environment).TotalDays&nbsp;&lt;&nbsp;1; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I'm not claiming that this class makes sense. These articles are deliberate kept abstract in order to focus on structure and behaviour, rather than on practical application. </p> <h3 id="d5d827cf83d242e6baecbda622ca5cb9"> Return <a href="#d5d827cf83d242e6baecbda622ca5cb9" title="permalink">#</a> </h3> <p> Apart from flattening or monadic bind, a monad must also define a way to put a normal value into the monad. Conceptually, I call this function <em>return</em> (because that's the name that Haskell uses): </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IReader&lt;R,&nbsp;A&gt;&nbsp;<span style="color:#74531f;">Return</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>&gt;(A&nbsp;<span style="color:#1f377f;">a</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ReturnReader&lt;R,&nbsp;A&gt;(a); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ReturnReader</span>&lt;<span style="color:#2b91af;">R</span>,&nbsp;<span style="color:#2b91af;">A</span>&gt;&nbsp;:&nbsp;IReader&lt;R,&nbsp;A&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;A&nbsp;a; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ReturnReader</span>(A&nbsp;<span style="color:#1f377f;">a</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.a&nbsp;=&nbsp;a; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;A&nbsp;<span style="color:#74531f;">Run</span>(R&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;a; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This implementation returns the <code>a</code> value and completely ignores the <code>environment</code>. You can do the same with a 'naked' function. </p> <h3 id="bd79fe3d97644f0f9edb12f08a0b5d01"> Left identity <a href="#bd79fe3d97644f0f9edb12f08a0b5d01" title="permalink">#</a> </h3> <p> We need to identify the <em>return</em> function in order to examine <a href="/2022/04/11/monad-laws">the monad laws</a>. Now that this is accomplished, let's see what the laws look like for the Reader monad, starting with the left identity law. </p> <p> <pre>[Theory] [InlineData(UriPartial.Authority,&nbsp;<span style="color:#a31515;">&quot;https://example.com/f?o=o&quot;</span>)] [InlineData(UriPartial.Path,&nbsp;<span style="color:#a31515;">&quot;https://example.net/b?a=r&quot;</span>)] [InlineData(UriPartial.Query,&nbsp;<span style="color:#a31515;">&quot;https://example.org/b?a=z&quot;</span>)] [InlineData(UriPartial.Scheme,&nbsp;<span style="color:#a31515;">&quot;https://example.gov/q?u=x&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">LeftIdentity</span>(UriPartial&nbsp;<span style="color:#1f377f;">a</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">u</span>) { &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;UriPartial,&nbsp;IReader&lt;Uri,&nbsp;UriPartial&gt;&gt;&nbsp;<span style="color:#1f377f;">@return</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">up</span>&nbsp;=&gt;&nbsp;Reader.Return&lt;Uri,&nbsp;UriPartial&gt;(up); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;UriPartial,&nbsp;IReader&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">h</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">up</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;UriPartReader(up); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;@return(a).SelectMany(h).Run(<span style="color:blue;">new</span>&nbsp;Uri(u)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;h(a).Run(<span style="color:blue;">new</span>&nbsp;Uri(u))); }</pre> </p> <p> In order to compare the two Reader values, the test has to <code>Run</code> them and then compare the return values. </p> <p> This test and the next uses a Reader implementation called <code>UriPartReader</code>, which almost makes sense: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">UriPartReader</span>&nbsp;:&nbsp;IReader&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;UriPartial&nbsp;part; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UriPartReader</span>(UriPartial&nbsp;<span style="color:#1f377f;">part</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.part&nbsp;=&nbsp;part; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#74531f;">Run</span>(Uri&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;environment.GetLeftPart(part); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Almost. </p> <h3 id="ad9edb1097be4eeb96170809272851eb"> Right identity <a href="#ad9edb1097be4eeb96170809272851eb" title="permalink">#</a> </h3> <p> In a similar manner, we can showcase the right identity law as a test. </p> <p> <pre>[Theory] [InlineData(UriPartial.Authority,&nbsp;<span style="color:#a31515;">&quot;https://example.com/q?u=ux&quot;</span>)] [InlineData(UriPartial.Path,&nbsp;<span style="color:#a31515;">&quot;https://example.net/q?u=uuz&quot;</span>)] [InlineData(UriPartial.Query,&nbsp;<span style="color:#a31515;">&quot;https://example.org/c?o=rge&quot;</span>)] [InlineData(UriPartial.Scheme,&nbsp;<span style="color:#a31515;">&quot;https://example.gov/g?a=rply&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">RightIdentity</span>(UriPartial&nbsp;<span style="color:#1f377f;">a</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">u</span>) { &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;UriPartial,&nbsp;IReader&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">f</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">up</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;UriPartReader(up); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">string</span>,&nbsp;IReader&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">@return</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;Reader.Return&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt;(s); &nbsp;&nbsp;&nbsp;&nbsp;IReader&lt;Uri,&nbsp;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">m</span>&nbsp;=&nbsp;f(a); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m.SelectMany(@return).Run(<span style="color:blue;">new</span>&nbsp;Uri(u)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m.Run(<span style="color:blue;">new</span>&nbsp;Uri(u))); }</pre> </p> <p> As always, even a parametrised test constitutes no <em>proof</em> that the law holds. I show the tests to illustrate what the laws look like in 'real' code. </p> <h3 id="67a2f225bd8f432dbeed30c2ba2b623a"> Associativity <a href="#67a2f225bd8f432dbeed30c2ba2b623a" title="permalink">#</a> </h3> <p> The last monad law is the associativity law that describes how (at least) three functions compose. We're going to need three functions. For the purpose of demonstrating the law, any three pure functions will do. While the following functions are silly and not at all 'realistic', they have the virtue of being as simple as they can be (while still providing a bit of variety). They don't 'mean' anything, so don't worry too much about their behaviour. It is, as far as I can tell, nonsensical. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">F</span>&nbsp;:&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">string</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">char</span>&nbsp;c; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">F</span>(<span style="color:blue;">char</span>&nbsp;<span style="color:#1f377f;">c</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.c&nbsp;=&nbsp;c; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#74531f;">Run</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>(c,&nbsp;environment); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">G</span>&nbsp;:&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">bool</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">string</span>&nbsp;s; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">G</span>(<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">s</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.s&nbsp;=&nbsp;s; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="color:#74531f;">Run</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;environment&nbsp;&lt;&nbsp;42&nbsp;||&nbsp;s.Contains(<span style="color:#a31515;">&quot;a&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">H</span>&nbsp;:&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;TimeSpan&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;b; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">H</span>(<span style="color:blue;">bool</span>&nbsp;<span style="color:#1f377f;">b</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.b&nbsp;=&nbsp;b; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TimeSpan&nbsp;<span style="color:#74531f;">Run</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">environment</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;b&nbsp;? &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromMinutes(environment)&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(environment); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Armed with these three classes, we can now demonstrate the Associativity law: </p> <p> <pre>[Theory] [InlineData(<span style="color:#a31515;">&#39;a&#39;</span>,&nbsp;0)] [InlineData(<span style="color:#a31515;">&#39;b&#39;</span>,&nbsp;1)] [InlineData(<span style="color:#a31515;">&#39;c&#39;</span>,&nbsp;42)] [InlineData(<span style="color:#a31515;">&#39;d&#39;</span>,&nbsp;2112)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Associativity</span>(<span style="color:blue;">char</span>&nbsp;<span style="color:#1f377f;">a</span>,&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">i</span>) { &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">char</span>,&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">string</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">f</span>&nbsp;=&nbsp;<span style="color:#1f377f;">c</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;F(c); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">string</span>,&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">bool</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">g</span>&nbsp;=&nbsp;<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;G(s); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">bool</span>,&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;TimeSpan&gt;&gt;&nbsp;<span style="color:#1f377f;">h</span>&nbsp;=&nbsp;<span style="color:#1f377f;">b</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;H(b); &nbsp;&nbsp;&nbsp;&nbsp;IReader&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">m</span>&nbsp;=&nbsp;f(a); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m.SelectMany(g).SelectMany(h).Run(i), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;m.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;g(x).SelectMany(h)).Run(i)); }</pre> </p> <p> In case you're wondering, the four test cases produce the outputs <code>00:00:00</code>, <code>00:01:00</code>, <code>00:00:42</code>, and <code>00:35:12</code>. You can see that reproduced below: </p> <h3 id="a46f77dddd914f0b9c8926dd2c06e9d6"> Haskell <a href="#a46f77dddd914f0b9c8926dd2c06e9d6" title="permalink">#</a> </h3> <p> In Haskell, normal functions <code>a -&gt; b</code> are already <code>Monad</code> instances, which means that you can easily replicate the functions from the <code>Associativity</code> test: </p> <p> <pre>&gt; f c = \env -&gt; replicate env c &gt; g s = \env -&gt; env &lt; 42 || 'a' `elem` s &gt; h b = \env -&gt; if b then secondsToDiffTime (toEnum env * 60) else secondsToDiffTime (toEnum env)</pre> </p> <p> I've chosen to write the <code>f</code>, <code>g</code>, and <code>h</code> as functions that return lambda expressions in order to emphasise that each of these functions return Readers. Since Haskell functions are already curried, I could also have written them in the more normal function style with two normal parameters, but that might have obscured the Reader aspect of each. </p> <p> Here's the composition in action: </p> <p> <pre>&gt; f 'a' &gt;&gt;= g &gt;&gt;= h $ 0 0s &gt; f 'b' &gt;&gt;= g &gt;&gt;= h $ 1 60s &gt; f 'c' &gt;&gt;= g &gt;&gt;= h $ 42 42s &gt; f 'd' &gt;&gt;= g &gt;&gt;= h $ 2112 2112s</pre> </p> <p> In case you are wondering, 2,112 seconds is 35 minutes and 12 seconds, so all outputs fit with the results reported for the C# example. </p> <p> What the above Haskell GHCi (<a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop">REPL</a>) session demonstrates is that it's possible to compose functions with Haskell's monadic bind operator <code>&gt;&gt;=</code> operator exactly because all functions are (Reader) monads. </p> <h3 id="fb5e50be9b66464ca61cd4a45eb7e756"> Conclusion <a href="#fb5e50be9b66464ca61cd4a45eb7e756" title="permalink">#</a> </h3> <p> In Haskell, it can occasionally be useful that a function can be used when a <code>Monad</code> is required. Some Haskell libraries are defined in very general terms. Their APIs may enable you to call functions with any monadic input value. You can, say, pass a <a href="/2022/04/25/the-maybe-monad">Maybe</a>, a <a href="/2022/04/19/the-list-monad">List</a>, an <a href="/2022/05/09/an-either-monad">Either</a>, a State, but you can also pass a function. </p> <p> C# and most other languages (F# included) doesn't come with that level of abstraction, so the fact that a function forms a monad is less useful there. In fact, I can't recall having made explicit use of this knowledge in C#, but one never knows if that day arrives. </p> <p> In a similar vein, knowing that <a href="/2018/04/16/endomorphic-composite-as-a-monoid">endomorphisms form monoids</a> (and thereby also <a href="/2017/11/27/semigroups">semigroups</a>) enabled me to <a href="/2020/12/14/validation-a-solved-problem">quickly identify the correct design for a validation problem</a>. </p> <p> Who knows? One day the knowledge that functions are monads may come in handy. </p> <p> <strong>Next:</strong> <a href="/2023/01/09/the-io-monad">The IO monad</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Applicative assertions https://blog.ploeh.dk/2022/11/07/applicative-assertions 2022-11-07T06:56:00+00:00 Mark Seemann <div id="post"> <p> <em>An exploration.</em> </p> <p> In a recent Twitter exchange, <a href="https://lucasdicioccio.github.io/">Lucas DiCioccio</a> made an interesting observation: </p> <blockquote> <p> "Imho the properties you want of an assertion-framework are really close (the same as?) applicative-validation: one assertion failure with multiple bullet points composed mainly from combinators." </p> <footer><cite><a href="https://twitter.com/lucasdicioccio/status/1572264819109003265">Lucas DiCioccio</a></cite></footer> </blockquote> <p> In another branch off my initial tweet <a href="https://www.joshka.net/">Josh McKinney</a> pointed out the short-circuiting nature of standard assertions: </p> <blockquote> <p> "short circuiting often causes weaker error messages in failing tests than running compound assertions. E.g. </p> <p> <pre>TransferTest { a.transfer(b,50); a.shouldEqual(50); b.shouldEqual(150); // never reached? }</pre> </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572232484884217864">Josh McK</a></cite></footer> </blockquote> <p> Most standard assertion libraries work by throwing exceptions when an assertion fails. Once you throw an exception, remaining code doesn't execute. This means that you only get the first assertion message. Further assertions are not evaluated. </p> <p> Josh McKinney <a href="https://twitter.com/joshuamck/status/1572528796125003777">later gave more details about a particular scenario</a>. Although in the general case I don't consider the short-circuiting nature of assertions to be a problem, I grant that there are cases where proper assertion composition would be useful. </p> <p> Lucas DiCioccio's suggestion seems worthy of investigation. </p> <h3 id="54fc71d7459d4251a79dc16f58bd79b3"> Ongoing exploration <a href="#54fc71d7459d4251a79dc16f58bd79b3" title="permalink">#</a> </h3> <p> <a href="https://twitter.com/ploeh/status/1572282314402721805">I asked</a> Lucas DiCioccio whether he'd done any work with his idea, and the day after <a href="https://twitter.com/lucasdicioccio/status/1572639255582867456">he replied</a> with a <a href="https://www.haskell.org">Haskell</a> proof of concept. </p> <p> I found the idea so interesting that I also wanted to carry out a few proofs of concept myself, perhaps within a more realistic setting. </p> <p> As I'm writing this, I've reached some preliminary conclusions, but I'm also aware that they may not hold in more general cases. I'm posting what I have so far, but you should expect this exploration to evolve over time. If I find out more, I'll update this post with more articles. </p> <ul> <li><a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">An initial proof of concept of applicative assertions in C#</a></li> <li><a href="/2022/12/19/error-accumulating-composable-assertions-in-c">Error-accumulating composable assertions in C#</a></li> <li><a href="/2023/01/30/built-in-alternatives-to-applicative-assertions">Built-in alternatives to applicative assertions</a></li> </ul> <p> A preliminary summary is in order. Based on the first two articles, applicative assertions look like overkill. I think, however, that it's because of the degenerate nature of the example. Some assertions are essentially one-stop verifications: Evaluate a predicate, and throw an exception if the result is <em>false</em>. These assertions return <a href="/2018/01/15/unit-isomorphisms">unit or void</a>. Examples from <a href="https://xunit.net/">xUnit</a> include <code>Assert.Equal</code>, <code>Assert.True</code>, <code>Assert.False</code>, <code>Assert.All</code>, and <code>Assert.DoesNotContain</code>. </p> <p> These are the kinds of assertions that the initial two articles explore. </p> <p> There are other kinds of assertions that return a value in case of success. xUnit.net examples include <code>Assert.Throws</code>, <code>Assert.Single</code>, <code>Assert.IsAssignableFrom</code>, and some overloads of <code>Assert.Contains</code>. <code>Assert.Single</code>, for example, verifies that a collection contains only a single element. While it throws an exception if the collection is either empty or has more than one element, in the success case it returns the single value. This can be useful if you want to add more assertions based on that value. </p> <p> I haven't experimented with this yet, but as far as can tell, you'll run into the following problem: If you make such an assertion return an <a href="/2018/10/01/applicative-functors">applicative functor</a>, you'll need some way to handle the success case. Combining it with another assertion-producing function, such as <code>a -> Asserted e b</code> (pseudocode) is possible with <a href="/2018/03/22/functors">functor</a> mapping, but will leave you with a nested functor. </p> <p> You'll probably want to flatten the nested functor, which is exactly what <a href="/2022/03/28/monads">monads</a> do. Monads, on the other hand, short circuit, so you don't want to make your applicative assertion type a monad. Instead, you'll need to use an isomorphic monad container (<a href="/2022/05/09/an-either-monad">Either</a> should do) to move in and out of. Doable, but is the complexity warranted? </p> <p> I realise that the above musings are abstract, and that I really should show rather than tell. I'll add some more content here if I ever collect something worthy of an article. if you ask me now, though, I consider that a bit of a toss-up. </p> <p> The first two examples also suffer from being written in C#, which doesn't have good syntactic support for applicative functors. Perhaps I'll add some articles that use <a href="https://fsharp.org/">F#</a> or Haskell. </p> <h3 id="676b3bf45f0841bc9a51d3510d917a6a"> Conclusion <a href="#676b3bf45f0841bc9a51d3510d917a6a" title="permalink">#</a> </h3> <p> There's the occasional need for composable assertions. You can achieve that with an applicative functor, but the question is whether it's warranted. Could you make something simpler based on the <a href="/2022/04/19/the-list-monad">list monad</a>? </p> <p> As I'm writing this, I don't consider that question settled. Even so, you may want to read on. </p> <p> <strong>Next:</strong> <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">An initial proof of concept of applicative assertions in C#</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="e3269279056146f985c8405f6d3ad286"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> Monads, on the other hand, short circuit, so you don't want to make your applicative assertion type a monad. </blockquote> <p> I want my assertion type to be both applicative and monadic. So does Paul Loath, the creator of Language Ext, which is most clearly seen via <a href="https://github.com/louthy/language-ext/blob/main/LanguageExt.Tests/ValidationTests.cs#L267-L277">this Validation test code</a>. So does Alexis King (as you pointed out to me) in her Haskell Validation package, which violiates Hakell's monad type class, and which she defends <a href="https://hackage.haskell.org/package/monad-validate-1.2.0.1/docs/Control-Monad-Validate.html#:~:text=ValidateT%20and%20the%20Monad%20laws">here</a>. </p> <p> When I want (or typically need) short-circuiting behavior, then I use the type's monadic API. When I want "error-collecting behavior", then I use the type's applicative API. </p> <blockquote> The first two examples also suffer from being written in C#, which doesn't have good syntactic support for applicative functors. </blockquote> <p> The best syntactic support for applicative functors in C# that I have seen is in Langauge Ext. <a href="https://github.com/louthy/language-ext/blob/15cd875ea40925e2ca9cd702c84f9142918dbb77/LanguageExt.Tests/ValidationTests.cs#L272-L277">A comment explains</a> in that same Validation test how it works, and the line after the comment shows it in action. </p> </div> <div class="comment-date">2023-01-16 21:13 UTC</div> </div> <div class="comment" id="a1516515c1b84cb2b8473f2c0321562e"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. Whether or not you want to enable monadic short-circuiting for assertions or validations depends, I think, on 'developer ergonomics'. It's a trade-off mainly between <em>ease</em> and <em>simplicity</em> as <a href="https://www.infoq.com/presentations/Simple-Made-Easy/">outlined by Rich Hickey</a>. Enabling a monadic API for something that isn't naturally monadic does indeed provide ease of use, in that the compositional capabilities of a monad are readily 'at hand'. </p> <p> If you don't have that capability you'll have to map back and forth between, say, <code>Validation</code> and <code>Either</code> (if using the <a href="https://hackage.haskell.org/package/validation">validation</a> package). This is tedious, but <a href="https://peps.python.org/pep-0020/">explicit</a>. </p> <p> Making validation or assertions monadic makes it easier to compose nested values, but also (in my experience) makes it easier to make mistakes, in the sense that you (or a colleague) may <em>think</em> that the behaviour is error-collecting, whereas in reality it's short-circuiting. </p> <p> In the end, the trade-off may reduce to how much you trust yourself (and co-workers) to steer clear of mistakes, and how important it is to avoid errors. In this case, how important is it to collect the errors, rather than short-circuiting? </p> <p> You can choose one alternative or the other by weighing such concerns. </p> </div> <div class="comment-date">2023-01-19 8:30 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. A regular grid emerges https://blog.ploeh.dk/2022/10/31/a-regular-grid-emerges 2022-10-31T06:44:00+00:00 Mark Seemann <div id="post"> <p> <em>The code behind a lecture animation.</em> </p> <p> If you've seen my presentation <a href="https://youtu.be/FPEEiX5unWI">Fractal Architecture</a>, you may have wondered how I made the animation where a regular(ish) hexagonal grid emerges from adding more and more blobs to an area. </p> <p> <img src="/content/binary/a-regular-grid-emerges.png" alt="A grid-like structure starting to emerge from tightly packing blobs."> </p> <p> Like <a href="/2021/04/05/mazes-on-voronoi-tesselations">a few</a> <a href="/2021/07/05/fractal-hex-flowers">previous</a> blog posts, today's article appears on <a href="https://observablehq.com">Observable</a>, which is where the animation and the code that creates it lives. <a href="https://observablehq.com/@ploeh/a-regular-grid-emerges">Go there to read it</a>. </p> <p> If you have time, watch the animation evolve. Personally I find it quite mesmerising. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Encapsulation in Functional Programming https://blog.ploeh.dk/2022/10/24/encapsulation-in-functional-programming 2022-10-24T05:54:00+00:00 Mark Seemann <div id="post"> <p> <em>Encapsulation is only relevant for object-oriented programming, right?</em> </p> <p> The concept of <em>encapsulation</em> is closely related to object-oriented programming (OOP), and you rarely hear the word in discussions about (statically-typed) functional programming (FP). I will argue, however, that the notion is relevant in FP as well. Typically, it just appears with a different catchphrase. </p> <h3 id="f0f64bfaaa6f4d22b990fae4775a8b89"> Contracts <a href="#f0f64bfaaa6f4d22b990fae4775a8b89" title="permalink">#</a> </h3> <p> I base my understanding of encapsulation on <a href="/ref/oosc">Object-Oriented Software Construction</a>. I've tried to distil it in my Pluralsight course <a href="/encapsulation-and-solid">Encapsulation and SOLID</a>. </p> <p> In short, encapsulation denotes the distinction between an object's contract and its implementation. An object should fulfil its contract in such a way that client code doesn't need to know about its implementation. </p> <p> Contracts, according to <a href="https://en.wikipedia.org/wiki/Bertrand_Meyer">Bertrand Meyer</a>, describe three properties of objects: </p> <ul> <li>Preconditions: What client code must fulfil in order to successfully interact with the object.</li> <li>Invariants: Statements about the object that are always true.</li> <li>Postconditions: Statements that are guaranteed to be true after a successful interaction between client code and object.</li> </ul> <p> You can replace <em>object</em> with <em>value</em> and I'd argue that the same concerns are relevant in FP. </p> <p> In OOP <em>invariants</em> often point to the properties of an object that are guaranteed to remain even in the face of state mutation. As you change the state of an object, the object should guarantee that its state remains valid. These are the properties (i.e. <em>qualities</em>, <em>traits</em>, <em>attributes</em>) that don't vary - i.e. are <em>invariant</em>. </p> <p> An example would be helpful around here. </p> <h3 id="72a37691c0c14d4a8673d52f25e7c3e2"> Table mutation <a href="#72a37691c0c14d4a8673d52f25e7c3e2" title="permalink">#</a> </h3> <p> Consider an object that models a table in a restaurant. You may, for example, be working on <a href="/2020/01/27/the-maitre-d-kata">the Maître d' kata</a>. In short, you may decide to model a table as being one of two kinds: Standard tables and communal tables. You can reserve seats at communal tables, but you still share the table with other people. </p> <p> You may decide to model the problem in such a way that when you reserve the table, you change the state of the object. You may decide to describe the contract of <code>Table</code> objects like this: </p> <ul> <li>Preconditions <ul> <li>To create a <code>Table</code> object, you must supply a type (standard or communal).</li> <li>To create a <code>Table</code> object, you must supply the size of the table, which is a measure of its capacity; i.e. how many people can sit at it.</li> <li>The capacity must be a natural number. <em>One</em> (1) is the smallest valid capacity.</li> <li>When reserving a table, you must supply a valid reservation.</li> <li>When reserving a table, the reservation quantity must be less than or equal to the table's remaining capacity.</li> </ul> </li> <li>Invariants <ul> <li>The table capacity doesn't change.</li> <li>The table type doesn't change.</li> <li>The number of remaining seats is never negative.</li> <li>The number of remaining seats is never greater than the table's capacity.</li> </ul> </li> <li>Postconditions <ul> <li>After reserving a table, the number of remaining seats can't be greater than the previous number of remaining seats minus the reservation quantity.</li> </ul> </li> </ul> <p> This list may be incomplete, and if you add more operations, you may have to elaborate on what that means to the contract. </p> <p> In C# you may implement a <code>Table</code> class like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Table</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;List&lt;Reservation&gt;&nbsp;reservations; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Table</span>(<span style="color:blue;">int</span>&nbsp;capacity,&nbsp;TableType&nbsp;type) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(capacity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentOutOfRangeException( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nameof(capacity), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Capacity&nbsp;must&nbsp;be&nbsp;greater&nbsp;than&nbsp;zero,&nbsp;but&nbsp;was:&nbsp;</span>{capacity}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;List&lt;Reservation&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Capacity&nbsp;=&nbsp;capacity; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Type&nbsp;=&nbsp;type; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RemaingSeats&nbsp;=&nbsp;capacity; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Capacity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TableType&nbsp;Type&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;RemaingSeats&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Reserve(Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(RemaingSeats&nbsp;&lt;&nbsp;reservation.Quantity) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;InvalidOperationException( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;The&nbsp;table&nbsp;has&nbsp;no&nbsp;remaining&nbsp;seats.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Type&nbsp;==&nbsp;TableType.Communal) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RemaingSeats&nbsp;-=&nbsp;reservation.Quantity; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RemaingSeats&nbsp;=&nbsp;0; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservations.Add(reservation); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This class has good encapsulation because it makes sure to fulfil the contract. You can't put it in an invalid state. </p> <h3 id="b37e173371f249d99f80d12d64b2bee2"> Immutable Table <a href="#b37e173371f249d99f80d12d64b2bee2" title="permalink">#</a> </h3> <p> Notice that two of the invariants for the above <code>Table</code> class is that the table can't change type or capacity. While OOP often revolves around state mutation, it seems reasonable that some data is immutable. A table doesn't all of a sudden change size. </p> <p> In FP data is immutable. Data doesn't change. Thus, data has that invariant property. </p> <p> If you consider the above contract, it still applies to FP. The specifics change, though. You'll no longer be dealing with <code>Table</code> objects, but rather <code>Table</code> data, and to make reservations, you call a function that returns a new <code>Table</code> value. </p> <p> In <a href="https://fsharp.org/">F#</a> you could model a <code>Table</code> like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Table&nbsp;=&nbsp;<span style="color:blue;">private</span>&nbsp;Standard&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;*&nbsp;Reservation&nbsp;list&nbsp;|&nbsp;Communal&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;*&nbsp;Reservation&nbsp;list <span style="color:blue;">module</span>&nbsp;Table&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;standard&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;0&nbsp;&lt;&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;Some&nbsp;(Standard&nbsp;(capacity,&nbsp;[])) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;None &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;communal&nbsp;capacity&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;0&nbsp;&lt;&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">then</span>&nbsp;Some&nbsp;(Communal&nbsp;(capacity,&nbsp;[])) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;None &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;remainingSeats&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Standard&nbsp;(capacity,&nbsp;[])&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;capacity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Standard&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;(capacity,&nbsp;rs)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;capacity&nbsp;-&nbsp;List.sumBy&nbsp;(<span style="color:blue;">fun</span>&nbsp;r&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;r.Quantity)&nbsp;rs &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;reserve&nbsp;r&nbsp;t&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;t&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Standard&nbsp;(capacity,&nbsp;[])&nbsp;<span style="color:blue;">when</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;remainingSeats&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Some&nbsp;(Standard&nbsp;(capacity,&nbsp;[r])) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Communal&nbsp;(capacity,&nbsp;rs)&nbsp;<span style="color:blue;">when</span>&nbsp;r.Quantity&nbsp;&lt;=&nbsp;remainingSeats&nbsp;t&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Some&nbsp;(Communal&nbsp;(capacity,&nbsp;r&nbsp;::&nbsp;rs)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;None</pre> </p> <p> While you'll often hear fsharpers say that one should <a href="https://blog.janestreet.com/effective-ml-video/">make illegal states unrepresentable</a>, in practice you often have to rely on <a href="https://www.hillelwayne.com/post/constructive/">predicative</a> data to enforce contracts. I've done this here by making the <code>Table</code> cases <code>private</code>. Code outside the module can't directly create <code>Table</code> data. Instead, it'll have to use one of two functions: <code>Table.standard</code> or <code>Table.communal</code>. These are functions that return <code>Table option</code> values. </p> <p> That's the idiomatic way to model predicative data in statically typed FP. In <a href="https://www.haskell.org/">Haskell</a> such functions are called <a href="https://wiki.haskell.org/Smart_constructors">smart constructors</a>. </p> <p> Statically typed FP typically use <a href="/2022/04/25/the-maybe-monad">Maybe</a> (<code>Option</code>) or <a href="/2022/05/09/an-either-monad">Either</a> (<code>Result</code>) values to communicate failure, rather than throwing exceptions, but apart from that a smart constructor is just an object constructor. </p> <p> The above F# <code>Table</code> API implements the same contract as the OOP version. </p> <p> If you want to see a more elaborate example of modelling table and reservations in F#, see <a href="/2020/04/27/an-f-implementation-of-the-maitre-d-kata">An F# implementation of the Maître d' kata</a>. </p> <h3 id="78027bc1d1414c2fa3604a68c9df6418"> Functional contracts in OOP languages <a href="#78027bc1d1414c2fa3604a68c9df6418" title="permalink">#</a> </h3> <p> You can adopt many FP concepts in OOP languages. My book <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a> contains sample code in C# that implements an online restaurant reservation system. It includes a <code>Table</code> class that, at first glance, looks like the above C# class. </p> <p> While it has the same contract, the book's <code>Table</code> class is implemented with the FP design principles in mind. Thus, it's an immutable class with this API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Table</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Table&nbsp;Standard(<span style="color:blue;">int</span>&nbsp;seats) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Table&nbsp;Communal(<span style="color:blue;">int</span>&nbsp;seats) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Capacity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;RemainingSeats&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Table&nbsp;Reserve(Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;Accept&lt;<span style="color:#2b91af;">T</span>&gt;(ITableVisitor&lt;T&gt;&nbsp;visitor) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>?&nbsp;obj) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;GetHashCode() }</pre> </p> <p> Notice that the <code>Reserve</code> method returns a <code>Table</code> object. That's the table with the reservation associated. The original <code>Table</code> instance remains unchanged. </p> <p> The entire book is written in the <a href="https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell">Functional Core, Imperative Shell</a> architecture, so all domain models are immutable objects with <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> as methods. </p> <p> The objects still have contracts. They have proper encapsulation. </p> <h3 id="ca2409555a5b4efe9b98e1c65e77256d"> Conclusion <a href="#ca2409555a5b4efe9b98e1c65e77256d" title="permalink">#</a> </h3> <p> Functional programmers may not use the term <em>encapsulation</em> much, but that doesn't mean that they don't share that kind of concern. They often throw around the phrase <em>make illegal states unrepresentable</em> or talk about smart constructors or <a href="https://en.wikipedia.org/wiki/Partial_function">partial versus total functions</a>. It's clear that they care about data modelling that prevents mistakes. </p> <p> The object-oriented notion of <em>encapsulation</em> is ultimately about separating the affordances of an API from its implementation details. An object's contract is an abstract description of the properties (i.e. <em>qualities</em>, <em>traits</em>, or <em>attributes</em>) of the object. </p> <p> Functional programmers care so much about the properties of data and functions that <em>property-based testing</em> is often the preferred way to perform automated testing. </p> <p> Perhaps you can find a functional programmer who might be slightly offended if you suggest that he or she should consider encapsulation. If so, suggest instead that he or she considers the properties of functions and data. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bb616acbc1ac41cb8f937fe7175ce061"> <div class="comment-author"><a href="http://www.raboof.com/">Atif Aziz</a></div> <div class="comment-content"> <p> I wonder what's the goal of illustrating OOP-ish examples exclusively in C# and FP-ish ones in F# when you could stick to just one language for the reader. It might not always be as effective depending on the topic, but for encapsulation and the examples shown in this article, a C# version would read just as effective as an F# one. I mean when you get round to making your points in the <strong>Immutable Table</strong> section of your article, you could demonstrate the ideas with a C# version that's nearly identical to and reads as succinct as the F# version: </p> <pre><span style="color:gray;">#nullable</span>&nbsp;<span style="color:gray;">enable</span> <span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">record</span>&nbsp;<span style="color:blue;">struct</span>&nbsp;<span style="color:darkblue;">Reservation</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">Quantity</span>); <span style="color:blue;">abstract</span>&nbsp;<span style="color:blue;">record</span>&nbsp;<span style="color:darkblue;">Table</span>; <span style="color:blue;">record</span>&nbsp;<span style="color:darkblue;">StandardTable</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">Capacity</span>,&nbsp;<span style="color:darkblue;">Reservation</span>?&nbsp;<span style="color:#1f377f;">Reservation</span>):&nbsp;<span style="color:darkblue;">Table</span>; <span style="color:blue;">record</span>&nbsp;<span style="color:darkblue;">CommunalTable</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">Capacity</span>,&nbsp;<span style="color:darkblue;">ImmutableArray</span>&lt;<span style="color:darkblue;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">Reservations</span>):&nbsp;<span style="color:darkblue;">Table</span>; <span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:darkblue;">TableModule</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:darkblue;">StandardTable</span>?&nbsp;<span style="color:darkcyan;">Standard</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">capacity</span>)&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&lt;&nbsp;capacity&nbsp;?&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:darkblue;">StandardTable</span>(capacity,&nbsp;<span style="color:blue;">null</span>)&nbsp;:&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:darkblue;">CommunalTable</span>?&nbsp;<span style="color:darkcyan;">Communal</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">capacity</span>)&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&lt;&nbsp;capacity&nbsp;?&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:darkblue;">CommunalTable</span>(capacity,&nbsp;<span style="color:darkblue;">ImmutableArray</span>&lt;<span style="color:darkblue;">Reservation</span>&gt;.Empty)&nbsp;:&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:darkcyan;">RemainingSeats</span>(<span style="color:blue;">this</span>&nbsp;<span style="color:darkblue;">Table</span>&nbsp;<span style="color:#1f377f;">table</span>)&nbsp;=&gt;&nbsp;table&nbsp;<span style="color:#8f08c4;">switch</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:darkblue;">StandardTable</span>&nbsp;{&nbsp;<span style="color:purple;">Reservation</span>:&nbsp;<span style="color:blue;">null</span>&nbsp;}&nbsp;t&nbsp;=&gt;&nbsp;t.<span style="color:purple;">Capacity</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:darkblue;">StandardTable</span>&nbsp;=&gt;&nbsp;0, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:darkblue;">CommunalTable</span>&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&gt;&nbsp;t.<span style="color:purple;">Capacity</span>&nbsp;-&nbsp;t.<span style="color:purple;">Reservations</span>.Sum(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;r.Quantity) &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:darkblue;">Table</span>?&nbsp;<span style="color:darkcyan;">Reserve</span>(<span style="color:blue;">this</span>&nbsp;<span style="color:darkblue;">Table</span>&nbsp;<span style="color:#1f377f;">table</span>,&nbsp;<span style="color:darkblue;">Reservation</span>&nbsp;<span style="color:#1f377f;">r</span>)&nbsp;=&gt;&nbsp;table&nbsp;<span style="color:#8f08c4;">switch</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:darkblue;">StandardTable</span>&nbsp;<span style="color:#1f377f;">t</span>&nbsp;<span style="color:#8f08c4;">when</span>&nbsp;r.<span style="color:purple;">Quantity</span>&nbsp;&lt;=&nbsp;t.<span style="color:darkcyan;">RemainingSeats</span>()&nbsp;=&gt;&nbsp;t&nbsp;<span style="color:blue;">with</span>&nbsp;{&nbsp;<span style="color:purple;">Reservation</span>&nbsp;=&nbsp;r&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:darkblue;">CommunalTable</span>&nbsp;<span style="color:#1f377f;">t</span>&nbsp;<span style="color:#8f08c4;">when</span>&nbsp;r.<span style="color:purple;">Quantity</span>&nbsp;&lt;=&nbsp;t.<span style="color:darkcyan;">RemainingSeats</span>()&nbsp;=&gt;&nbsp;t&nbsp;<span style="color:blue;">with</span>&nbsp;{&nbsp;<span style="color:purple;">Reservations</span>&nbsp;=&nbsp;t.<span style="color:purple;">Reservations</span>.Add(r)&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">_</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;}; } </pre> <p> This way, I can just point someone to your article for enlightenment, &#x1F609; but not leave them feeling frustrated that they need F# to (practice and) model around data instead of state mutating objects. It might still be worthwhile to show an F# version to draw the similarities and also call out some differences; like <code>Table</code> being a true discriminated union in F#, and while it appears to be emulated in C#, they desugar to the same thing in terms of CLR types and hierarchies. </p> <p> By the way, in the C# example above, I modeled the standard table variant differently because if it can hold only one reservation at a time then the model should reflect that. </p> </div> <div class="comment-date">2022-10-27 16:09 UTC</div> </div> <div class="comment" id="7069ea2b33a64a1caf7247c3a1543bac"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Atif, thank you for supplying and example of an immutable C# implementation. </p> <p> I already have an example of an immutable, functional C# implementation in <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>, so I wanted to supply something else here. I also tend to find it interesting to compare how to model similar ideas in different languages, and it felt natural to supply an F# example to show how a 'natural' FP implementation might look. </p> <p> Your point is valid, though, so I'm not insisting that this was the right decision. </p> </div> <div class="comment-date">2022-10-28 8:50 UTC</div> </div> <div class="comment" id="a7b4d4d0dcc8432fb3b49cb7189d8123"> <div class="comment-author"><a href="https://github.com/sebastianfrelle">Sebastian Frelle Koch</a></div> <div class="comment-content"> <p>I took your idea, Atif, and wrote something that I think is more congruent with the example <a href="#78027bc1d1414c2fa3604a68c9df6418">here</a>. In short, I’m</p> <ul> <li>using polymorphism to avoid having to switch over the Table type</li> <li>hiding subtypes of Table to simplify the interface.</li> </ul> <p>Here's the code:</p> <div class="language-cs highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="n">nullable</span> <span class="n">enable</span> <span class="k">using</span> <span class="nn">System.Collections.Immutable</span><span class="p">;</span> <span class="k">readonly</span> <span class="n">record</span> <span class="k">struct</span> <span class="nc">Reservation</span><span class="p">(</span><span class="kt">int</span> <span class="n">Quantity</span><span class="p">);</span> <span class="k">abstract</span> <span class="n">record</span> <span class="n">Table</span> <span class="p">{</span> <span class="k">public</span> <span class="k">abstract</span> <span class="n">Table</span><span class="p">?</span> <span class="nf">Reserve</span><span class="p">(</span><span class="n">Reservation</span> <span class="n">r</span><span class="p">);</span> <span class="k">public</span> <span class="k">abstract</span> <span class="kt">int</span> <span class="nf">RemainingSeats</span><span class="p">();</span> <span class="k">public</span> <span class="k">static</span> <span class="n">Table</span><span class="p">?</span> <span class="nf">Standard</span><span class="p">(</span><span class="kt">int</span> <span class="n">capacity</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="n">capacity</span> <span class="p">&gt;</span> <span class="m">0</span> <span class="p">?</span> <span class="k">new</span> <span class="nf">StandardTable</span><span class="p">(</span><span class="n">capacity</span><span class="p">,</span> <span class="k">null</span><span class="p">)</span> <span class="p">:</span> <span class="k">null</span><span class="p">;</span> <span class="k">public</span> <span class="k">static</span> <span class="n">Table</span><span class="p">?</span> <span class="nf">Communal</span><span class="p">(</span><span class="kt">int</span> <span class="n">capacity</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="n">capacity</span> <span class="p">&gt;</span> <span class="m">0</span> <span class="p">?</span> <span class="k">new</span> <span class="nf">CommunalTable</span><span class="p">(</span> <span class="n">capacity</span><span class="p">,</span> <span class="n">ImmutableArray</span><span class="p">&lt;</span><span class="n">Reservation</span><span class="p">&gt;.</span><span class="n">Empty</span><span class="p">)</span> <span class="p">:</span> <span class="k">null</span><span class="p">;</span> <span class="k">private</span> <span class="n">record</span> <span class="nf">StandardTable</span><span class="p">(</span><span class="kt">int</span> <span class="n">Capacity</span><span class="p">,</span> <span class="n">Reservation</span><span class="p">?</span> <span class="n">Reservation</span><span class="p">)</span> <span class="p">:</span> <span class="n">Table</span> <span class="p">{</span> <span class="k">public</span> <span class="k">override</span> <span class="n">Table</span><span class="p">?</span> <span class="nf">Reserve</span><span class="p">(</span><span class="n">Reservation</span> <span class="n">r</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="nf">RemainingSeats</span><span class="p">()</span> <span class="k">switch</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">seats</span> <span class="n">when</span> <span class="n">seats</span> <span class="p">&gt;=</span> <span class="n">r</span><span class="p">.</span><span class="n">Quantity</span> <span class="p">=&gt;</span> <span class="k">this</span> <span class="n">with</span> <span class="p">{</span> <span class="n">Reservation</span> <span class="p">=</span> <span class="n">r</span> <span class="p">},</span> <span class="n">_</span> <span class="p">=&gt;</span> <span class="k">null</span><span class="p">,</span> <span class="p">};</span> <span class="k">public</span> <span class="k">override</span> <span class="kt">int</span> <span class="nf">RemainingSeats</span><span class="p">()</span> <span class="p">=&gt;</span> <span class="n">Reservation</span> <span class="k">switch</span> <span class="p">{</span> <span class="k">null</span> <span class="p">=&gt;</span> <span class="n">Capacity</span><span class="p">,</span> <span class="n">_</span> <span class="p">=&gt;</span> <span class="m">0</span><span class="p">,</span> <span class="p">};</span> <span class="p">}</span> <span class="k">private</span> <span class="n">record</span> <span class="nf">CommunalTable</span><span class="p">(</span> <span class="kt">int</span> <span class="n">Capacity</span><span class="p">,</span> <span class="n">ImmutableArray</span><span class="p">&lt;</span><span class="n">Reservation</span><span class="p">&gt;</span> <span class="n">Reservations</span><span class="p">)</span> <span class="p">:</span> <span class="n">Table</span> <span class="p">{</span> <span class="k">public</span> <span class="k">override</span> <span class="n">Table</span><span class="p">?</span> <span class="nf">Reserve</span><span class="p">(</span><span class="n">Reservation</span> <span class="n">r</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="nf">RemainingSeats</span><span class="p">()</span> <span class="k">switch</span> <span class="p">{</span> <span class="kt">var</span> <span class="n">seats</span> <span class="n">when</span> <span class="n">seats</span> <span class="p">&gt;=</span> <span class="n">r</span><span class="p">.</span><span class="n">Quantity</span> <span class="p">=&gt;</span> <span class="k">this</span> <span class="n">with</span> <span class="p">{</span> <span class="n">Reservations</span> <span class="p">=</span> <span class="n">Reservations</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">r</span><span class="p">)</span> <span class="p">},</span> <span class="n">_</span> <span class="p">=&gt;</span> <span class="k">null</span><span class="p">,</span> <span class="p">};</span> <span class="k">public</span> <span class="k">override</span> <span class="kt">int</span> <span class="nf">RemainingSeats</span><span class="p">()</span> <span class="p">=&gt;</span> <span class="n">Capacity</span> <span class="p">-</span> <span class="n">Reservations</span><span class="p">.</span><span class="nf">Sum</span><span class="p">(</span><span class="n">r</span> <span class="p">=&gt;</span> <span class="n">r</span><span class="p">.</span><span class="n">Quantity</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>I’d love to hear your thoughts on this approach. I think that one of its weaknesses is that calls to <code class="language-plaintext highlighter-rouge">Table.Standard()</code> and <code class="language-plaintext highlighter-rouge">Table.Communal()</code> will yield two instances of <code class="language-plaintext highlighter-rouge">Table</code> that can never be equal. For instance, <code class="language-plaintext highlighter-rouge">Table.Standard(4) != Table.Communal(4)</code>, even though they’re both of type <code class="language-plaintext highlighter-rouge">Table?</code> and have the same number of seats. </p> <p> Calling <code class="language-plaintext highlighter-rouge">GetType()</code> on each of the instances reveals that their types are actually <code class="language-plaintext highlighter-rouge">Table+StandardTable</code> and <code class="language-plaintext highlighter-rouge">Table+CommunalTable</code> respectively; however, this isn't transparent to callers. Another solution might be to expose the <code class="language-plaintext highlighter-rouge">Table</code> subtypes and give them private constructors &ndash; I just like the simplicity of not exposing the individual types of tables the same way you’re doing <a href="#b37e173371f249d99f80d12d64b2bee2">here</a>, Mark.</p> </div> <div class="comment-date">2022-11-29 11:28 UTC</div> </div> <div class="comment" id="f925c18ec3a746c393cfae319200baac"> <div class="comment-author"><a href="https://github.com/alexmurari">Alexandre Murari Jr</a></div> <div class="comment-content"> <p> Mark, <p> How do you differentiate encapsulation from abstraction? </p> <p> Here's an excerpt from your book Dependency Injection: Principles, Practices, and Patterns. </p> <p> Section: 1.3 - What to inject and what not to inject Subsection: 1.3.1 - Stable Dependencies </p> <blockquote> "Other examples [of libraries that do not require to be injected] may include specialized libraries that encapsulate alogorithms relevant to your application". </blockquote> <p> In that section, you and Steven were giving examples of stable dependencies that do not require to be injected to keep modularity. You define a library that "encapsulates an algorithm" as an example. </p> <p> Now, to me, encapsulation is "protecting data integrity", plain and simple. A class is encapsulated as long as it's impossible or nearly impossible to bring it to an invalid or inconsistent state. </p> <p> Protection of invariants, implementation hiding, bundling data and operations together, pre- and postconditions, Postel's Law all come into play to achieve this goal. </p> <p> Thus, a class, to be "encapsulatable", has to have a state that can be initialized and/or modified by the client code. </p> <p> Now I ask: most of the time when we say that something is encapsulating another, don't we really mean abstracting? </p> <p> Why is it relevant to know that the hypothetical algorithm library protects it's invariants by using the term "encapsulate"? </p> <p> Abstraction, under the light of Robert C. Martin's definition of it, makes much more sense in that context: "a specialized library that abstracts algorithms relevant to your application". It amplifies the essential (by providing a clear API), but eliminates the irrelevant (by hiding the alogirthm's implementation details). </p> <p> Granted, there is some overlap between encapsulation and abstraction, specially when you bundle data and operations together (rich domain models), but they are not the same thing, you just use one to achieve another sometimes. </p> <p> Would it be correct to say that the .NET Framework encapsulates math algorithms in the System.Math class? Is there any state there to be preserved? They're all static methods and constants. On the other hand, they're surely eliminating some pretty irrelevant (from a consumer POV) trigonometric algorithms. </p> <p> Thanks. </p> </div> <div class="comment-date">2022-12-04 02:35 UTC</div> </div> <div class="comment" id="5af1535933bf4b28bb5c2fc14ce0a01a"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Alexandre, thank you for writing. How do I distinguish between abstraction and encapsulation? </p> <p> There's much overlap, to be sure. </p> <p> As I write, my view on encapsulation is influenced by Bertrand Meyer's notion of contract. Likewise, I do use Robert C. Martin's notion of amplifying the essentials while hiding the irrelevant details as a guiding light when discussing abstraction. </p> <p> While these concepts may seem synonymous, they're not quite the same. I can't say that I've spent too much time considering how these two words relate, but shooting from the hip I think that <em>abstraction</em> is a wider concept. </p> <p> You don't need to read much of Robert C. Martin before he'll tell you that the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a> is an important part of abstraction: </p> <blockquote> <p> "Abstractions should not depend on details. Details should depend on abstractions." </p> <footer><cite>Robert C. Martin, <a href="/ref/appp">Agile Principles, Patterns, and Practices in C#</a></cite></footer> </blockquote> <p> It's possible to implement a code base where this isn't true, even if classes have good encapsulation. You could imagine a domain model that depends on database details like a particular ORM. I've seen plenty of those in my career, although I grant that most of them have had poor encapsulation as well. It is not, however, impossible to imagine such a system with good encapsulation, but suboptimal abstraction. </p> <p> Does it go the other way as well? Can we have good abstraction, but poor encapsulation? </p> <p> An example doesn't come immediately to mind, but as I wrote, it's not an ontology that I've given much thought. </p> </div> <div class="comment-date">2022-12-06 22:11 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Stubs and mocks break encapsulation https://blog.ploeh.dk/2022/10/17/stubs-and-mocks-break-encapsulation 2022-10-17T08:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Favour Fakes over dynamic mocks.</em> </p> <p> For a while now, I've <a href="/2019/02/18/from-interaction-based-to-state-based-testing">favoured Fakes over Stubs and Mocks</a>. Using <a href="http://xunitpatterns.com/Fake%20Object.html">Fake Objects</a> over other <a href="https://martinfowler.com/bliki/TestDouble.html">Test Doubles</a> makes test suites more robust. I wrote the code base for my book <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a> entirely with Fakes and the occasional <a href="http://xunitpatterns.com/Test%20Spy.html">Test Spy</a>, and I rarely had to fix broken tests. No <a href="https://moq.github.io/moq4/">Moq</a>, <a href="https://fakeiteasy.github.io/">FakeItEasy</a>, <a href="https://nsubstitute.github.io/">NSubstitute</a>, nor <a href="https://hibernatingrhinos.com/oss/rhino-mocks">Rhino Mocks</a>. Just hand-written Test Doubles. </p> <p> It recently occurred to me that a way to explain the problem with <a href="http://xunitpatterns.com/Mock%20Object.html">Mocks</a> and <a href="http://xunitpatterns.com/Test%20Stub.html">Stubs</a> is that they break encapsulation. </p> <p> You'll see some examples soon, but first it's important to be explicit about terminology. </p> <h3 id="bde25ebecc664e99b529755c3f0829fb"> Terminology <a href="#bde25ebecc664e99b529755c3f0829fb" title="permalink">#</a> </h3> <p> Words like <em>Mocks</em>, <em>Stubs</em>, as well as <em>encapsulation</em>, have different meanings to different people. They've fallen victim to <a href="https://martinfowler.com/bliki/SemanticDiffusion.html">semantic diffusion</a>, if ever they were well-defined to begin with. </p> <p> When I use the words <em>Test Double</em>, <em>Fake</em>, <em>Mock</em>, and <em>Stub</em>, I use them as they are defined in <a href="/ref/xunit-patterns">xUnit Test Patterns</a>. I usually try to avoid the terms <em>Mock</em> and <em>Stub</em> since people use them vaguely and inconsistently. The terms <em>Test Double</em> and <em>Fake</em> fare better. </p> <p> We do need, however, a name for those libraries that generate Test Doubles on the fly. In .NET, they are libraries like Moq, FakeItEasy, and so on, as listed above. Java has <a href="https://site.mockito.org/">Mockito</a>, <a href="https://easymock.org/">EasyMock</a>, <a href="https://jmockit.github.io/">JMockit</a>, and possibly more like that. </p> <p> What do we call such libraries? Most people call them <em>mock libraries</em> or <em>dynamic mock libraries</em>. Perhaps <em>dynamic Test Double library</em> would be more consistent with the <em>xUnit Test Patterns</em> vocabulary, but nobody calls them that. I'll call them <em>dynamic mock libraries</em> to at least emphasise the dynamic, on-the-fly object generation these libraries typically use. </p> <p> Finally, it's important to define <em>encapsulation</em>. This is another concept where people may use the same word and yet mean different things. </p> <p> I base my understanding of encapsulation on <a href="/ref/oosc">Object-Oriented Software Construction</a>. I've tried to distil it in my Pluralsight course <a href="/encapsulation-and-solid">Encapsulation and SOLID</a>. </p> <p> In short, encapsulation denotes the distinction between an object's contract and its implementation. An object should fulfil its contract in such a way that client code doesn't need to know about its implementation. </p> <p> Contracts, according to Meyer, describe three properties of objects: </p> <ul> <li>Preconditions: What client code must fulfil in order to successfully interact with the object.</li> <li>Invariants: Statements about the object that are always true.</li> <li>Postconditions: Statements that are guaranteed to be true after a successful interaction between client code and object.</li> </ul> <p> As I'll demonstrate in this article, objects generated by dynamic mock libraries often break their contracts. </p> <h3 id="2d05b5926474448589c3953f6796c2a9"> Create-and-read round-trip <a href="#2d05b5926474448589c3953f6796c2a9" title="permalink">#</a> </h3> <p> Consider the <code>IReservationsRepository</code> interface from <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span> { &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;Create(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;Task&lt;IReadOnlyCollection&lt;Reservation&gt;&gt;&nbsp;ReadReservations( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;DateTime&nbsp;min,&nbsp;DateTime&nbsp;max); &nbsp;&nbsp;&nbsp;&nbsp;Task&lt;Reservation?&gt;&nbsp;ReadReservation(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Guid&nbsp;id); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;Update(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;Delete(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Guid&nbsp;id); }</pre> </p> <p> I already discussed some of the contract properties of this interface in <a href="/2021/12/06/the-liskov-substitution-principle-as-a-profunctor">an earlier article</a>. Here, I want to highlight a certain interaction. </p> <p> What is the contract of the <code>Create</code> method? </p> <p> There are a few preconditions: </p> <ul> <li>The client must have a properly initialised <code>IReservationsRepository</code> object.</li> <li>The client must have a valid <code>restaurantId</code>.</li> <li>The client must have a valid <code>reservation</code>.</li> </ul> <p> A client that fulfils these preconditions can successfully call and await the <code>Create</code> method. What are the invariants and postconditions? </p> <p> I'll skip the invariants because they aren't relevant to the line of reasoning that I'm pursuing. One postcondition, however, is that the <code>reservation</code> passed to <code>Create</code> must now be 'in' the repository. </p> <p> How does that manifest as part of the object's contract? </p> <p> This implies that a client should be able to retrieve the <code>reservation</code>, either with <code>ReadReservation</code> or <code>ReadReservations</code>. This suggests a kind of property that Scott Wlaschin calls <a href="https://fsharpforfunandprofit.com/posts/property-based-testing-2/">There and back again</a>. </p> <p> Picking <code>ReadReservation</code> for the verification step we now have a property: If client code successfully calls and awaits <code>Create</code> it should be able to use <code>ReadReservation</code> to retrieve the reservation it just saved. That's implied by the <code>IReservationsRepository</code> contract. </p> <h3 id="f787d89599a04573a2c9b7afc28b2315"> SQL implementation <a href="#f787d89599a04573a2c9b7afc28b2315" title="permalink">#</a> </h3> <p> The 'real' implementation of <code>IReservationsRepository</code> used in production is an implementation that stores reservations in SQL Server. This class should obey the contract. </p> <p> While it might be possible to write a true property-based test, running hundreds of randomly generated test cases against a real database is going to take time. Instead, I chose to only write a parametrised test: </p> <p> <pre>[Theory] [InlineData(Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-06-29&nbsp;12:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;e@example.gov&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Enigma&quot;</span>,&nbsp;1)] [InlineData(Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-07-27&nbsp;11:40&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;c@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Carlie&quot;</span>,&nbsp;2)] [InlineData(2,&nbsp;<span style="color:#a31515;">&quot;2021-09-03&nbsp;14:32&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;bon@example.edu&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Jovi&quot;</span>,&nbsp;4)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;CreateAndReadRoundTrip( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Guid.NewGuid(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;DateTime.Parse(at,&nbsp;CultureInfo.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(name), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;connectionString&nbsp;=&nbsp;ConnectionStrings.Reservations; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;SqlReservationsRepository(connectionString); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual); }</pre> </p> <p> The part that we care about is the three last lines: </p> <p> <pre><span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); Assert.Equal(expected,&nbsp;actual);</pre> </p> <p> First call <code>Create</code> and subsequently <code>ReadReservation</code>. The value created should equal the value retrieved, which is also the case. All tests pass. </p> <h3 id="dc7d9e2730364cf8895d73d2adcd37b0"> Fake <a href="#dc7d9e2730364cf8895d73d2adcd37b0" title="permalink">#</a> </h3> <p> The Fake implementation is effectively an in-memory database, so we expect it to also fulfil the same contract. We can test it with an almost identical test: </p> <p> <pre>[Theory] [InlineData(RestApi.Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-06-29&nbsp;12:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;e@example.gov&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Enigma&quot;</span>,&nbsp;1)] [InlineData(RestApi.Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-07-27&nbsp;11:40&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;c@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Carlie&quot;</span>,&nbsp;2)] [InlineData(2,&nbsp;<span style="color:#a31515;">&quot;2021-09-03&nbsp;14:32&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;bon@example.edu&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Jovi&quot;</span>,&nbsp;4)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;CreateAndReadRoundTrip( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Guid.NewGuid(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;DateTime.Parse(at,&nbsp;CultureInfo.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(name), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;FakeDatabase(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual); }</pre> </p> <p> The only difference is that the <code>sut</code> is a different class instance. These test cases also all pass. </p> <p> How is <code>FakeDatabase</code> implemented? That's not important, because it obeys the contract. <code>FakeDatabase</code> has good encapsulation, which makes it possible to use it without knowing anything about its internal implementation details. That, after all, is the point of encapsulation. </p> <h3 id="a13bc3a894914ce59b1c11ea9702b3f8"> Dynamic mock <a href="#a13bc3a894914ce59b1c11ea9702b3f8" title="permalink">#</a> </h3> <p> How does a dynamic mock fare if subjected to the same test? Let's try with Moq 4.18.2 (and I'm not choosing Moq to single it out - I chose Moq because it's the dynamic mock library I used to love the most): </p> <p> <pre>[Theory] [InlineData(RestApi.Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-06-29&nbsp;12:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;e@example.gov&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Enigma&quot;</span>,&nbsp;1)] [InlineData(RestApi.Grandfather.Id,&nbsp;<span style="color:#a31515;">&quot;2022-07-27&nbsp;11:40&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;c@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Carlie&quot;</span>,&nbsp;2)] [InlineData(2,&nbsp;<span style="color:#a31515;">&quot;2021-09-03&nbsp;14:32&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;bon@example.edu&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Jovi&quot;</span>,&nbsp;4)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;CreateAndReadRoundTrip( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Guid.NewGuid(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;DateTime.Parse(at,&nbsp;CultureInfo.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(name), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Mock&lt;IReservationsRepository&gt;().Object; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual); }</pre> </p> <p> If you've worked a little with dynamic mock libraries, you will not be surprised to learn that all three tests fail. Here's one of the failure messages: </p> <p> <pre>Ploeh.Samples.Restaurants.RestApi.Tests.MoqRepositoryTests.CreateAndReadRoundTrip(↩ &nbsp;&nbsp;&nbsp;&nbsp;restaurantId:&nbsp;1,&nbsp;at:&nbsp;&quot;2022-06-29&nbsp;12:00&quot;,&nbsp;email:&nbsp;&quot;e@example.gov&quot;,&nbsp;name:&nbsp;&quot;Enigma&quot;,&nbsp;quantity:&nbsp;1) &nbsp;Source:&nbsp;<span style="color:blue;">MoqRepositoryTests.cs</span>&nbsp;line&nbsp;17 &nbsp;Duration:&nbsp;1&nbsp;ms Message:&nbsp; &nbsp;&nbsp;Assert.Equal()&nbsp;Failure &nbsp;&nbsp;Expected:&nbsp;Reservation↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;2022-06-29T12:00:00.0000000,↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;e@example.gov,↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;c9de4f95-3255-4e1f-a1d6-63591b58ff0c,↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;Enigma,↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;1↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;Actual:&nbsp;&nbsp;&nbsp;(null) Stack&nbsp;Trace:&nbsp; &nbsp;&nbsp;<span style="color:red;">MoqRepositoryTests.CreateAndReadRoundTrip(↩ &nbsp;&nbsp;&nbsp;&nbsp;Int32&nbsp;restaurantId,&nbsp;String&nbsp;at,&nbsp;String&nbsp;email,&nbsp;String&nbsp;name,&nbsp;Int32&nbsp;quantity)</span>&nbsp;line&nbsp;35 &nbsp;&nbsp;---&nbsp;End&nbsp;of&nbsp;stack&nbsp;trace&nbsp;from&nbsp;previous&nbsp;location&nbsp;where&nbsp;exception&nbsp;was&nbsp;thrown&nbsp;---</pre> </p> <p> (I've introduced line breaks and indicated them with the ↩ symbol to make the output more readable. I'll do that again later in the article.) </p> <p> Not surprisingly, the return value of <code>Create</code> is null. You typically have to configure a dynamic mock in order to give it any sort of behaviour, and I didn't do that here. In that case, the dynamic mock returns the default value for the return type, which in this case correctly is null. </p> <p> You may object that the above example is unfair. How can a dynamic mock know what to do? You have to configure it. That's the whole point of it. </p> <h3 id="307eb992e4c54aae8a2f331c05d9d7e4"> Retrieval without creation <a href="#307eb992e4c54aae8a2f331c05d9d7e4" title="permalink">#</a> </h3> <p> Okay, let's set up the dynamic mock: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;dm&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Mock&lt;IReservationsRepository&gt;(); dm.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservation(restaurantId,&nbsp;expected.Id)).ReturnsAsync(expected); <span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;dm.Object;</pre> </p> <p> These are the only lines I've changed from the previous listing of the test, which now passes. </p> <p> A common criticism of dynamic-mock-heavy tests is that they mostly 'just test the mocks', and this is exactly what happens here. </p> <p> You can make that more explicit by deleting the <code>Create</code> method call: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;dm&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Mock&lt;IReservationsRepository&gt;(); dm.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservation(restaurantId,&nbsp;expected.Id)).ReturnsAsync(expected); <span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;dm.Object; <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); Assert.Equal(expected,&nbsp;actual);</pre> </p> <p> The test still passes. Clearly it only tests the dynamic mock. </p> <p> You may, again, demur that this is expected, and it doesn't demonstrate that dynamic mocks break encapsulation. Keep in mind, however, the nature of the contract: Upon successful completion of <code>Create</code>, the reservation is 'in' the repository and can later be retrieved, either with <code>ReadReservation</code> or <code>ReadReservations</code>. </p> <p> This variation of the test no longer calls <code>Create</code>, yet <code>ReadReservation</code> still returns the <code>expected</code> value. </p> <p> Do <code>SqlReservationsRepository</code> or <code>FakeDatabase</code> behave like that? No, they don't. </p> <p> Try to delete the <code>Create</code> call from the test that exercises <code>SqlReservationsRepository</code>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;SqlReservationsRepository(connectionString); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); Assert.Equal(expected,&nbsp;actual);</pre> </p> <p> Hardly surprising, the test now fails because <code>actual</code> is null. The same happens if you delete the <code>Create</code> call from the test that exercises <code>FakeDatabase</code>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;FakeDatabase(); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); Assert.Equal(expected,&nbsp;actual);</pre> </p> <p> Again, the assertion fails because <code>actual</code> is null. </p> <p> The classes <code>SqlReservationsRepository</code> and <code>FakeDatabase</code> behave according to contract, while the dynamic mock doesn't. </p> <h3 id="81c7041024ad4c5d99055494c5b0bdf0"> Alternative retrieval <a href="#81c7041024ad4c5d99055494c5b0bdf0" title="permalink">#</a> </h3> <p> There's another way in which the dynamic mock breaks encapsulation. Recall what the contract states: Upon successful completion of <code>Create</code>, the reservation is 'in' the repository and can later be retrieved, either with <code>ReadReservation</code> or <code>ReadReservations</code>. </p> <p> In other words, it should be possible to change the interaction from <code>Create</code> followed by <code>ReadReservation</code> to <code>Create</code> followed by <code>ReadReservations</code>. </p> <p> First, try it with <code>SqlReservationsRepository</code>: </p> <p> <pre><span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); <span style="color:blue;">var</span>&nbsp;min&nbsp;=&nbsp;expected.At.Date; <span style="color:blue;">var</span>&nbsp;max&nbsp;=&nbsp;min.AddDays(1); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservations(restaurantId,&nbsp;min,&nbsp;max); Assert.Contains(expected,&nbsp;actual);</pre> </p> <p> The test still passes, as expected. </p> <p> Second, try the same change with <code>FakeDatabase</code>: </p> <p> <pre><span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); <span style="color:blue;">var</span>&nbsp;min&nbsp;=&nbsp;expected.At.Date; <span style="color:blue;">var</span>&nbsp;max&nbsp;=&nbsp;min.AddDays(1); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservations(restaurantId,&nbsp;min,&nbsp;max); Assert.Contains(expected,&nbsp;actual);</pre> </p> <p> Notice that this is the exact same code as in the <code>SqlReservationsRepository</code> test. That test also passes, as expected. </p> <p> Third, try it with the dynamic mock: </p> <p> <pre><span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); <span style="color:blue;">var</span>&nbsp;min&nbsp;=&nbsp;expected.At.Date; <span style="color:blue;">var</span>&nbsp;max&nbsp;=&nbsp;min.AddDays(1); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservations(restaurantId,&nbsp;min,&nbsp;max); Assert.Contains(expected,&nbsp;actual);</pre> </p> <p> Same code, different <code>sut</code>, and the test fails. The dynamic mock breaks encapsulation. You'll have to go and fix the <code>Setup</code> of it to make the test pass again. That's not the case with <code>SqlReservationsRepository</code> or <code>FakeDatabase</code>. </p> <h3 id="09410dcfb758412d991a47f08be885b5"> Dynamic mocks break the SUT, not the tests <a href="#09410dcfb758412d991a47f08be885b5" title="permalink">#</a> </h3> <p> Perhaps you're still not convinced that this is of practical interest. After all, <a href="https://en.wikipedia.org/wiki/Bertrand_Meyer">Bertrand Meyer</a> had limited success getting mainstream adoption of his thought on contract-based programming. </p> <p> That dynamic mocks break encapsulation does, however, have real implications. </p> <p> What if, instead of using <code>FakeDatabase</code>, I'd used dynamic mocks when testing my online restaurant reservation system? A test might have looked like this: </p> <p> <pre>[Theory] [InlineData(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [InlineData(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [InlineData(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [InlineData(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;DateTime.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dm&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Mock&lt;IReservationsRepository&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;dm.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservations(Grandfather.Id,&nbsp;at.Date,&nbsp;at.Date.AddDays(1).AddTicks(-1))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReturnsAsync(Array.Empty&lt;Reservation&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ReservationsController( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;SystemClock(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;InMemoryRestaurantDatabase(Grandfather.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dm.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Guid(<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(expected.ToDto()); &nbsp;&nbsp;&nbsp;&nbsp;dm.Verify(r&nbsp;=&gt;&nbsp;r.Create(Grandfather.Id,&nbsp;expected)); }</pre> </p> <p> This is yet another riff on the <code>PostValidReservationWhenDatabaseIsEmpty</code> test - the gift that keeps giving. I've previously discussed this test in other articles: </p> <ul> <li><a href="/2020/12/07/branching-tests">Branching tests</a></li> <li><a href="/2021/01/11/waiting-to-happen">Waiting to happen</a></li> <li><a href="/2021/01/18/parametrised-test-primitive-obsession-code-smell">Parametrised test primitive obsession code smell</a></li> <li><a href="/2021/09/27/the-equivalence-contravariant-functor">The Equivalence contravariant functor</a></li> </ul> <p> Here I've replaced the <code>FakeDatabase</code> Test Double with a dynamic mock. (I am, again, using Moq, but keep in mind that the fallout of using a dynamic mock is unrelated to specific libraries.) </p> <p> To go 'full dynamic mock' I should also have replaced <code>SystemClock</code> and <code>InMemoryRestaurantDatabase</code> with dynamic mocks, but that's not necessary to illustrate the point I wish to make. </p> <p> This, and other tests, describe the desired outcome of making a reservation against the REST API. It's an interaction that looks like this: </p> <p> <pre>POST /restaurants/90125/reservations?sig=aco7VV%2Bh5sA3RBtrN8zI8Y9kLKGC60Gm3SioZGosXVE%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-12-12T20:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Pearl&nbsp;Yvonne&nbsp;Gates&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;pearlygates@example.net&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 } HTTP/1.1 201 Created Content-Length: 151 Content-Type: application/json; charset=utf-8 Location: [...]/restaurants/90125/reservations/82e550b1690742368ea62d76e103b232?sig=fPY1fSr[...] { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;82e550b1690742368ea62d76e103b232&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-12-12T20:00:00.0000000&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;pearlygates@example.net&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Pearl&nbsp;Yvonne&nbsp;Gates&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 }</pre> </p> <p> What's of interest here is that the response includes the JSON representation of the resource that the interaction created. It's mostly a copy of the posted data, but enriched with a server-generated ID. </p> <p> The code responsible for the database interaction looks like this: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&lt;ActionResult&gt;&nbsp;TryCreate(Restaurant&nbsp;restaurant,&nbsp;Reservation&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;scope&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TransactionScope(TransactionScopeAsyncFlowOption.Enabled); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations(restaurant.Id,&nbsp;reservation.At) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(restaurant.Id,&nbsp;reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;scope.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(restaurant.Id,&nbsp;reservation); }</pre> </p> <p> The last line of code creates a <code>201 Created</code> response with the <code>reservation</code> as content. Not shown in this snippet is the origin of the <code>reservation</code> parameter, but it's <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">the input JSON document parsed to a <code>Reservation</code> object</a>. Each <code>Reservation</code> object has <a href="/2022/09/12/coalescing-dtos">an ID that the server creates when it's not supplied by the client</a>. </p> <p> The above <code>TryCreate</code> helper method contains all the database interaction code related to creating a new reservation. It first calls <code>ReadReservations</code> to retrieve the existing reservations. Subsequently, it calls <code>Create</code> if it decides to accept the reservation. The <code>ReadReservations</code> method is actually an <code>internal</code> extension method: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Task&lt;IReadOnlyCollection&lt;Reservation&gt;&gt;&nbsp;ReadReservations( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IReservationsRepository&nbsp;repository, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;DateTime&nbsp;date) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;min&nbsp;=&nbsp;date.Date; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;max&nbsp;=&nbsp;min.AddDays(1).AddTicks(-1); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;repository.ReadReservations(restaurantId,&nbsp;min,&nbsp;max); }</pre> </p> <p> Notice how the dynamic-mock-based test has to replicate this <code>internal</code> implementation detail to the <a href="https://docs.microsoft.com/dotnet/api/system.datetime.ticks">tick</a>. If I ever decide to change this just one tick, the test is going to fail. That's already bad enough (and something that <code>FakeDatabase</code> gracefully handles), but not what I'm driving towards. </p> <p> At the moment the <code>TryCreate</code> method echoes back the <code>reservation</code>. What if, however, you instead want to query the database and return the record that you got from the database? In this particular case, there's no reason to do that, but perhaps in other cases, something happens in the data layer that either enriches or normalises the data. So you make an innocuous change: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&lt;ActionResult&gt;&nbsp;TryCreate(Restaurant&nbsp;restaurant,&nbsp;Reservation&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;scope&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TransactionScope(TransactionScopeAsyncFlowOption.Enabled); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations(restaurant.Id,&nbsp;reservation.At) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(restaurant.Id,&nbsp;reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;storedReservation&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservation(restaurant.Id,&nbsp;reservation.Id) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;scope.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(restaurant.Id,&nbsp;storedReservation!); }</pre> </p> <p> Now, instead of echoing back <code>reservation</code>, the method calls <code>ReadReservation</code> to retrieve the (possibly enriched or normalised) <code>storedReservation</code> and returns that value. Since this value could, conceivably, be null, for now the method uses the <code>!</code> operator to insist that this is not the case. A new test case might be warranted to cover the scenario where the query returns null. </p> <p> This is perhaps a little less efficient because it implies an extra round-trip to the database, but <em>it shouldn't change the behaviour of the system!</em> </p> <p> But when you run the test suite, that <code>PostValidReservationWhenDatabaseIsEmpty</code> test fails: </p> <p> <pre>Ploeh.Samples.Restaurants.RestApi.Tests.ReservationsTests.PostValidReservationWhenDatabaseIsEmpty(↩ days: 433, hours: 17, minutes: 30, email: "shli@example.org", name: "Shanghai Li", quantity: 5)↩ [FAIL] System.NullReferenceException : Object reference not set to an instance of an object. Stack Trace: [...]\Restaurant.RestApi\ReservationsController.cs(94,0): at↩ [...].RestApi.ReservationsController.Reservation201Created↩ (Int32 restaurantId, Reservation r) [...]\Restaurant.RestApi\ReservationsController.cs(79,0): at↩ [...].RestApi.ReservationsController.TryCreate↩ (Restaurant restaurant, Reservation reservation) [...]\Restaurant.RestApi\ReservationsController.cs(57,0): at↩ [...].RestApi.ReservationsController.Post↩ (Int32 restaurantId, ReservationDto dto) [...]\Restaurant.RestApi.Tests\ReservationsTests.cs(73,0): at↩ [...].RestApi.Tests.ReservationsTests.PostValidReservationWhenDatabaseIsEmpty↩ (Int32 days, Int32 hours, Int32 minutes, String email, String name, Int32 quantity) --- End of stack trace from previous location where exception was thrown ---</pre> </p> <p> Oh, the dreaded <code>NullReferenceException</code>! This happens because <code>ReadReservation</code> returns null, since the dynamic mock isn't configured. </p> <p> The typical reaction that most people have is: <em>Oh no, the tests broke!</em> </p> <p> I think, though, that this is the wrong perspective. The dynamic mock broke the System Under Test (SUT) because it passed an implementation of <code>IReservationsRepository</code> that breaks the contract. The test didn't 'break', because it was never correct from the outset. </p> <h3 id="f720608a0e3b45aa88f1831a4bb1f4e8"> Shotgun surgery <a href="#f720608a0e3b45aa88f1831a4bb1f4e8" title="permalink">#</a> </h3> <p> When a test code base uses dynamic mocks, it tends to do so pervasively. Most tests create one or more dynamic mocks that they pass to their SUT. Most of these dynamic mocks break encapsulation, so when you refactor, the dynamic mocks break the SUT. </p> <p> You'll typically need to revisit and 'fix' all the failing tests to accommodate the refactoring: </p> <p> <pre>[Theory] [InlineData(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [InlineData(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [InlineData(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [InlineData(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;DateTime.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Guid(<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dm&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Mock&lt;IReservationsRepository&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;dm.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservations(Grandfather.Id,&nbsp;at.Date,&nbsp;at.Date.AddDays(1).AddTicks(-1))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReturnsAsync(Array.Empty&lt;Reservation&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;dm.Setup(r&nbsp;=&gt;&nbsp;r.ReadReservation(Grandfather.Id,&nbsp;expected.Id)).ReturnsAsync(expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ReservationsController( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;SystemClock(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;InMemoryRestaurantDatabase(Grandfather.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dm.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(expected.ToDto()); &nbsp;&nbsp;&nbsp;&nbsp;dm.Verify(r&nbsp;=&gt;&nbsp;r.Create(Grandfather.Id,&nbsp;expected)); }</pre> </p> <p> The test now passes (until the next change in the SUT), but notice how top-heavy it becomes. That's a test code smell when using dynamic mocks. Everything has to happen in <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">the Arrange phase</a>. </p> <p> You typically have many such tests that you need to edit. The name of this antipattern is <a href="https://en.wikipedia.org/wiki/Shotgun_surgery">Shotgun Surgery</a>. </p> <p> The implication is that <em>refactoring</em> by definition is impossible: </p> <blockquote> <p> "to refactor, the essential precondition is [...] solid tests" </p> <footer><cite>Martin Fowler, <a href="/ref/refactoring">Refactoring</a></cite></footer> </blockquote> <p> You need tests that don't break when you refactor. When you use dynamic mocks, tests tend to fail whenever you make changes in SUTs. Even though you have tests, they don't enable refactoring. </p> <p> To add spite to injury, <a href="/2013/04/02/why-trust-tests">every time you edit existing tests, they become less trustworthy</a>. </p> <p> To address these problems, use Fakes instead of Mocks and Stubs. With the <code>FakeDatabase</code> the entire sample test suite for the online restaurant reservation system gracefully handles the change described above. No tests fail. </p> <h3 id="8b30b068b4a4412ca3a797de2760b2be"> Spies <a href="#8b30b068b4a4412ca3a797de2760b2be" title="permalink">#</a> </h3> <p> If you spelunk the test code base for the book, you may also find this Test Double: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SpyPostOffice</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;Collection&lt;SpyPostOffice.Observation&gt;,&nbsp;IPostOffice { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;EmailReservationCreated( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;Observation(Event.Created,&nbsp;restaurantId,&nbsp;reservation)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;EmailReservationDeleted( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;Observation(Event.Deleted,&nbsp;restaurantId,&nbsp;reservation)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;EmailReservationUpdating( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;Observation(Event.Updating,&nbsp;restaurantId,&nbsp;reservation)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;EmailReservationUpdated( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;Observation(Event.Updated,&nbsp;restaurantId,&nbsp;reservation)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">enum</span>&nbsp;<span style="color:#2b91af;">Event</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Created&nbsp;=&nbsp;0, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Updating, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Updated, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Deleted &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Observation</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Observation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Event&nbsp;@event, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Event&nbsp;=&nbsp;@event; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RestaurantId&nbsp;=&nbsp;restaurantId; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;=&nbsp;reservation; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Event&nbsp;Event&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;RestaurantId&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Reservation&nbsp;Reservation&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>?&nbsp;obj) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;Observation&nbsp;observation&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Event&nbsp;==&nbsp;observation.Event&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RestaurantId&nbsp;==&nbsp;observation.RestaurantId&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;Reservation&gt;.Default.Equals(Reservation,&nbsp;observation.Reservation); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;GetHashCode() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;HashCode.Combine(Event,&nbsp;RestaurantId,&nbsp;Reservation); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As you can see, I've chosen to name this class with the <em>Spy</em> prefix, indicating that this is a Test Spy rather than a Fake Object. A Spy is a Test Double whose main purpose is to observe and record interactions. Does that break or realise encapsulation? </p> <p> While I favour Fakes whenever possible, consider the interface that <code>SpyPostOffice</code> implements: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IPostOffice</span> { &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;EmailReservationCreated(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;EmailReservationDeleted(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;EmailReservationUpdating(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;EmailReservationUpdated(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;Reservation&nbsp;reservation); }</pre> </p> <p> This interface consist entirely of <a href="https://en.wikipedia.org/wiki/Command%E2%80%93query_separation">Commands</a>. There's no way to query the interface to examine the state of the object. Thus, you can't check that postconditions hold <em>exclusively via the interface</em>. Instead, you need an additional <a href="http://xunitpatterns.com/Test%20Spy.html">retrieval interface</a> to examine the posterior state of the object. The <code>SpyPostOffice</code> concrete class exposes such an interface. </p> <p> In a sense, you can view <code>SpyPostOffice</code> as an in-memory message sink. It fulfils the contract. </p> <h3 id="b36b228b2b4f4187918c8c224c9aed04"> Concurrency <a href="#b36b228b2b4f4187918c8c224c9aed04" title="permalink">#</a> </h3> <p> Perhaps you're still not convinced. You may argue, for example, that the (partial) contract that I stated is naive. Consider, again, the implications expressed as code: </p> <p> <pre><span style="color:blue;">await</span>&nbsp;sut.Create(restaurantId,&nbsp;expected); <span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.ReadReservation(restaurantId,&nbsp;expected.Id); Assert.Equal(expected,&nbsp;actual);</pre> </p> <p> You may argue that in the face of concurrency, another thread or process could be making changes to the reservation <em>after</em> <code>Create</code>, but <em>before</em> <code>ReadReservation</code>. Thus, you may argue, the contract I've stipulated is false. In a real system, we can't expect that to be the case. </p> <p> I agree. </p> <p> Concurrency makes things much harder. Even in that light, I think the above line of reasoning is appropriate, for two reasons. </p> <p> First, I chose to model <code>IReservationsRepository</code> like I did because I didn't expect high contention on individual reservations. In other words, I don't expect two or more concurrent processes to attempt to modify <em>the same reservation</em> at the same time. Thus, I found it appropriate to model the Repository as </p> <blockquote> <p> "a collection-like interface for accessing domain objects." </p> <footer><cite>Edward Hieatt and Rob Mee, in Martin Fowler, <a href="/ref/peaa">Patterns of Enterprise Application Architecture</a>, <em>Repository</em> pattern</cite></footer> </blockquote> <p> A <em>collection-like interface</em> implies both data retrieval and collection manipulation members. In low-contention scenarios like the reservation system, this turns out to be a useful model. As the aphorism goes, <a href="https://en.wikipedia.org/wiki/All_models_are_wrong">all models are wrong, but some models are useful</a>. Treating <code>IReservationsRepository</code> as a collection accessed in a non-concurrent manner turned out to be useful in this code base. </p> <p> Had I been more worried about data contention, a move towards <a href="https://martinfowler.com/bliki/CQRS.html">CQRS</a> seems promising. This leads to another object model, with different contracts. </p> <p> Second, even in the face of concurrency, most unit test cases are implicitly running on a single thread. While they may run in parallel, each unit test exercises the SUT on a single thread. This implies that reads and writes against Test Doubles are serialised. </p> <p> Even if concurrency is a real concern, you'd still expect that <em>if only one thread is manipulating the Repository object</em>, then what you <code>Create</code> you should be able to retrieve. The contract may be a little looser, but it'd still be a violation of the <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">principle of least surprise</a> if it was any different. </p> <h3 id="7db1c826945e417db7eacf65ac5207bb"> Conclusion <a href="#7db1c826945e417db7eacf65ac5207bb" title="permalink">#</a> </h3> <p> In object-oriented programming, encapsulation is the notion of separating the affordances of an object from its implementation details. I find it most practical to think about this in terms of contracts, which again can be subdivided into sets of preconditions, invariants, and postconditions. </p> <p> Polymorphic objects (like interfaces and base classes) come with contracts as well. When you replace 'real' implementations with Test Doubles, the Test Doubles should also fulfil the contracts. Fake objects do that; Test Spies may also fit that description. </p> <p> When Test Doubles obey their contracts, you can refactor your SUT without breaking your test suite. </p> <p> By default, however, dynamic mocks break encapsulation because they don't fulfil the objects' contracts. This leads to fragile tests. </p> <p> Favour Fakes over dynamic mocks. You can read more about this way to write tests by following many of the links in this article, or by reading my book <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="6b55998427954772bed6da4ed8be8694"> <div class="comment-author">Matthew Wiemer</div> <div class="comment-content"> <p> Excellent article exploring the nuances of encapsulation as it relates to testing. That said, the examples here left me with one big question: what exactly is covered by the tests using `FakeDatabase`? </p> <p>This line in particular is confusing me (as to its practical use in a "real-world" setting): `var sut = new FakeDatabase();`</p> <p> How can I claim to have tested the real system's implementation when the "system under test" is, in this approach, explicitly _not_ my real system? It appears the same criticism of dynamic mocks surfaces: "you're only testing the fake database". Does this approach align with any claim you are testing the "real database"? </p> <p> When testing the data-layer, I have historically written (heavier) tests that integrate with a real database to exercise a system's data-layer (as you describe with `SqlReservationsRepository`). I find myself reaching for dynamic mocks in the context of exercising an application's domain layer -- where the data-layer is a dependency providing indirect input/output. Does this use of mocks violate encapsulation in the way this article describes? I _think_ not, because in that case a dynamic mock is used to represent states that are valid "according to the contract", but I'm hoping you could shed a bit more light on the topic. Am I putting the pieces together correctly? </p> <p> Rephrasing the question using your Reservations example code, I would typically inject `IReservationsRepository` into `MaitreD` (which you opt not to do) and outline the posssible database return values (or commands) using dynamic mocks in a test suite of `MaitreD`. What drawbacks, if any, would that approach lead to with respect to encapsulation and test fragility? </p> </div> <div class="comment-date">2022-11-02 20:11 UTC</div> </div> <div class="comment" id="98ef29aec08e408382205b49bea697b6"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Matthew, thank you for writing. I apologise if the article is unclear about this, but nowhere in the <em>real</em> code base do I have a test of <code>FakeDatabase</code>. I only wrote the tests that exercise the Test Doubles to illustrate the point I was trying to make. These tests only exist for the benefit of this article. </p> <p> The first <code>CreateAndReadRoundTrip</code> test in the article shows a real integration test. The System Under Test (SUT) is the <code>SqlReservationsRepository</code> class, which is part of the production code - not a Test Double. </p> <p> That class implements the <code>IReservationsRepository</code> interface. The point I was trying to make is that the <code>CreateAndReadRoundTrip</code> test already exercises a particular subset of the contract of the interface. Thus, if one replaces one implementation of the interface with another implementation, according to the <a href="https://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov Substitution Principle</a> (LSP) the test should still pass. </p> <p> This is true for <code>FakeDatabase</code>. While the behaviour is different (it doesn't persist data), it still fulfils the contract. Dynamic mocks, on the other hand, don't automatically follow the LSP. Unless one is careful and explicit, dynamic mocks tend to weaken postconditions. For example, a dynamic mock doesn't automatically return the added reservation when you call <code>ReadReservation</code>. </p> <p> This is an essential flaw of dynamic mock objects that is independent of where you use them. My article <a href="#09410dcfb758412d991a47f08be885b5">already describes</a> how a fairly innocuous change in the production code will cause a dynamic mock to break the test. </p> <p> I no longer inject dependencies into domain models, since doing so <a href="/2017/01/27/from-dependency-injection-to-dependency-rejection">makes the domain model impure</a>. Even if I did, however, I'd still have the same problem with dynamic mocks breaking encapsulation. </p> </div> <div class="comment-date">2022-11-04 7:06 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Refactoring a saga from the State pattern to the State monad https://blog.ploeh.dk/2022/10/10/refactoring-a-saga-from-the-state-pattern-to-the-state-monad 2022-10-10T06:27:00+00:00 Mark Seemann <div id="post"> <p> <em>A slightly less unrealistic example in C#.</em> </p> <p> This article is one of the examples that I promised in the earlier article <a href="/2022/09/05/the-state-pattern-and-the-state-monad">The State pattern and the State monad</a>. That article examines the relationship between the <a href="https://en.wikipedia.org/wiki/State_pattern">State design pattern</a> and the <a href="/2022/06/20/the-state-monad">State monad</a>. It's deliberately abstract, so one or more examples are in order. </p> <p> In the <a href="/2022/09/26/refactoring-the-tcp-state-pattern-example-to-pure-functions">previous example</a> you saw how to refactor <a href="/ref/dp">Design Patterns</a>' <em>TCP connection</em> example. That example is, unfortunately, hardly illuminating due to its nature, so a second example is warranted. </p> <p> This second example shows how to refactor a stateful asynchronous message handler from the State pattern to the State monad. </p> <h3 id="20b0274c10b14a84a07ebd2086ab1fa0"> Shipping policy <a href="#20b0274c10b14a84a07ebd2086ab1fa0" title="permalink">#</a> </h3> <p> Instead of inventing an example from scratch, I decided to use <a href="https://docs.particular.net/tutorials/nservicebus-sagas/1-saga-basics/">an NServiceBus saga tutorial</a> as a foundation. Read on even if you don't know <a href="https://particular.net/nservicebus">NServiceBus</a>. You don't have to know anything about NServiceBus in order to follow along. I just thought that I'd embed the example code in a context that actually executes and does something, instead of faking it with a bunch of unit tests. Hopefully this will help make the example a bit more realistic and relatable. </p> <p> The example is a simple demo of asynchronous message handling. In a web store shipping department, you should only ship an item once you've received the order and a billing confirmation. When working with asynchronous messaging, you can't, however, rely on message ordering, so perhaps the <code>OrderBilled</code> message arrives before the <code>OrderPlaced</code> message, and sometimes it's the other way around. </p> <p> <img src="/content/binary/shipping-policy-state-diagram.png" alt="Shipping policy state diagram."> </p> <p> Only when you've received both messages may you ship the item. </p> <p> It's a simple workflow, and you don't <em>really</em> need the State pattern. So much is clear from the sample code implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ShippingPolicy</span>&nbsp;:&nbsp;Saga&lt;ShippingPolicyData&gt;, &nbsp;&nbsp;&nbsp;&nbsp;IAmStartedByMessages&lt;OrderBilled&gt;, &nbsp;&nbsp;&nbsp;&nbsp;IAmStartedByMessages&lt;OrderPlaced&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">static</span>&nbsp;ILog&nbsp;log&nbsp;=&nbsp;LogManager.GetLogger&lt;ShippingPolicy&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">protected</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ConfigureHowToFindSaga(SagaPropertyMapper&lt;ShippingPolicyData&gt;&nbsp;mapper) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mapper.MapSaga(sagaData&nbsp;=&gt;&nbsp;sagaData.OrderId) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToMessage&lt;OrderPlaced&gt;(message&nbsp;=&gt;&nbsp;message.OrderId) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToMessage&lt;OrderBilled&gt;(message&nbsp;=&gt;&nbsp;message.OrderId); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;Handle(OrderPlaced&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderPlaced&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ProcessOrder(context); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;Handle(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderBilled&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ProcessOrder(context); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;ProcessOrder(IMessageHandlerContext&nbsp;context) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;context.SendLocal(<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;Data.OrderId&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MarkAsComplete(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I don't expect you to be familiar with the NServiceBus API, so don't worry about the base class, the interfaces, or the <code>ConfigureHowToFindSaga</code> method. What you need to know is that this class handles two types of messages: <code>OrderPlaced</code> and <code>OrderBilled</code>. What the base class and the framework does is handling message correlation, hydration and dehydration, and so on. </p> <p> For the purposes of this demo, all you need to know about the <code>context</code> object is that it enables you to send and publish messages. The code sample uses <code>context.SendLocal</code> to send a new <code>ShipOrder</code> Command. </p> <p> Messages arrive asynchronously and conceptually with long wait times between them. You can't just rely on in-memory object state because a <code>ShippingPolicy</code> instance may receive one message and then risk that the server it's running on shuts down before the next message arrives. The NServiceBus framework handles message correlation and hydration and dehydration of state data. The latter is modelled by the <code>ShippingPolicyData</code> class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ShippingPolicyData</span>&nbsp;:&nbsp;ContainSagaData { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;OrderId&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsOrderPlaced&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsOrderBilled&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> Notice that the above sample code inspects and manipulates the <code>Data</code> property defined by the <code>Saga&lt;ShippingPolicyData&gt;</code> base class. </p> <p> When the <code>ShippingPolicy</code> methods are called by the NServiceBus framework, the <code>Data</code> is automatically populated. When you modify the <code>Data</code>, the state data is automatically persisted when the message handler shuts down to wait for the next message. </p> <h3 id="0edda65734e64a6a99a865c43d6de5be"> Characterisation tests <a href="#0edda65734e64a6a99a865c43d6de5be" title="permalink">#</a> </h3> <p> While you can draw an explicit state diagram like the one above, the sample code doesn't explicitly model the various states as objects. Instead, it relies on reading and writing two Boolean values. </p> <p> There's nothing wrong with this implementation. It's the simplest thing that could possibly work, so why make it more complicated? </p> <p> In this article, I <em>am</em> going to make it more complicated. First, I'm going to refactor the above sample code to use the State design pattern, and then I'm going to refactor that code to use the State monad. From a perspective of maintainability, this isn't warranted, but on the other hand, I hope it's educational. The sample code is just complex enough to showcase the structures of the State pattern and the State monad, yet simple enough that the implementation logic doesn't get in the way. </p> <p> Simplicity can be deceiving, however, and no refactoring is without risk. </p> <blockquote> <p> "to refactor, the essential precondition is [...] solid tests" </p> <footer><cite><a href="https://martinfowler.com/">Martin Fowler</a>, <a href="/ref/refactoring">Refactoring</a></cite></footer> </blockquote> <p> I found it safest to first add a few <a href="https://en.wikipedia.org/wiki/Characterization_test">Characterisation Tests</a> to make sure I didn't introduce any errors as I changed the code. It did catch a few copy-paste goofs that I made, so adding tests turned out to be a good idea. </p> <p> Testing NServiceBus message handlers isn't too hard. All the tests I wrote look similar, so one should be enough to give you an idea. </p> <p> <pre>[Theory] [InlineData(<span style="color:#a31515;">&quot;1337&quot;</span>)] [InlineData(<span style="color:#a31515;">&quot;baz&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;OrderPlacedAndBilled(<span style="color:blue;">string</span>&nbsp;orderId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingPolicy&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingPolicyData&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;orderId&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ctx&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TestableMessageHandlerContext(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Handle(<span style="color:blue;">new</span>&nbsp;OrderPlaced&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;orderId&nbsp;},&nbsp;ctx); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Handle(<span style="color:blue;">new</span>&nbsp;OrderBilled&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;orderId&nbsp;},&nbsp;ctx); &nbsp;&nbsp;&nbsp;&nbsp;Assert.True(sut.Completed); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;msg&nbsp;=&nbsp;Assert.Single(ctx.SentMessages.Containing&lt;ShipOrder&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(orderId,&nbsp;msg.Message.OrderId); }</pre> </p> <p> The tests use <a href="https://xunit.net/">xUnit.net</a> 2.4.2. When I downloaded the <a href="https://docs.particular.net/tutorials/nservicebus-sagas/1-saga-basics/">NServiceBus saga sample code</a> it targeted .NET Framework 4.8, and I didn't bother to change the version. </p> <p> While the NServiceBus framework will automatically hydrate and populate <code>Data</code>, in a unit test you have to remember to explicitly populate it. The <code>TestableMessageHandlerContext</code> class is a <a href="http://xunitpatterns.com/Test%20Spy.html">Test Spy</a> that is part of <a href="https://docs.particular.net/nservicebus/testing/">NServiceBus testing API</a>. </p> <p> You'd think I was paid by <a href="https://particular.net/">Particular Software</a> to write this article, but I'm not. All this is really just the introduction. You're excused if you've forgotten the topic of this article, but my goal is to show a State pattern example. Only now can we begin in earnest. </p> <h3 id="7b1878c85e134ac5a24e9ce13ba064d2"> State pattern implementation <a href="#7b1878c85e134ac5a24e9ce13ba064d2" title="permalink">#</a> </h3> <p> Refactoring to the State pattern, I chose to let the <code>ShippingPolicy</code> class fill the role of the pattern's <code>Context</code>. Instead of a base class with virtual method, I used an interface to define the <code>State</code> object, as that's more <a href="/2015/08/03/idiomatic-or-idiosyncratic">Idiomatic</a> in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;OrderPlaced(OrderPlaced&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context,&nbsp;ShippingPolicy&nbsp;policy); &nbsp;&nbsp;&nbsp;&nbsp;Task&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context,&nbsp;ShippingPolicy&nbsp;policy); }</pre> </p> <p> The State pattern only shows examples where the <code>State</code> methods take a single argument: The <code>Context</code>. In this case, that's the <code>ShippingPolicy</code>. Careful! There's also a parameter called <code>context</code>! That's the NServiceBus context, and is an artefact of the original example. The two other parameters, <code>message</code> and <code>context</code>, are run-time values passed on from the <code>ShippingPolicy</code>'s <code>Handle</code> methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;IShippingState&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;} <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderPlaced&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderPlaced&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;State.OrderPlaced(message,&nbsp;context,&nbsp;<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderBilled&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;State.OrderBilled(message,&nbsp;context,&nbsp;<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(); }</pre> </p> <p> The <code>Hydrate</code> method isn't part of the State pattern, but finds an appropriate state based on <code>Data</code>: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Hydrate() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;!Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;InitialShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;(Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;!Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;AwaitingBillingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;(!Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;AwaitingPlacementState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;CompletedShippingState.Instance; }</pre> </p> <p> In more recent versions of C# you'd be able to use more succinct pattern matching, but since this code base is on .NET Framework 4.8 I'm constrained to C# 7.3 and this is as good as I cared to make it. It's not important to the topic of the State pattern, but I'm showing it in case you where wondering. It's typical that you need to translate between data that exists in the 'external world' and your object-oriented, polymorphic code, since <a href="/2011/05/31/AttheBoundaries,ApplicationsareNotObject-Oriented">at the boundaries, applications aren't object-oriented</a>. </p> <p> Likewise, the <code>Dehydrate</code> method translates the other way: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Dehydrate() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(State&nbsp;<span style="color:blue;">is</span>&nbsp;AwaitingBillingState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(State&nbsp;<span style="color:blue;">is</span>&nbsp;AwaitingPlacementState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(State&nbsp;<span style="color:blue;">is</span>&nbsp;CompletedShippingState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">false</span>; }</pre> </p> <p> In any case, <code>Hydrate</code> and <code>Dehydrate</code> are distractions. The important part is that the <code>ShippingPolicy</code> (the State <em>Context</em>) now delegates execution to its <code>State</code>, which performs the actual work and updates the <code>State</code>. </p> <h3 id="0ad9f36d329d45cda95fc92630e65eed"> Initial state <a href="#0ad9f36d329d45cda95fc92630e65eed" title="permalink">#</a> </h3> <p> The first time the saga runs, both <code>Data.IsOrderPlaced</code> and <code>Data.IsOrderBilled</code> are <code>false</code>, which means that the <code>State</code> is <code>InitialShippingState</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">InitialShippingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;InitialShippingState&nbsp;Instance&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;InitialShippingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">InitialShippingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;OrderPlaced( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderPlaced&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;AwaitingBillingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;OrderBilled( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderBilled&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;AwaitingPlacementState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As the above state transition diagram indicates, the only thing that each of the methods do is that they transition to the next appropriate state: <code>AwaitingBillingState</code> if the first event was <code>OrderPlaced</code>, and <code>AwaitingPlacementState</code> when the event was <code>OrderBilled</code>. </p> <blockquote> <p> "State object are often Singletons" </p> <footer><cite><a href="/ref/dp">Design Patterns</a></cite></footer> </blockquote> <p> Like in the <a href="/2022/09/26/refactoring-the-tcp-state-pattern-example-to-pure-functions">previous example</a> I've made all the State objects <a href="https://en.wikipedia.org/wiki/Singleton_pattern">Singletons</a>. It's not that important, but since they are all stateless, we might as well. At least, it's in the spirit of the book. </p> <h3 id="1a3c6882c82d444fb4dff0cb9ffee2e5"> Awaiting billing <a href="#1a3c6882c82d444fb4dff0cb9ffee2e5" title="permalink">#</a> </h3> <p> <code>AwaitingBillingState</code> is another <code>IShippingState</code> implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IShippingState&nbsp;Instance&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;AwaitingBillingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;OrderPlaced( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderPlaced&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;OrderBilled( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderBilled&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;context.SendLocal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;policy.Data.OrderId&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This State doesn't react to <code>OrderPlaced</code> because it assumes that an order has already been placed. It only reacts to an <code>OrderBilled</code> event. When that happens, all requirements have been fulfilled to ship the item, so it sends a <code>ShipOrder</code> Command, marks the saga as completed, and changes the <code>State</code> to <code>CompletedShippingState</code>. </p> <p> The <code>Complete</code> method is a little wrapper method I had to add to the <code>ShippingPolicy</code> class, since <code>MarkAsComplete</code> is a <code>protected</code> method: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Complete() { &nbsp;&nbsp;&nbsp;&nbsp;MarkAsComplete(); }</pre> </p> <p> The <code>AwaitingPlacementState</code> class is similar to <code>AwaitingBillingState</code>, except that it reacts to <code>OrderPlaced</code> rather than <code>OrderBilled</code>. </p> <h3 id="503262f1d69a4042bb3f93b16aa04d44"> Terminal state <a href="#503262f1d69a4042bb3f93b16aa04d44" title="permalink">#</a> </h3> <p> The fourth and final state is the <code>CompletedShippingState</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">CompletedShippingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IShippingState&nbsp;Instance&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;CompletedShippingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">CompletedShippingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;OrderPlaced( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderPlaced&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Task&nbsp;OrderBilled( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderBilled&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Task.CompletedTask; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> In this state, the saga is completed, so it ignores both events. </p> <h3 id="3769c2c7504548c884cdc38d788a8f46"> Move Commands to output <a href="#3769c2c7504548c884cdc38d788a8f46" title="permalink">#</a> </h3> <p> The saga now uses the State pattern to manage state-specific behaviour as well as state transitions. To be clear, this complexity isn't warranted for the simple requirements. This is, after all, an example. All tests still pass, and smoke testing also indicates that everything still works as it's supposed to. </p> <p> The goal of this article is now to refactor the State pattern implementation to <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. When the saga runs it has an observable side effect: It eventually sends a <code>ShipOrder</code> Command. During processing it also updates its internal state. Both of these are sources of impurity that we have to <a href="/2016/09/26/decoupling-decisions-from-effects">decouple from the decision logic</a>. </p> <p> I'll do this in several steps. The first impure action I'll address is the externally observable message transmission. A common functional-programming trick is to turn a side effect into a return value. So far, the <code>IShippingState</code> methods don't return anything. (This is strictly not true; they each return <a href="https://docs.microsoft.com/dotnet/api/system.threading.tasks.task">Task</a>, but we can regard <code>Task</code> as 'asynchronous <code>void</code>'.) Thus, return values are still available as a communications channel. </p> <p> Refactor the <code>IShippingState</code> methods to return Commands instead of actually sending them. Each method may send an arbitrary number of Commands, including none, so the return type has to be a collection: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderPlaced( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderPlaced&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy); &nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderBilled( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderBilled&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy); }</pre> </p> <p> When you change the interface you also have to change all the implementing classes, including <code>AwaitingBillingState</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IShippingState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;AwaitingBillingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderPlaced( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderPlaced&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Array.Empty&lt;ICommand&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderBilled( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OrderBilled&nbsp;message, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;policy.Data.OrderId&nbsp;}&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> In order to do nothing a method like <code>OrderPlaced</code> now has to return an empty collection of Commands. In order to 'send' a Command, <code>OrderBilled</code> now returns it instead of using the <code>context</code> to send it. The <code>context</code> is already redundant, but since I prefer to <a href="https://stackoverflow.blog/2022/04/06/use-git-tactically/">move in small steps</a>, I'll remove it in a separate step. </p> <p> It's now the responsibility of the <code>ShippingPolicy</code> class to do something with the Commands returned by the <code>State</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderBilled&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;result&nbsp;=&nbsp;State.OrderBilled(message,&nbsp;context,&nbsp;<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Interpret(result,&nbsp;context); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Interpret( &nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;commands, &nbsp;&nbsp;&nbsp;&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">foreach</span>&nbsp;(var&nbsp;cmd&nbsp;<span style="color:blue;">in</span>&nbsp;commands) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;context.SendLocal(cmd); }</pre> </p> <p> In functional programming, you often run an interpreter over the instructions returned by a pure function. Here the interpreter is just a private helper method. </p> <p> The <code>IShippingState</code> methods are no longer asynchronous. Now they just return collections. I consider that a simplification. </p> <h3 id="31ef1f01413143359d91e067da9d9a35"> Remove context parameter <a href="#31ef1f01413143359d91e067da9d9a35" title="permalink">#</a> </h3> <p> The <code>context</code> parameter is now redundant, so remove it from the <code>IShippingState</code> interface: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderPlaced(OrderPlaced&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy); &nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy); }</pre> </p> <p> I used Visual Studio's built-in refactoring tools to remove the parameter, which automatically removed it from all the call sites and implementations. </p> <p> This takes us part of the way towards implementing the states as pure functions, but there's still work to be done. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy) { &nbsp;&nbsp;&nbsp;&nbsp;policy.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;policy.Data.OrderId&nbsp;}&nbsp;}; }</pre> </p> <p> The above <code>OrderBilled</code> implementation calls <code>policy.Complete</code> to indicate that the saga has completed. That's another state mutation that must be eliminated to make this a pure function. </p> <h3 id="51ae5d89697b47fe8611b58b4540163e"> Return complex result <a href="#51ae5d89697b47fe8611b58b4540163e" title="permalink">#</a> </h3> <p> How do you refactor from state mutation to pure function? You turn the mutation statement into an instruction, which is a value that you return. In this case you might want to return a Boolean value: True to complete the saga. False otherwise. </p> <p> There seems to be a problem, though. The <code>IShippingState</code> methods already return data: They return a collection of Commands. How do we get around this conundrum? </p> <p> Introduce a complex object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ShippingStateResult</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ShippingStateResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;commands, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">bool</span>&nbsp;completeSaga) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Commands&nbsp;=&nbsp;commands; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CompleteSaga&nbsp;=&nbsp;completeSaga; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;ICommand&gt;&nbsp;Commands&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;CompleteSaga&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>&nbsp;obj) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;ShippingStateResult&nbsp;result&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;IReadOnlyCollection&lt;ICommand&gt;&gt;.Default &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Equals(Commands,&nbsp;result.Commands)&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CompleteSaga&nbsp;==&nbsp;result.CompleteSaga; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;GetHashCode() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hashCode&nbsp;=&nbsp;-1668187231; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;hashCode&nbsp;=&nbsp;hashCode&nbsp;*&nbsp;-1521134295&nbsp;+&nbsp;EqualityComparer&lt;IReadOnlyCollection&lt;ICommand&gt;&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Default.GetHashCode(Commands); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;hashCode&nbsp;=&nbsp;hashCode&nbsp;*&nbsp;-1521134295&nbsp;+&nbsp;CompleteSaga.GetHashCode(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;hashCode; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> That looks rather horrible, but most of the code is generated by Visual Studio. The only thing I wrote myself was the class declaration and the two read-only properties. I then used Visual Studio's <em>Generate constructor</em> and <em>Generate Equals and GetHashCode</em> Quick Actions to produce the rest of the code. </p> <p> With more modern versions of C# I could have used a <a href="https://docs.microsoft.com/dotnet/csharp/language-reference/builtin-types/record">record</a>, but as I've already mentioned, I'm on C# 7.3 here. </p> <p> The <code>IShippingState</code> interface can now define its methods with this new return type: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;ShippingStateResult&nbsp;OrderPlaced(OrderPlaced&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy); &nbsp;&nbsp;&nbsp;&nbsp;ShippingStateResult&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy); }</pre> </p> <p> This change reminds me of the <a href="https://refactoring.com/catalog/introduceParameterObject.html">Introduce Parameter Object</a> refactoring, but instead applied to the return value instead of input. </p> <p> Implementers now have to return values of this new type: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IShippingState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;AwaitingBillingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;ShippingStateResult&nbsp;OrderPlaced(OrderPlaced&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult(Array.Empty&lt;ICommand&gt;(),&nbsp;<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;ShippingStateResult&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;policy.Data.OrderId&nbsp;}&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Moving a statement to an output value implies that the effect must happen somewhere else. It seems natural to put it in the <code>ShippingPolicy</code> class' <code>Interpret</code> method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderBilled&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;result&nbsp;=&nbsp;State.OrderBilled(message,&nbsp;<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Interpret(result,&nbsp;context); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Interpret(ShippingStateResult&nbsp;result,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">foreach</span>&nbsp;(var&nbsp;cmd&nbsp;<span style="color:blue;">in</span>&nbsp;result.Commands) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;context.SendLocal(cmd); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(result.CompleteSaga) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MarkAsComplete(); }</pre> </p> <p> Since <code>Interpret</code> is an instance method on the <code>ShippingPolicy</code> class I can now also delete the internal <code>Complete</code> method, since <code>MarkAsComplete</code> is already callable (it's a <code>protected</code> method defined by the <code>Saga</code> base class). </p> <h3 id="ce2f9545c94b4c6ab9536050ed030b8b"> Use message data <a href="#ce2f9545c94b4c6ab9536050ed030b8b" title="permalink">#</a> </h3> <p> Have you noticed an odd thing about the code so far? It doesn't use any of the <code>message</code> data! </p> <p> This is an artefact of the original code example. Refer back to the original <code>ProcessOrder</code> helper method. It uses neither <code>OrderPlaced</code> nor <code>OrderBilled</code> for anything. Instead, it pulls the <code>OrderId</code> from the saga's <code>Data</code> property. It can do that because NServiceBus makes sure that all <code>OrderId</code> values are correlated. It'll only instantiate a saga for which <code>Data.OrderId</code> matches <code>OrderPlaced.OrderId</code> or <code>OrderBilled.OrderId</code>. Thus, these values are guaranteed to be the same, and that's why <code>ProcessOrder</code> can get away with using <code>Data.OrderId</code> instead of the <code>message</code> data. </p> <p> So far, through all refactorings, I've retained this detail, but it seems odd. It also couples the implementation methods to the <code>ShippingPolicy</code> class rather than the message classes. For these reasons, refactor the methods to use the message data instead. Here's the <code>AwaitingBillingState</code> implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;ShippingStateResult&nbsp;OrderBilled(OrderBilled&nbsp;message,&nbsp;ShippingPolicy&nbsp;policy) { &nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;message.OrderId&nbsp;}&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>); }</pre> </p> <p> Compare this version with the previous iteration, where it used <code>policy.Data.OrderId</code> instead of <code>message.OrderId</code>. </p> <p> Now, the only reason to pass <code>ShippingPolicy</code> as a method parameter is to mutate <code>policy.State</code>. We'll get to that in due time, but first, there's another issue I'd like to address. </p> <h3 id="800453851c1944e3835a05f8ff2e3272"> Immutable arguments <a href="#800453851c1944e3835a05f8ff2e3272" title="permalink">#</a> </h3> <p> Keep in mind that the overall goal of the exercise is to refactor the state machine to pure functions. For good measure, method parameters should be immutable as well. Consider a method like <code>OrderBilled</code> shown above in its most recent iteration. It mutates <code>policy</code> by setting <code>policy.State</code>. The long-term goal is to get rid of that statement. </p> <p> The method doesn't mutate the other argument, <code>message</code>, but the <code>OrderBilled</code> class is actually mutable: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">OrderBilled</span>&nbsp;:&nbsp;IEvent { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;OrderId&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> The same is true for the other message type, <code>OrderPlaced</code>. </p> <p> For good measure, pure functions shouldn't take mutable arguments. You could argue that, since none of the implementation methods actually mutate the messages, it doesn't really matter. I am, however, enough of a neat freak that I don't like to leave such a loose strand dangling. I'd like to refactor the <code>IShippingState</code> API so that only immutable message data is passed as arguments. </p> <p> In a situation like this, there are (at least) three options: </p> <ul> <li> Make the message types immutable. This would mean making <code>OrderBilled</code> and <code>OrderPlaced</code> immutable. These message types are by default mutable <a href="https://en.wikipedia.org/wiki/Data_transfer_object">Data Transfer Objects</a> (DTO), because NServiceBus needs to serialise and deserialise them to transmit them over durable queues. There are ways you can configure NServiceBus to use serialisation mechanisms that enable immutable records as messages, but for an example code base like this, I might be inclined to reach for an easier solution if one presents itself. </li> <li> Add an immutable 'mirror' class. This may often be a good idea if you have a rich domain model that you'd like to represent. You can see an example of that in <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>, where there's both a mutable <code>ReservationDto</code> class and an immutable <code>Reservation</code> <a href="https://www.martinfowler.com/bliki/ValueObject.html">Value Object</a>. This makes sense if the invariants of the domain model are sufficiently stronger than the DTO. That hardly seems to be the case here, since both messages only contain an <code>OrderId</code>. </li> <li> Dissolve the DTO into its constituents and pass each as an argument. This doesn't work if the DTO is complex and nested, but here there's only a single constituent element, and that's the <code>OrderId</code> property. </li> </ul> <p> The third option seems like the simplest solution, so refactor the <code>IShippingState</code> methods to take an <code>orderId</code> parameter instead of a message: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;ShippingStateResult&nbsp;OrderPlaced(<span style="color:blue;">string</span>&nbsp;orderId,&nbsp;ShippingPolicy&nbsp;policy); &nbsp;&nbsp;&nbsp;&nbsp;ShippingStateResult&nbsp;OrderBilled(<span style="color:blue;">string</span>&nbsp;orderId,&nbsp;ShippingPolicy&nbsp;policy); }</pre> </p> <p> While this is the easiest of the three options given above, the refactoring doesn't hinge on this. It would work just as well with one of the two other options. </p> <p> Implementations now look like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;ShippingStateResult&nbsp;OrderBilled(<span style="color:blue;">string</span>&nbsp;orderId,&nbsp;ShippingPolicy&nbsp;policy) { &nbsp;&nbsp;&nbsp;&nbsp;policy.State&nbsp;=&nbsp;CompletedShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;orderId&nbsp;}&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>); }</pre> </p> <p> The only impure action still lingering is the mutation of <code>policy.State</code>. Once we're rid of that, the API consists of pure functions. </p> <h3 id="1034a4283bb44e639805778d5ae4504a"> Return state <a href="#1034a4283bb44e639805778d5ae4504a" title="permalink">#</a> </h3> <p> As outlined by the <a href="/2022/09/05/the-state-pattern-and-the-state-monad">parent article</a>, instead of mutating the caller's state, you can return the state as part of a tuple. This means that you no longer need to pass <code>ShippingPolicy </code> as a parameter: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IShippingState</span> { &nbsp;&nbsp;&nbsp;&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&nbsp;OrderPlaced(<span style="color:blue;">string</span>&nbsp;orderId); &nbsp;&nbsp;&nbsp;&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&nbsp;OrderBilled(<span style="color:blue;">string</span>&nbsp;orderId); }</pre> </p> <p> Why not expand the <code>ShippingStateResult</code> class, or conversely, dissolve that class and instead return a triple (a three-tuple)? All of these are possible as alternatives, as they'd be isomorphic to this particular design. The reason I've chosen this particular return type is that it's the idiomatic implementation of the State monad: The result is the first element of a tuple, and the state is the second element. This means that you can use a standard, reusable State monad library to manipulate the values, as you'll see later. </p> <p> An implementation now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>&nbsp;:&nbsp;IShippingState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IShippingState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;AwaitingBillingState(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">AwaitingBillingState</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&nbsp;OrderPlaced(<span style="color:blue;">string</span>&nbsp;orderId) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Tuple.Create( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult(Array.Empty&lt;ICommand&gt;(),&nbsp;<span style="color:blue;">false</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(IShippingState)<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&nbsp;OrderBilled(<span style="color:blue;">string</span>&nbsp;orderId) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Tuple.Create( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;ShipOrder()&nbsp;{&nbsp;OrderId&nbsp;=&nbsp;orderId&nbsp;}&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">true</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CompletedShippingState.Instance); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Since the <code>ShippingPolicy</code> class that calls these methods now directly receives the state as part of the output, it no longer needs a mutable <code>State</code> property. Instead, it immediately handles the return value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderPlaced&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderPlaced&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;state&nbsp;=&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;result&nbsp;=&nbsp;state.OrderPlaced(message.OrderId); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Interpret(result.Item1,&nbsp;context); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(result.Item2); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;Handle(OrderBilled&nbsp;message,&nbsp;IMessageHandlerContext&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;log.Info(<span style="color:#a31515;">$&quot;OrderBilled&nbsp;message&nbsp;received.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;state&nbsp;=&nbsp;Hydrate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;result&nbsp;=&nbsp;state.OrderBilled(message.OrderId); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Interpret(result.Item1,&nbsp;context); &nbsp;&nbsp;&nbsp;&nbsp;Dehydrate(result.Item2); }</pre> </p> <p> Each <code>Handle</code> method is now an <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>. </p> <p> Since the <code>result</code> is now a tuple, the <code>Handle</code> methods now have to pass the first element (<code>result.Item1</code>) to the <code>Interpret</code> helper method, and the second element (<code>result.Item2</code>) - the state - to <code>Dehydrate</code>. It's also possible to pattern match (or <em>destructure</em>) each of the elements directly; you'll see an example of that later. </p> <p> Since the mutable <code>State</code> property is now gone, the <code>Hydrate</code> method returns the hydrated state: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;IShippingState&nbsp;Hydrate() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;!Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;InitialShippingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;(Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;!Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;AwaitingBillingState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span>&nbsp;<span style="color:blue;">if</span>&nbsp;(!Data.IsOrderPlaced&nbsp;&amp;&amp;&nbsp;Data.IsOrderBilled) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;AwaitingPlacementState.Instance; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;CompletedShippingState.Instance; }</pre> </p> <p> Likewise, the <code>Dehydrate</code> method takes the new state as an input parameter: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Dehydrate(IShippingState&nbsp;state) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(state&nbsp;<span style="color:blue;">is</span>&nbsp;AwaitingBillingState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(state&nbsp;<span style="color:blue;">is</span>&nbsp;AwaitingPlacementState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(state&nbsp;<span style="color:blue;">is</span>&nbsp;CompletedShippingState) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">true</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderPlaced&nbsp;=&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;Data.IsOrderBilled&nbsp;=&nbsp;<span style="color:blue;">false</span>; }</pre> </p> <p> Since each <code>Handle</code> method only calls a single State-valued method, they don't need the State monad machinery. This only becomes useful when you need to compose multiple State-based operations. </p> <p> This might be useful in unit tests, so let's examine that next. </p> <h3 id="8559bfcefc6e49459fa16fce5977cc58"> State monad <a href="#8559bfcefc6e49459fa16fce5977cc58" title="permalink">#</a> </h3> <p> In <a href="/2022/06/20/the-state-monad">previous articles about the State monad</a> you've seen it implemented based on an <code>IState</code> interface. I've also dropped hints here and there that you don't <em>need</em> the interface. Instead, you can implement the monad functions directly on State-valued functions. That's what I'm going to do here: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;S,&nbsp;Tuple&lt;T1,&nbsp;S&gt;&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">S</span>,&nbsp;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">T1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Func&lt;S,&nbsp;Tuple&lt;T,&nbsp;S&gt;&gt;&nbsp;source, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;Func&lt;S,&nbsp;Tuple&lt;T1,&nbsp;S&gt;&gt;&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;s&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;tuple&nbsp;=&nbsp;source(s); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;f&nbsp;=&nbsp;selector(tuple.Item1); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;f(tuple.Item2); &nbsp;&nbsp;&nbsp;&nbsp;}; }</pre> </p> <p> This <code>SelectMany</code> implementation works directly on another function, <code>source</code>. This function takes a state of type <code>S</code> as input and returns a tuple as a result. The first element is the result of type <code>T</code>, and the second element is the new state, still of type <code>S</code>. Compare that to <a href="/2021/07/19/the-state-functor">the IState interface</a> to convince yourself that these are just two representations of the same idea. </p> <p> The return value is a new function with the same shape, but where the result type is <code>T1</code> rather than <code>T</code>. </p> <p> You can implement the special <code>SelectMany</code> overload that enables query syntax in <a href="/2022/03/28/monads">the standard way</a>. </p> <p> The <em>return</em> function also mirrors the previous interface-based implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;S,&nbsp;Tuple&lt;T,&nbsp;S&gt;&gt;&nbsp;Return&lt;<span style="color:#2b91af;">S</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt;(T&nbsp;x) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;s&nbsp;=&gt;&nbsp;Tuple.Create(x,&nbsp;s); }</pre> </p> <p> You can also implement <a href="/2022/07/04/get-and-put-state">the standard Get, Put, and Modify functions</a>, but we are not going to need them here. Try it as an exercise. </p> <h3 id="4fe0da2976e34c8eaf8490186f7d022a"> State-valued event handlers <a href="#4fe0da2976e34c8eaf8490186f7d022a" title="permalink">#</a> </h3> <p> The <code>IShippingState</code> methods almost look like State values, but the arguments are in the wrong order. A State value is a function that takes state as input and returns a tuple. The methods on <code>IShippingState</code>, however, take <code>orderId</code> as input and return a tuple. The state is also present, but as the instance that exposes the methods. We have to flip the arguments: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;IShippingState,&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&gt;&nbsp;Billed( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">string</span>&nbsp;orderId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;s&nbsp;=&gt;&nbsp;s.OrderBilled(orderId); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Func&lt;IShippingState,&nbsp;Tuple&lt;ShippingStateResult,&nbsp;IShippingState&gt;&gt;&nbsp;Placed( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">string</span>&nbsp;orderId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;s&nbsp;=&gt;&nbsp;s.OrderPlaced(orderId); }</pre> </p> <p> This is a typical example of how you have to turn things on their heads in functional programming, compared to object-oriented programming. These two methods convert <code>OrderBilled</code> and <code>OrderPlaced</code> to State monad values. </p> <h3 id="2aa26c7863bc47f6a37ad50f54d97d24"> Testing state results <a href="#2aa26c7863bc47f6a37ad50f54d97d24" title="permalink">#</a> </h3> <p> A unit test demonstrates how this enables you to compose multiple stateful operations using query syntax: </p> <p> <pre>[Theory] [InlineData(<span style="color:#a31515;">&quot;90125&quot;</span>)] [InlineData(<span style="color:#a31515;">&quot;quux&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;StateResultExample(<span style="color:blue;">string</span>&nbsp;orderId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sf&nbsp;=&nbsp;<span style="color:blue;">from</span>&nbsp;x&nbsp;<span style="color:blue;">in</span>&nbsp;orderId.Placed() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;y&nbsp;<span style="color:blue;">in</span>&nbsp;orderId.Billed() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;x,&nbsp;y&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;var&nbsp;(results,&nbsp;finalState)&nbsp;=&nbsp;sf(InitialShippingState.Instance); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">false</span>,&nbsp;<span style="color:blue;">true</span>&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;results.Select(r&nbsp;=&gt;&nbsp;r.CompleteSaga)); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Single( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;results &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.SelectMany(r&nbsp;=&gt;&nbsp;r.Commands) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.OfType&lt;ShipOrder&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Select(msg&nbsp;=&gt;&nbsp;msg.OrderId), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;orderId); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(CompletedShippingState.Instance,&nbsp;finalState); }</pre> </p> <p> Keep in mind that a State monad value is a function. That's the reason I called the composition <code>sf</code> - for <em>State Function</em>. When you execute it with <code>InitialShippingState</code> as input it returns a tuple that the test immediately pattern matches (destructures) into its constituent elements. </p> <p> The test then asserts that the <code>results</code> and <code>finalState</code> are as expected. The assertions against <code>results</code> are a bit awkward, since C# collections don't have structural equality. These assertions would have been simpler in <a href="https://fsharp.org/">F#</a> or <a href="https://www.haskell.org/">Haskell</a>. </p> <h3 id="c899bd7d1b8f4bcea29a5ad946b0f3b8"> Testing with an interpreter <a href="#c899bd7d1b8f4bcea29a5ad946b0f3b8" title="permalink">#</a> </h3> <p> While <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">the Arrange and Act phases of the above test</a> are simple, the Assertion phase seems awkward. Another testing strategy is to run a test-specific interpreter over the instructions returned as the State computation result: </p> <p> <pre>[Theory] [InlineData(<span style="color:#a31515;">&quot;1984&quot;</span>)] [InlineData(<span style="color:#a31515;">&quot;quuz&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;StateInterpretationExample(<span style="color:blue;">string</span>&nbsp;orderId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sf&nbsp;=&nbsp;<span style="color:blue;">from</span>&nbsp;x&nbsp;<span style="color:blue;">in</span>&nbsp;orderId.Placed() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;y&nbsp;<span style="color:blue;">in</span>&nbsp;orderId.Billed() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;x,&nbsp;y&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;var&nbsp;(results,&nbsp;finalState)&nbsp;=&nbsp;sf(InitialShippingState.Instance); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(CompletedShippingState.Instance,&nbsp;finalState); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;result&nbsp;=&nbsp;Interpret(results); &nbsp;&nbsp;&nbsp;&nbsp;Assert.True(result.CompleteSaga); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Single( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;result.Commands.OfType&lt;ShipOrder&gt;().Select(msg&nbsp;=&gt;&nbsp;msg.OrderId), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;orderId); }</pre> </p> <p> It helps a little, but the assertions still have to work around the lack of structural equality of <code>result.Commands</code>. </p> <h3 id="80405400d4ed4cfd9c89bf539df21b3b"> Monoid <a href="#80405400d4ed4cfd9c89bf539df21b3b" title="permalink">#</a> </h3> <p> The test-specific <code>Interpret</code> helper method is interesting in its own right, though: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;ShippingStateResult&nbsp;Interpret(IEnumerable&lt;ShippingStateResult&gt;&nbsp;results) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;identity&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult(Array.Empty&lt;ICommand&gt;(),&nbsp;<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;ShippingStateResult&nbsp;Combine(ShippingStateResult&nbsp;x,&nbsp;ShippingStateResult&nbsp;y) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ShippingStateResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x.Commands.Concat(y.Commands).ToArray(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x.CompleteSaga&nbsp;||&nbsp;y.CompleteSaga); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;results.Aggregate(identity,&nbsp;Combine); }</pre> </p> <p> It wasn't until I started implementing this helper method that I realised that <code>ShippingStateResult</code> gives rise to a <a href="/2017/10/06/monoids">monoid</a>! Since <a href="/2017/11/20/monoids-accumulate">monoids accumulate</a>, you can start with the <code>identity</code> and use the binary operation (here called <code>Combine</code>) to <code>Aggregate</code> an arbitrary number of <code>ShippingStateResult</code> values into one. </p> <p> The <code>ShippingStateResult</code> class is composed of two constituent values (a collection and a Boolean value), and since both of these give rise to one or more monoids, a <a href="/2017/10/30/tuple-monoids">tuple of those monoids itself gives rise to one or more monoids</a>. The <code>ShippingStateResult</code> is isomorphic to a tuple, so this result carries over. </p> <p> Should you move the <code>Combine</code> method and the <code>identity</code> value to the <code>ShippingStateResult</code> class itself. After all, putting them in a test-specific helper method smells a bit of <a href="https://wiki.c2.com/?FeatureEnvySmell">Feature Envy</a>. </p> <p> This seems compelling, but it's not clear that arbitrary client code might need this particular monoid. After all, there are four monoids over Boolean values, and at least two over collections. That's eight possible combinations. Which one should <code>ShippingStateResult</code> expose as members? </p> <p> The monoid used in <code>Interpret</code> combines the normal <a href="/2017/10/10/strings-lists-and-sequences-as-a-monoid">collection monoid</a> with the <em>any</em> monoid. That seems appropriate in this case, but other clients might rather need the <em>all</em> monoid. </p> <p> Without more usage examples, I decided to leave the code as an <code>Interpret</code> implementation detail for now. </p> <p> In any case, I find it worth noting that by decoupling the state logic from the NServiceBus framework, it's possible to test it <a href="/2019/02/11/asynchronous-injection">without running asynchronous workflows</a>. </p> <h3 id="a1bc6585723044a9b795aa23099d9d36"> Conclusion <a href="#a1bc6585723044a9b795aa23099d9d36" title="permalink">#</a> </h3> <p> In this article you saw how to implement an asynchronous messaging saga in three different ways. First, as a simple ad-hoc solution, second using the State pattern, and third implemented with the State monad. Both the State pattern and State monad implementations are meant exclusively to showcase these two techniques. The first solution using two Boolean flags is by far the simplest solution, and the one I'd use in a production system. </p> <p> The point is that you can use the State monad if you need to write stateful computations. This may include finite state machines, as otherwise addressed by the State design pattern, but could also include other algorithms where you need to keep track of state. </p> <p> <strong>Next:</strong> <a href="/2021/11/29/postels-law-as-a-profunctor">Postel's law as a profunctor</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Some thoughts on the economics of programming https://blog.ploeh.dk/2022/10/03/some-thoughts-on-the-economics-of-programming 2022-10-03T05:53:00+00:00 Mark Seemann <div id="post"> <p> <em>On the net value of process and code quality.</em> </p> <p> Once upon a time there was a software company that had a special way of doing things. No other company had ever done things quite like that before, but the company had much success. In short time it rose to dominance in the market, outcompeting all serious competition. Some people disliked the company because of its business tactics and sheer size, but others admired it. </p> <p> Even more wanted to be like it. </p> <p> How did the company achieve its indisputable success? It looked as though it was really, really good at making software. How did they write such good software? </p> <p> It turned out that the company had a special software development process. </p> <p> Other software organisations, hoping to be able to be as successful, tried to copy the special process. The company was willing to share. Its employees wrote about the process. They gave conference presentations on their special sauce. </p> <p> Which company do I have in mind, and what was the trick that made it so much better than its competition? Was it <a href="https://en.wikipedia.org/wiki/Microservices">microservices</a>? <a href="https://en.wikipedia.org/wiki/Monorepo">Monorepos</a>? <a href="https://en.wikipedia.org/wiki/Kubernetes">Kubernetes</a>? <a href="https://en.wikipedia.org/wiki/DevOps">DevOps</a>? <a href="https://en.wikipedia.org/wiki/Serverless_computing">Serverless</a>? </p> <p> No, the company was <a href="https://microsoft.com">Microsoft</a> and the development process was called <a href="https://en.wikipedia.org/wiki/Microsoft_Solutions_Framework">Microsoft Solutions Framework</a> (MSF). </p> <p> <em>What?!</em> do you say. </p> <p> You've never heard of MSF? </p> <p> That's hardly surprising. I doubt that MSF was in any way related to Microsoft's success. </p> <h3 id="9333d3993dbb4125a7d40faf5221ea02"> Net profits <a href="#9333d3993dbb4125a7d40faf5221ea02" title="permalink">#</a> </h3> <p> These days, many people in technology consider Microsoft an embarrassing dinosaur. While you know that it's still around, does it really <em>matter</em>, these days? </p> <p> You can't deny, however, that Microsoft made a lot of money in the Nineties. They still do. </p> <p> What's the key to making a lot of money? Have a revenue larger than your costs. </p> <p> I'm too lazy to look up the actual numbers, but clearly Microsoft had (and still has) a revenue vastly larger than its costs: </p> <p> <img src="/content/binary/great-net-value-chart.png" alt="Revenue and cost line chart. The revenue is visibly and significantly greater than the cost over the entire time line."> </p> <p> Compared to real, historic numbers, this may be exaggerated, but I'm trying to make a general point - not one that hinges on actual profit numbers of Microsoft, <a href="https://apple.com">Apple</a>, <a href="https://amzn.to/3QLHkly">Amazon</a>, <a href="http://google.com">Google</a>, or any other tremendously profitable company. I'm also aware that real companies have costs that aren't directly related to software development: Marketing, operations, buildings, sales, etcetera. They also make money in other ways than from their software, mainly from investments of the profits. </p> <p> The difference between the revenue and the cost is the profit or net value. </p> <p> If the graph looks like the above, is <em>managing cost</em> the main cause of success? Hardly. The cost is almost a rounding error on the profits. </p> <p> If so, is the technology or process key to such a company's success? Was it MSF that made Microsoft the wealthiest company in the world? Are two-pizza teams the only explanation of Amazon's success? Is Google the dominant search engine because the source code is organised in a monorepo? </p> <p> I'd be surprised were that the case. Rather, I think that these companies were at the right place at the right time. While there were search engines before Google, Google was so much better that users quickly migrated. Google was also better at making money than earlier search engines like <a href="https://en.wikipedia.org/wiki/AltaVista">AltaVista</a> or <a href="https://en.wikipedia.org/wiki/Yahoo!">Yahoo!</a> Likewise, Microsoft made more successful PC operating systems than the competition (which in the early Windows era consisted exclusively of <a href="https://en.wikipedia.org/wiki/OS/2">OS/2</a>) and better professional software (word processor, spreadsheet, etcetera). Amazon made a large-scale international web shop before anyone else. Apple made affordable computers with graphical user interfaces before other companies. Later, they introduced a smartphone at the right time. </p> <p> All of this is simplified. For example, it's not really true that Apple made the first smartphone. When the iPhone was introduced, I already carried a <a href="https://en.wikipedia.org/wiki/Pocket_PC">Pocket PC Phone Edition</a> device that could browse the internet, had email, phone, SMS, and so on. There were other precursors even earlier. </p> <p> I'm not trying to explain away <em>excellence of execution</em>. These companies succeeded for a variety of reasons, including that they were good at what they were doing. Lots of companies, however, are good at what they are doing, and still they fail. Being at the right place at the right time matters. Once in a while, a company finds itself in such favourable circumstances that success is served on a silver platter. While good execution is important, it doesn't explain the magnitude of the success. </p> <p> Bad execution is likely to eliminate you in the long run, but it doesn't follow logically that good execution guarantees success. </p> <p> Perhaps the successful companies succeeded because of circumstances, and <em>despite</em> mediocre execution. As usual, you should be wary not to mistake correlation for causation. </p> <h3 id="d1ab697df716465ab7eb8c841d3a1615"> Legacy code <a href="#d1ab697df716465ab7eb8c841d3a1615" title="permalink">#</a> </h3> <p> You should be sceptical of adopting processes or technology just because a <a href="https://en.wikipedia.org/wiki/Big_Tech">Big Tech</a> company uses it. Still, if that was all I had in mind, I could probably had said that shorter. I have another point to make. </p> <p> I often encounter resistance to ideas about better software development on the grounds that the status quo is good enough. Put bluntly, </p> <blockquote> <p>""legacy," [...] is condescending-engineer-speak for "actually makes money.""</p> <footer><cite><a href="https://www.lastweekinaws.com/blog/right-sizing-your-instances-is-nonsense">Corey Quinn</a></cite></footer> </blockquote> <p> To be clear, I have nothing against the author or the cited article, which discusses something (right-sizing VMs) that I know nothing about. The phrase, or variations thereof, however, is such a fit <a href="https://en.wikipedia.org/wiki/Meme">meme</a> that it spreads. It strongly indicates that people who discuss code quality are wankers, while 'real programmers' produce code that makes money. I consider that a <a href="https://en.wikipedia.org/wiki/False_dilemma">false dichotomy</a>. </p> <p> Most software organisations aren't in the fortunate situation that revenues are orders of magnitude greater than costs. Most software organisations can make a decent profit if they find a market and execute on a good idea. Perhaps the revenue starts at 'only' double the cost. </p> <p> <img src="/content/binary/decreasing-profit-margin-from-increased-cost.png" alt="Revenue and cost line chart. The revenue starts at about double that of the cost. The cost line, however, grows by a steeper rater and eventually overtakes the revenue."> </p> <p> If you can consistently make the double of your costs, you'll be in business for a long time. As the above line chart indicates, however, is that if the costs rise faster than the revenue, you'll eventually hit a point when you start losing money. </p> <p> The Big Tech companies aren't likely to run into that situation because their profit margins are so great, but normal companies are very much at risk. </p> <p> The area between the revenue and the cost represents the profit. Thus, looking back, it may be true that a software system has been making money. This doesn't mean, however, that it will keep making money. </p> <p> In the above chart, the cost eventually exceeds the revenue. If this cost is mainly driven by rising software development costs, then the company is in deep trouble. </p> <p> I've worked with such a company. When I started with it, it was a thriving company with many employees, most of them developers or IT professionals. In the previous decade, it had turned a nice profit every year. </p> <p> This all started to change around the time that I arrived. (I will, again, remind the reader that correlation does not imply causation.) One reason I was engaged was that the developers were stuck. Due to external market pressures they had to deliver a tremendous amount of new features, and they were stuck in <a href="https://en.wikipedia.org/wiki/Analysis_paralysis">analysis paralysis</a>. </p> <p> I helped them get unstuck, but as we started working on the new features, we discovered the size of the mess of the legacy code base. </p> <p> I recall a conversation I later had with the CEO. He told me, after having discussed the situation with several key people: <em>"I knew that we had a legacy code base... but I didn't know it was </em>this<em> bad!"</em> </p> <p> Revenue remained constant, but costs kept rising. Today, the company is no longer around. </p> <p> This was a 100% digital service company. All revenue was ultimately based on software. The business idea was good, but the company couldn't keep up with competitors. As far as I can tell, it was undone by its legacy code base. </p> <h3 id="33ded378d771431fa00f37c8b3e85bfc"> Conclusion <a href="#33ded378d771431fa00f37c8b3e85bfc" title="permalink">#</a> </h3> <p> Software should provide some kind of value. Usually profits, but sometimes savings, and occasionally wider concerns are in scope. It's reasonable and professional to consider value as you produce software. You should, however, be aware of a too myopic focus on immediate and past value. </p> <p> Finding safety in past value is indulging in complacency. Legacy software can make money from day one, but that doesn't mean that it'll keep making money. The main problem with legacy code is that costs keep rising. When non-technical business stakeholders start to notice this, it may be too late. </p> <p> The is one of many reasons I believe that we, software developers, have a <em>responsibility</em> to combat the mess. I don't think there's anything condescending about that attitude. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Refactoring the TCP State pattern example to pure functions https://blog.ploeh.dk/2022/09/26/refactoring-the-tcp-state-pattern-example-to-pure-functions 2022-09-26T05:50:00+00:00 Mark Seemann <div id="post"> <p> <em>A C# example.</em> </p> <p> This article is one of the examples that I promised in the earlier article <a href="/2022/09/05/the-state-pattern-and-the-state-monad">The State pattern and the State monad</a>. That article examines the relationship between the <a href="https://en.wikipedia.org/wiki/State_pattern">State design pattern</a> and the <a href="/2022/06/20/the-state-monad">State monad</a>. That article is deliberately abstract, so one or more examples are in order. </p> <p> In this article, I show you how to start with the example from <a href="/ref/dp">Design Patterns</a> and refactor it to an immutable solution using <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. </p> <p> The code shown here is <a href="https://github.com/ploeh/TCPStateCSharp">available on GitHub</a>. </p> <h3 id="f70ca28a361241e5a352399f8cf912d7"> TCP connection <a href="#f70ca28a361241e5a352399f8cf912d7" title="permalink">#</a> </h3> <p> The example is a class that handles <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">TCP</a> connections. The book's example is in C++, while I'll show my C# interpretation. </p> <p> A TCP connection can be in one of several states, so the <code>TcpConnection</code> class keeps an instance of the polymorphic <code>TcpState</code>, which implements the state and transitions between them. </p> <p> <code>TcpConnection</code> plays the role of the State pattern's <code>Context</code>, and <code>TcpState</code> of the <code>State</code>. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpConnection</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TcpState&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TcpConnection</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;TcpClosed.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ActiveOpen() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State.ActiveOpen(<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PassiveOpen() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State.PassiveOpen(<span style="color:blue;">this</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;that&nbsp;delegate&nbsp;to&nbsp;State&nbsp;follows...</span></pre> </p> <p> The <code>TcpConnection</code> class' methods delegate to a corresponding method on <code>TcpState</code>, passing itself an argument. This gives the <code>TcpState</code> implementation an opportunity to change the <code>TcpConnection</code>'s <code>State</code> property, which has an <code>internal</code> setter. </p> <h3 id="ffea200499f94145a0686e2481aebf2c"> State <a href="#ffea200499f94145a0686e2481aebf2c" title="permalink">#</a> </h3> <p> This is the <code>TcpState</code> class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpState</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Transmit(TcpConnection&nbsp;connection,&nbsp;TcpOctetStream&nbsp;stream) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ActiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PassiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Close(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Synchronize(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Acknowledge(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Send(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I don't consider this entirely <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C# code, but it seems closer to the book's C++ example. (It's been a couple of decades since I wrote C++, so I could be mistaken.) It doesn't matter in practice, but instead of a concrete class with <a href="https://en.wikipedia.org/wiki/NOP_(code)">no-op</a> <code>virtual</code> methods, I would usually define an interface. I'll do that in the next example article. </p> <p> The methods have the same names as the methods on <code>TcpConnection</code>, but the signatures are different. All the <code>TcpState</code> methods take a <code>TcpConnection</code> parameter, whereas the <code>TcpConnection</code> methods take no arguments. </p> <p> While the <code>TcpState</code> methods don't do anything, various classes can inherit from the class and override some or all of them. </p> <h3 id="d68a32e30a314c9ea7f45004482a7d98"> Connection closed <a href="#d68a32e30a314c9ea7f45004482a7d98" title="permalink">#</a> </h3> <p> The book shows implementations of three classes that inherit from <code>TcpState</code>, starting with <code>TcpClosed</code>. Here's my translation to C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpClosed</span>&nbsp;:&nbsp;TcpState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TcpState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TcpClosed(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">TcpClosed</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ActiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Send&nbsp;SYN,&nbsp;receive&nbsp;SYN,&nbsp;Ack,&nbsp;etc.</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;connection.State&nbsp;=&nbsp;TcpEstablished.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;PassiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;connection.State&nbsp;=&nbsp;TcpListen.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This implementation overrides <code>ActiveOpen</code> and <code>PassiveOpen</code>. In both cases, after performing some work, they change <code>connection.State</code>. </p> <blockquote> <p> "<code>TCPState</code> subclasses maintain no local state, so they can be shared, and only one instance of each is required. The unique instance of <code>TCPState</code> subclass is obtained by the static <code>Instance</code> operation. [...] </p> <p> "This make each <code>TCPState</code> subclass a Singleton [...]." </p> <footer><cite><a href="/ref/dp">Design Patterns</a></cite></footer> </blockquote> <p> I've maintained that property of each subclass in my C# code, even though it has no impact on the structure of the State pattern. </p> <h3 id="8e21a702cffe4ef09455a29c2981ab39"> The other subclasses <a href="#8e21a702cffe4ef09455a29c2981ab39" title="permalink">#</a> </h3> <p> The next subclass, <code>TcpEstablished</code>, is cast in the same mould: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpEstablished</span>&nbsp;:&nbsp;TcpState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TcpState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TcpEstablished(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">TcpEstablished</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Close(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;send&nbsp;FIN,&nbsp;receive&nbsp;ACK&nbsp;of&nbsp;FIN</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;connection.State&nbsp;=&nbsp;TcpListen.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Transmit( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TcpConnection&nbsp;connection, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TcpOctetStream&nbsp;stream) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;connection.ProcessOctet(stream); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As is <code>TcpListen</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpListen</span>&nbsp;:&nbsp;TcpState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TcpState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TcpListen(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">TcpListen</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Send(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Send&nbsp;SYN,&nbsp;receive&nbsp;SYN,&nbsp;ACK,&nbsp;etc.</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;connection.State&nbsp;=&nbsp;TcpEstablished.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I admit that I find these examples a bit anaemic, since there's really no logic going on. None of the overrides change state <em>conditionally</em>, which would be possible and make the examples a little more interesting. If you're interested in an example where this happens, see my article <a href="/2021/05/24/tennis-kata-using-the-state-pattern">Tennis kata using the State pattern</a>. </p> <h3 id="68a957ca3dcd40e9986b078223f0763e"> Refactor to pure functions <a href="#68a957ca3dcd40e9986b078223f0763e" title="permalink">#</a> </h3> <p> There's only one obvious source of impurity in the example: The literal <code>State</code> mutation of <code>TcpConnection</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;TcpState&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> While client code can't <code>set</code> the <code>State</code> property, subclasses can, and they do. After all, it's how the State pattern works. </p> <p> It's quite a stretch to claim that if we can only get rid of that property setter then all else will be pure. After all, who knows what all those comments actually imply: </p> <p> <pre><span style="color:green;">//&nbsp;Send&nbsp;SYN,&nbsp;receive&nbsp;SYN,&nbsp;ACK,&nbsp;etc.</span></pre> </p> <p> To be honest, we must imagine that <a href="https://en.wikipedia.org/wiki/Input/output">I/O</a> takes place here. This means that even though it's possible to refactor away from mutating the <code>State</code> property, these implementations are not really going to be pure functions. </p> <p> I could try to imagine what that <code>SYN</code> and <code>ACK</code> would look like, but it would be unfounded and hypothetical. I'm not going to do that here. Instead, that's the reason I'm going to publish a second article with a more realistic and complex example. When it comes to the present example, I'm going to proceed with the unreasonable assumption that the comments hide no nondeterministic behaviour or side effects. </p> <p> As outlined in the <a href="/2022/09/05/the-state-pattern-and-the-state-monad">article that compares the State pattern and the State monad</a>, you can refactor state mutation to a pure function by instead returning the new state. Usually, you'd have to return a tuple, because you'd also need to return the 'original' return value. Here, however, the 'return type' of all methods is <code>void</code>, so this isn't necessary. </p> <p> <code>void</code> is <a href="/2018/01/15/unit-isomorphisms">isomorphic to unit</a>, so strictly speaking you could refactor to a return type like <code>Tuple&lt;Unit, TcpConnection&gt;</code>, but that is isomorphic to <code>TcpConnection</code>. (If you need to understand why that is, try writing two functions: One that converts a <code>Tuple&lt;Unit, TcpConnection&gt;</code> to a <code>TcpConnection</code>, and another that converts a <code>TcpConnection</code> to a <code>Tuple&lt;Unit, TcpConnection&gt;</code>.) </p> <p> There's no reason to make things more complicated than they have to be, so I'm going to use the simplest representation: <code>TcpConnection</code>. Thus, you can get rid of the <code>State</code> mutation by instead returning a new <code>TcpConnection</code> from all methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpConnection</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TcpState&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TcpConnection</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;TcpClosed.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">TcpConnection</span>(TcpState&nbsp;state) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;state; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TcpConnection&nbsp;ActiveOpen() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;TcpConnection(State.ActiveOpen(<span style="color:blue;">this</span>)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TcpConnection&nbsp;PassiveOpen() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;TcpConnection(State.PassiveOpen(<span style="color:blue;">this</span>)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;that&nbsp;delegate&nbsp;to&nbsp;State&nbsp;follows...</span></pre> </p> <p> The <code>State</code> property no longer has a setter; there's only a public getter. In order to 'change' the state, code must return a new <code>TcpConnection</code> object with the new state. To facilitate that, you'll need to add a constructor overload that takes the new state as an input. Here I made it <code>private</code>, but making it more accessible is not prohibited. </p> <p> This implies, however, that the <code>TcpState</code> methods <em>also</em> return values instead of mutating state. The base class now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpState</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;TcpState&nbsp;Transmit(TcpConnection&nbsp;connection,&nbsp;TcpOctetStream&nbsp;stream) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;TcpState&nbsp;ActiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;TcpState&nbsp;PassiveOpen(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">this</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;And&nbsp;so&nbsp;on...</span></pre> </p> <p> Again, all the methods previously 'returned' <code>void</code>, so while, according to the State monad, you should strictly speaking return <code>Tuple&lt;Unit, TcpState&gt;</code>, this simplifies to <code>TcpState</code>. </p> <p> Individual subclasses now do their work and return other <code>TcpState</code> implementations. I'm not going to tire you with all the example subclasses, so here's just <code>TcpEstablished</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">TcpEstablished</span>&nbsp;:&nbsp;TcpState { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TcpState&nbsp;Instance&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TcpEstablished(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">TcpEstablished</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;TcpState&nbsp;Close(TcpConnection&nbsp;connection) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;send&nbsp;FIN,&nbsp;receive&nbsp;ACK&nbsp;of&nbsp;FIN</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;TcpListen.Instance; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;TcpState&nbsp;Transmit( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TcpConnection&nbsp;connection, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TcpOctetStream&nbsp;stream) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TcpConnection&nbsp;newConnection&nbsp;=&nbsp;connection.ProcessOctet(stream); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;newConnection.State; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The trickiest implementation is <code>Transmit</code>, since <code>ProcessOctet</code> returns a <code>TcpConnection</code> while the <code>Transmit</code> method has to return a <code>TcpState</code>. Fortunately, the <code>Transmit</code> method can achieve that goal by returning <code>newConnection.State</code>. It feels a bit roundabout, but highlights a point I made in the <a href="/2022/09/05/the-state-pattern-and-the-state-monad">previous article</a>: The <code>TcpConnection</code> and <code>TcpState</code> classes are isomorphic - or, they would be if we made the <code>TcpConnection</code> constructor overload public. Thus, the <code>TcpConnection</code> class is redundant and might be deleted. </p> <h3 id="a94a0e101f6b49d3b4caea30060ae1e1"> Conclusion <a href="#a94a0e101f6b49d3b4caea30060ae1e1" title="permalink">#</a> </h3> <p> This article shows how to refactor the <em>TCP connection</em> sample code from <a href="/ref/dp">Design Patterns</a> to pure functions. </p> <p> If it feels as though something's missing there's a good reason for that. The example, as given, is degenerate because all methods 'return' <code>void</code>, and we don't really know what the actual implementation code (all that <em>Send SYN, receive SYN, ACK, etc.</em>) looks like. This means that we actually don't have to make use of the State monad, because we can get away with <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphisms</a>. All methods on <code>TcpConnection</code> are really functions that take <code>TcpConnection</code> as input (the instance itself) and return <code>TcpConnection</code>. If you want to see a more realistic example showcasing that perspective, see my article <a href="/2021/05/31/from-state-tennis-to-endomorphism">From State tennis to endomorphism</a>. </p> <p> Even though the example is degenerate, I wanted to show it because otherwise you might wonder how the book's example code fares when exposed to the State monad. To be clear, because of the nature of the example, the State monad never becomes necessary. Thus, we need a second example. </p> <p> <strong>Next:</strong> <a href="/2022/10/10/refactoring-a-saga-from-the-state-pattern-to-the-state-monad">Refactoring a saga from the State pattern to the State monad</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. When to refactor https://blog.ploeh.dk/2022/09/19/when-to-refactor 2022-09-19T06:36:00+00:00 Mark Seemann <div id="post"> <p> <em>FAQ: How do I convince my manager to let me refactor?</em> </p> <p> This question frequently comes up. Developers want to refactor, but are under the impression that managers or other stakeholders will not let them. </p> <p> Sometimes people ask me how to convince their managers to get permission to refactor. I can't answer that. <a href="/2021/03/22/the-dispassionate-developer">I don't know how to convince other people</a>. That's not my métier. </p> <p> I also believe that professional programmers <a href="/2019/03/18/the-programmer-as-decision-maker">should make their own decisions</a>. You don't ask permission to add three lines to a file, or create a new class. Why do you feel that you have to ask permission to refactor? </p> <h3 id="4af4356cd706457ebf8968e92850b140"> Does refactoring take time? <a href="#4af4356cd706457ebf8968e92850b140" title="permalink">#</a> </h3> <p> In <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a> I tell the following story: </p> <blockquote> <p> "I once led an effort to <a href="/ref/ddd">refactor towards deeper insight</a>. My colleague and I had identified that the key to implementing a new feature would require changing a fundamental class in our code base. </p> <p> "While such an insight rarely arrives at an opportune time, we wanted to make the change, and our manager allowed it. </p> <p> "A week later, our code still didn’t compile. </p> <p> "I’d hoped that I could make the change to the class in question and then <a href="/ref/welc">lean on the compiler</a> to identify the call sites that needed modification. The problem was that there was an abundance of compilation errors, and fixing them wasn’t a simple question of search-and-replace. </p> <p> "My manager finally took me aside to let me know that he wasn’t satisfied with the situation. I could only concur. </p> <p> "After a mild dressing down, he allowed me to continue the work, and a few more days of heroic effort saw the work completed. </p> <p> "That’s a failure I don’t intend to repeat." </p> <footer><cite><a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a></cite></footer> </blockquote> <p> There's a couple of points to this story. Yes, I <em>did</em> ask for permission before refactoring. I expected the process to take time, and I felt that making such a choice of prioritisation should involve my manager. While this manager trusted me, I felt a moral obligation to be transparent about the work I was doing. I didn't consider it professional to take a week out of the calendar and work on one thing while the rest of the organisation was expecting me to be working on something else. </p> <p> So I can understand why developers feel that they have to ask permission to refactor. After all, refactoring takes time... Doesn't it? </p> <h3 id="ddb8e36130fe475da5ea914ccf06ae45"> Small steps <a href="#ddb8e36130fe475da5ea914ccf06ae45" title="permalink">#</a> </h3> <p> This may unearth the underlying assumption that prevents developers from refactoring: The notion that refactoring takes time. </p> <p> As I wrote in <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>, that was a failure I didn't intend to repeat. I've never again asked permission to refactor, because I've never since allowed myself to be in a situation where refactoring would take significant time. </p> <p> The reason I tell the story in the book is that I use it to motivate using the <a href="https://martinfowler.com/bliki/StranglerFigApplication.html">Strangler pattern</a> at the code level. The book proceeds to show an example of that. </p> <p> Migrating code to a new API by allowing the old and the new to coexist for a while is only one of many techniques for taking smaller steps. Another is the use of <a href="https://en.wikipedia.org/wiki/Feature_toggle">feature flags</a>, a technique that I also show in the book. <a href="https://martinfowler.com/">Martin Fowler</a>'s <a href="/ref/refactoring">Refactoring</a> is literally an entire book about how to improve code bases in small, controlled steps. </p> <p> Follow the <a href="/2019/10/21/a-red-green-refactor-checklist">red-green-refactor checklist</a> and commit after each <em>green</em> and <em>refactor</em> step. Move in small steps and <a href="https://stackoverflow.blog/2022/04/06/use-git-tactically/">use Git tactically</a>. </p> <p> I'm beginning to realise, though, that <em>moving in small steps</em> is a skill that must be explicitly learned. This may seem obvious once posited, but it may also be helpful to explicitly state it. </p> <p> Whenever I've had a chance to talk to other software professionals and <a href="https://twitter.com/hillelogram/status/1445435617047990273">thought leaders</a>, they agree. As far as I can tell, universities and coding boot camps don't teach this skill, and if (like me) you're autodidact, you probably haven't learned it either. After all, few people insist that this is an important skill. It may, however, be one of the most important programming skills you can learn. </p> <h3 id="11afa6718a3c4a6f8e50aa5066474ac8"> Make it work, then make it right <a href="#11afa6718a3c4a6f8e50aa5066474ac8" title="permalink">#</a> </h3> <p> When should you refactor? As <a href="https://wiki.c2.com/?BoyScoutRule">the boy scout rule</a> suggests: All the time. </p> <p> You can, specifically, do it after implementing a new feature. As <a href="https://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast">Kent Beck perhaps said or wrote</a>: <em>Make it work, then make it right</em>. </p> <p> How long does it take to make it right? </p> <p> Perhaps you think that it takes as much time as it does to make it work. </p> <p> <img src="/content/binary/make-it-work-then-right-50-50.png" alt="A timeline with two sections: 'make it work' and 'make it right'. Each section has the same size."> </p> <p> Perhaps you think that making it right takes even more time. </p> <p> <img src="/content/binary/make-it-work-then-right-20-80.png" alt="A timeline with two sections: 'make it work' and 'make it right'. The 'make it right' section is substantially larger than the 'make it work' section."> </p> <p> If this is how much time making the code right takes, I can understand why you feel that you need to ask your manager. That's what I did, those many years ago. But what if the proportions are more like this? </p> <p> <img src="/content/binary/make-it-work-then-right-80-20.png" alt="A timeline with two sections: 'make it work' and 'make it right'. The 'make it right' section is substantially smaller than the 'make it work' section."> </p> <p> Do you still feel that you need to ask for permission to refactor? </p> <p> Writing code so that the team can keep a sustainable pace is your job. It's not something you should have to ask for permission to do. </p> <blockquote> <p> "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." </p> <footer><cite>Martin Fowler, <a href="/ref/refactoring">Refactoring</a></cite></footer> </blockquote> <p> Making the code right is not always a huge endeavour. It can be, if you've already made a mess of it, but if it's in good condition, keeping it that way doesn't have to take much extra effort. It's part of the ongoing design process that programming is. </p> <p> How do you know what <em>right</em> is? Doesn't this make-it-work-make-it-right mentality lead to <a href="https://wiki.c2.com/?SpeculativeGenerality">speculative generality</a>? </p> <p> No-one expects you to be able to predict the future, so don't try. Making it right means making the code good in the current context. Use good names, remove duplication, get rid of code smells, keep methods small and complexity low. <a href="/2020/04/13/curb-code-rot-with-thresholds">Refactor if you exceed a threshold</a>. </p> <h3 id="1a2eb1f23d70440c84e518babf7dc373"> Make code easy to change <a href="#1a2eb1f23d70440c84e518babf7dc373" title="permalink">#</a> </h3> <p> The purpose of keeping code in a good condition is to make future changes as easy as possible. If you can't predict the future, however, then how do you know how to factor the code? </p> <p> Another <a href="https://en.wikipedia.org/wiki/Kent_Beck">Kent Beck</a> aphorism suggests a tactic: </p> <blockquote> <p> "for each desired change, make the change easy (warning: this may be hard), then make the easy change" </p> <footer><cite><a href="https://twitter.com/KentBeck/status/250733358307500032">Kent Beck</a></cite></footer> </blockquote> <p> In other words, when you know what you need to accomplish, first refactor the code so that it becomes easier to achieve the goal, and only then write the code to do that. </p> <p> <img src="/content/binary/refactor-implement-40-60.png" alt="A timeline with two sections: Refactor and Implement. The Implement section is visibly larger than the Refactor section."> </p> <p> Should you ask permission to refactor in such a case? Only if you sincerely believe that you can complete the entire task significantly faster without first improving the code. How likely is that? If the code base is already a mess, how easy is it to make changes? Not easy, and granted: That will also be true for refactoring. The difference between first refactoring and <em>not</em> refactoring, however, is that if you refactor, you leave the code in a better state. If you don't, you leave it in a worse state. </p> <p> These decisions compound. </p> <p> But what if, as Kent Beck implies, refactoring is hard? Then the situation might look like this: </p> <p> <img src="/content/binary/refactor-implement-80-20.png" alt="A timeline with two sections: Refactor and Implement. The Refactor section is significantly larger than the Implement section."> </p> <p> Should you ask for permission to refactor? I don't think so. While refactoring in this diagram is most of the work, it makes the change easy. Thus, once you're done refactoring, you make the easy change. The total amount of time this takes may turn out to be quicker than if you hadn't refactored (compare this figure to the previous figure: they're to scale). You also leave the code base in a better state so that future changes may be easier. </p> <h3 id="ff4ce77459af4e64a9e3df797fc01d8d"> Conclusion <a href="#ff4ce77459af4e64a9e3df797fc01d8d" title="permalink">#</a> </h3> <p> There are lots of opportunities for refactoring. Every time you see something that could be improved, why not improve it? The fact that you're already looking at a piece of code suggests that it's somehow relevant to your current task. If it takes ten, fifteen minutes to improve it, why not do it? What if it takes an hour? </p> <p> Most people think nothing of spending hours in meetings without asking their managers. If this is true, you can also decide to use a couple of hours improving code. They're likely as well spent as the meeting hours. </p> <p> The key, however, is to be able to perform opportunistic refactoring. You can't do that if you can only move in day-long iterations; if hours, or days, go by when you can't compile, or when most tests fail. </p> <p> On the other hand, if you're able to incrementally improve the code base in one-minute, or fifteen-minute, steps, then you can improve the code base every time an occasion arrives. </p> <p> This is a skill that you need to learn. You're not born with the ability to improve in small steps. You'll have to practice - for example by <a href="/2020/01/13/on-doing-katas">doing katas</a>. One customer of mine told me that they found Kent Beck's <a href="https://medium.com/@kentbeck_7670/test-commit-revert-870bbd756864">TCR</a> a great way to teach that skill. </p> <p> You can refactor in small steps. It's part of software engineering. Usually, you don't need to ask for permission. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="4dc52487bcee4459857d6fb6741dbcad"> <div class="comment-author"><a href="https://danielt1263.medium.com">Daniel Tartaglia</a></div> <div class="comment-content"> <p> I've always had a problem with the notion of "red, green, refactor" and "first get it working, then make it right." I think the order is completely wrong. </p> <p> As an explanation, I refer you to the first chapter of the first edition of Martin Fowler's Refactoring book. In that chapter is an example of a working system and we are presented with a change request. </p> <p> In the example, the first thing that Fowler points out and does is the refactoring. And one of the highlighted ideas in the chapter says: <blockquote>When you find you have to add a feature to a program, and the program's code is not structured in a convenient way to add the feature, first refactor the program to make it easy to add the feature, then add the feature.</blockquote> </p> <p> In other words, the refactoring <i>comes first</i>. You refactor as part of adding the feature, not as a separate thing that is done after you have working code. It may not trip off the tongue as nicely, but the saying should be "refactor, red, green." </p> <p> Once you have working code, you are done, and when you are estimating the time it will take to add the feature, <i>you include the refactoring time</i>. Lastly, you never refactor "just because," you refactor in order to make a new feature easy to add. </p> <p> This mode of working makes much more sense to me. I feel that refactoring with no clear goal in mind ("improve the design" is not a clear goal) just leads to an over-designed/unnecessarily complex system. What do you think of this idea? </p> </div> <div class="comment-date">2022-09-24 02:28 UTC</div> </div> <div class="comment" id="f9df3a46ce5f4cb8a386304f64030137"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Daniel, thank you for writing. You make some good points. </p> <p> The red-green-refactor cycle is useful as a feedback cycle for new development work. It's not the only way to work. Particularly, as you point out, when you have existing code, first refactoring and then adding new code is a useful order. </p> <p> Typically, though, when you're adding a new feature, you can rarely implement a new feature <em>only</em> by refactoring existing code. Normally you also need to add some new code. I still find the red-green-refactor cycle useful for that kind of work. I don't view it as an <em>either-or</em> proposition, but rather as a <em>both-this-and-that</em> way of working. </p> <blockquote> <p> "you never refactor "just because," you refactor in order to make a new feature easy to add." </p> </blockquote> <p> Never say never. I don't agree with that point of view. There are more than one reason for refactoring, and making room for a new feature is certainly one of them. This does not, however, rule out other reasons. I can easily think of a handful of other reasons that I consider warranted, but I don't want to derail the discussion by listing all of them. The list is not going to be complete anyway. I'll just outline one: </p> <p> Sometimes, you read existing code because you need to understand what's going on. If the code is badly structured, it can take significant time and effort to reach such understanding. If, at that point you can see a simpler way to achieve the same behaviour, why not refactor the code? In that way, you make it easier for future readers of the code to understand what's going on. If you've already spent (wasted) significant time understanding something, why let other readers suffer and waste time if you can simplify the code? </p> <p> This is essentially the boy scout rule, but as I claimed, there are other reasons to refactor as well. </p> <p> Finally, thank you for the quote from <em>Refactoring</em>. I've usually been using this Kent Beck quote: </p> <blockquote> <p> "for each desired change, make the change easy (warning: this may be hard), then make the easy change" </p> <footer><cite><a href="https://twitter.com/kentbeck/status/250733358307500032">Kent Beck</a></cite></footer> </blockquote> <p> but that's from 2012, and <em>Refactoring</em> is from 1999. It's such a Kent Beck thing to say, though, and Kent is a coauthor of <em>Refactoring</em>, so who knows who came up with that. I'm happy to know of the earlier quote, though. </p> </div> <div class="comment-date">2022-10-02 18:19 UTC</div> </div> <div class="comment" id="d2f99d2cf0d2471ea591103b9eda7e9a"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> I don't view it as an either-or proposition, but rather as a both-this-and-that way of working. </blockquote> <p> I think it is worth elaborating on this. I think am correct in saying that Mark believes that type-driven development and test-driven development are a both-this-and-that way of working instead of an either-or way of working. He did exactly this in his <a href="https://www.pluralsight.com/courses/fsharp-type-driven-development">Pluralsight course titled Type-Driven Development with F#</a> by first obtaining an implementation using type-driven development and then <i>deleting his implementation</i> but keeping his types and obtaining a second implementation using test-driven development. </p> <p> When implementing a new feature, it is important to as quickly as possible derisk by discovering any surprises (aka unknown unknowns) and analyze all challenges (aka known unknowns). The reason for this is to make sure the intended approach is feasible. During this phase of work, we are in the "green" step of test-driven development. Anything goes. There are no rules. The code can horribly ugly or unmaintainable. Just get the failing test to pass. </p> <p> After the test passes, you have proved that the approach is sound. Now you need to share your solution with others. Here is where refactoring first occurs. Just like in Mark's course, I often find it helpful to <i>start over</i>. Now that I know where I am going, I can first refactor the code to make the functional change, which I know will make the test pass. In this way, I know that all my refactors have a clear goal. </p> <blockquote> You refactor as part of adding the feature, not as a separate thing that is done after you have working code. </blockquote> <p> I agree that refactoring should be done as part of the feature, but I disagree that it should (always) be done before you have working code. It is often done after you have working code. </p> <blockquote> Once you have working code, you are done, and when you are estimating the time it will take to add the feature, <i>you include the refactoring time</i>. </blockquote> <p> I agree that estimating should include the refactoring time, but I disagree that you are done when you have working code. When you have working code, you are approximately halfway done. Your code is currently optimized for writing. You still need to optimize it for reading. </p> </div> <div class="comment-date">2022-10-08 17:42 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Coalescing DTOs https://blog.ploeh.dk/2022/09/12/coalescing-dtos 2022-09-12T07:35:00+00:00 Mark Seemann <div id="post"> <p> <em>Refactoring to a universal abstraction.</em> </p> <p> Despite my best efforts, no code base I write is perfect. This is also true for the code base that accompanies <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>. </p> <p> One (among several) warts that has annoyed me for long is this: </p> <p> <pre>[HttpPost(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/reservations&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&lt;ActionResult&gt;&nbsp;Post( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;ReservationDto&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;Guid.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;Reservation?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;BadRequestResult(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;follows...</span></pre> </p> <p> Passing <code>id</code> to <code>Validate</code> annoys me. Why does <code>Validate</code> need an <code>id</code>? </p> <p> When you see it in context, it <em>may</em> makes some sort of sense, but in isolation, it seems arbitrary: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Reservation?&nbsp;Validate(Guid&nbsp;id)</pre> </p> <p> Why does the method need an <code>id</code>? Doesn't <code>ReservationDto</code> have an <code>Id</code>? </p> <h3 id="90765fa49a8149d0bf54d8d9de19ffee"> Abstraction, broken <a href="#90765fa49a8149d0bf54d8d9de19ffee" title="permalink">#</a> </h3> <p> Yes, indeed, <code>ReservationDto</code> <em>has</em> an <code>Id</code> property: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> Then why do callers have to pass an <code>id</code> argument? Doesn't <code>Validate</code> use the <code>Id</code> property? It's almost as though the <code>Validate</code> method <em>begs</em> you to read the implementing code: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Reservation?&nbsp;Validate(Guid&nbsp;id) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!DateTime.TryParse(At,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;d)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Email&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;d, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity); }</pre> </p> <p> Indeed, the method doesn't use the <code>Id</code> property. Reading the code may not be of much help, but at least we learn that <code>id</code> is passed to the <code>Reservation</code> constructor. It's still not clear why the method isn't trying to parse the <code>Id</code> property, like it's doing with <code>At</code>. </p> <p> I'll return to the motivation in a moment, but first I'd like to dwell on the problems of this design. </p> <p> It's a typical example of ad-hoc design. I had a set of behaviours I needed to implement, and in order to avoid code duplication, I came up with a method that seemed to solve the problem. </p> <p> And indeed, the <code>Validate</code> method does solve the problem of code duplication. It also passes all tests. It could be worse. </p> <p> It could also be better. </p> <p> The problem with an ad-hoc design like this is that the motivation is unclear. As a reader, you feel that you're missing the full picture. Perhaps you feel compelled to read the implementation code to gain a better understanding. Perhaps you look for other call sites. Perhaps you search the Git history to find a helpful comment. Perhaps you ask a colleague. </p> <p> It slows you down. Worst of all, it may leave you apprehensive of refactoring. If you feel that there's something you don't fully understand, you may decide to leave the API alone, instead of improving it. </p> <p> It's one of the many ways that code slowly rots. </p> <p> What's missing here is a proper abstraction. </p> <h3 id="c6b50930baf541eb9243611e2090b1e6"> Motivation <a href="#c6b50930baf541eb9243611e2090b1e6" title="permalink">#</a> </h3> <p> I recently hit upon a design that I like better. Before I describe it, however, you need to understand the problem I was trying to solve. </p> <p> The code base for the book is a restaurant reservation REST API, and I was evolving the code as I wrote it. I wanted the code base (and its Git history) to be as realistic as possible. In a real-world situation, you don't always know all requirements up front, or even if you do, they may change. </p> <p> At one point I decided that a REST client could supply a <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier">GUID</a> when making a new reservation. On the other hand, I had lots of existing tests (and a deployed system) that accepted reservations <em>without</em> IDs. In order to not break compatibility, I decided to use the ID if it was supplied with the <a href="https://en.wikipedia.org/wiki/Data_transfer_object">DTO</a>, and otherwise create one. (I later explored <a href="/2021/09/20/keep-ids-internal-with-rest">an API without explicit IDs</a>, but that's a different story.) </p> <p> The <code>id</code> is a JSON property, however, so there's no guarantee that it's properly formatted. Thus, the need to first parse it: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;Guid.NewGuid();</pre> </p> <p> To make matters even more complicated, when you <code>PUT</code> a reservation, the ID is actually part of the resource address, which means that even if it's present in the JSON document, that value should be ignored: </p> <p> <pre>[HttpPut(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/reservations/{id}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&lt;ActionResult&gt;&nbsp;Put( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;ReservationDto&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!Guid.TryParse(id,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;rid)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;NotFoundResult(); &nbsp;&nbsp;&nbsp;&nbsp;Reservation?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(rid); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;BadRequestResult(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;follows...</span></pre> </p> <p> Notice that this <code>Put</code> implementation exclusively considers the resource address <code>id</code> parameter. Recall that the <code>Validate</code> method ignores the <code>dto</code>'s <code>Id</code> property. </p> <p> This is knowledge about implementation details that leaks through to the calling code. As a client developer, you need to know and keep this knowledge in your head while you write your own code. That's not really code that fits in your head. </p> <p> As I usually put it: If you have to read the code, it implies that encapsulation is broken. </p> <p> At the time, however, I couldn't think of a better alternative, and since the problem is still fairly small and localised, I decided to move on. After all, <a href="https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good">perfect is the enemy of good</a>. </p> <h3 id="7e10ecc024e94a20817f0054bde56c29"> Why don't you just..? <a href="#7e10ecc024e94a20817f0054bde56c29" title="permalink">#</a> </h3> <p> Is there a better way? Perhaps you think that you've spotted an obvious improvement. Why don't I just try to parse <code>dto.Id</code> and then create a <code>Guid.NewGuid()</code> if parsing fails? Like this: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Reservation?&nbsp;<span style="color:#74531f;">Validate</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!Guid.TryParse(Id,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">id</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id&nbsp;=&nbsp;Guid.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!DateTime.TryParse(At,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">d</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Email&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;d, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity); }</pre> </p> <p> The short answer is: Because it doesn't work. </p> <p> It may work for <code>Get</code>, but then <code>Put</code> doesn't have a way to tell the <code>Validate</code> method which ID to use. </p> <p> Or rather: That's not entirely true, because this is possible: </p> <p> <pre>dto.Id&nbsp;=&nbsp;id; Reservation?&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;=&nbsp;dto.Validate();</pre> </p> <p> This does suggest an even better way. Before we go there, however, there's another reason I don't like this particular variation: It makes <code>Validate</code> impure. </p> <p> <em>Why care?</em> you may ask. </p> <p> I always end up regretting making an otherwise potentially <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a> non-deterministic. Sooner or later, it turns out to have been a bad decision, regardless of how alluring it initially looked. <a href="/2022/05/23/waiting-to-never-happen">I recently gave an example of that</a>. </p> <p> When weighing the advantages and disadvantages, I preferred passing <code>id</code> explicitly rather than relying on <code>Guid.NewGuid()</code> inside <code>Validate</code>. </p> <h3 id="cb79dc6987854f8b9a897285af649360"> First monoid <a href="#cb79dc6987854f8b9a897285af649360" title="permalink">#</a> </h3> <p> One of the reasons I find <a href="/2017/10/04/from-design-patterns-to-category-theory">universal abstractions</a> beneficial is that you only have to learn them once. As Felienne Hermans writes in <a href="/ref/programmers-brain">The Programmer's Brain</a> our working memory juggles a combination of ephemeral data and knowledge from our long-term memory. The better you can leverage existing knowledge, the easier it is to read code. </p> <p> Which universal abstraction enables you to choose from a prioritised list of candidates? The <a href="/2018/04/03/maybe-monoids">First monoid</a>! </p> <p> In C# with <a href="https://docs.microsoft.com/dotnet/csharp/nullable-references">nullable reference types</a> the <a href="https://docs.microsoft.com/dotnet/csharp/language-reference/operators/null-coalescing-operator">null-coalescing operator</a> <code>??</code> already implements the desired functionality. (If you're using another language or an older version of C#, you can instead use <a href="/2018/06/04/church-encoded-maybe">Maybe</a>.) </p> <p> Once I got that idea I was able to simplify the API. </p> <h3 id="ad2f0a4d20da4c9c8006e9587b011911"> Parsing and coalescing DTOs <a href="#ad2f0a4d20da4c9c8006e9587b011911" title="permalink">#</a> </h3> <p> Instead of that odd <code>Validate</code> method which isn't quite a validator and not quite a parser, this insight suggests to <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">parse, don't validate</a>: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Reservation?&nbsp;<span style="color:#74531f;">TryParse</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!Guid.TryParse(Id,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">id</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!DateTime.TryParse(At,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">d</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Email&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;d, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity); }</pre> </p> <p> This function only returns a parsed <code>Reservation</code> object when the <code>Id</code> is present and well-formed. What about the cases where the <code>Id</code> is absent? </p> <p> The calling <code>ReservationsController</code> can deal with that: </p> <p> <pre>Reservation?&nbsp;<span style="color:#1f377f;">candidate1</span>&nbsp;=&nbsp;dto.TryParse(); dto.Id&nbsp;=&nbsp;Guid.NewGuid().ToString(<span style="color:#a31515;">&quot;N&quot;</span>); Reservation?&nbsp;<span style="color:#1f377f;">candidate2</span>&nbsp;=&nbsp;dto.TryParse(); Reservation?&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;=&nbsp;candidate1&nbsp;??&nbsp;candidate2; <span style="color:#8f08c4;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;BadRequestResult();</pre> </p> <p> First try to parse the <code>dto</code>, then explicitly overwrite its <code>Id</code> property with a new <code>Guid</code>, and then try to parse it again. Finally, pick the first of these that aren't null, using the null-coalescing <code>??</code> operator. </p> <p> This API also works consistently in the <code>Put</code> method: </p> <p> <pre>dto.Id&nbsp;=&nbsp;id; Reservation?&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;=&nbsp;dto.TryParse(); <span style="color:#8f08c4;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;BadRequestResult();</pre> </p> <p> Why is this better? I consider it better because the <code>TryParse</code> function should be a familiar abstraction. Once you've seen a couple of those, you know that a well-behaved parser either returns a valid object, or nothing. You don't have to go and read the implementation of <code>TryParse</code> to (correctly) guess that. Thus, encapsulation is maintained. </p> <h3 id="72d1eececc3f4166ba426d6a39cd6721"> Where does mutation go? <a href="#72d1eececc3f4166ba426d6a39cd6721" title="permalink">#</a> </h3> <p> The <code>ReservationsController</code> mutates the <code>dto</code> and relies on the impure <code>Guid.NewGuid()</code> method. Why is that okay when it wasn't okay to do this inside of <code>Validate</code>? </p> <p> This is because the code base follows the <a href="https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell">functional core, imperative shell</a> architecture. Specifically, Controllers make up the imperative shell, so I consider it appropriate to put impure actions there. After all, they have to go somewhere. </p> <p> This means that the <code>TryParse</code> function remains pure. </p> <h3 id="85ce3840fad04a0c8cb106f6d36459d5"> Conclusion <a href="#85ce3840fad04a0c8cb106f6d36459d5" title="permalink">#</a> </h3> <p> Sometimes a good API design can elude you for a long time. When that happens, I move on with the best solution I can think of in the moment. As it often happens, though, ad-hoc abstractions leave me unsatisfied, so I'm always happy to improve such code later, if possible. </p> <p> In this article, you saw an example of an ad-hoc API design that represented the best I could produce at the time. Later, it dawned on me that an implementation based on a universal abstraction would be possible. In this case, the universal abstraction was null coalescing (which is a specialisation of the <em>monoid</em> abstraction). </p> <p> I like universal abstractions because, once you know them, you can trust that they work in well-understood ways. You don't have to waste time reading implementation code in order to learn whether it's safe to call a method in a particular way. </p> <p> This saves time when you have to work with the code, because, after all, we spend more time reading code than writing it. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="abb3e1c91878423a94835a798fc8b32d"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> After the refactor in this article, is the entirety of your Post method (including the part you didn't show in this article) an <a href="https://blog.ploeh.dk/2020/03/02/impureim-sandwich/">impureim sandwich</a>? </p> </div> <div class="comment-date">2022-09-17 18:37 UTC</div> </div> <div class="comment" id="2352255931b1499184b5ba3782436d61"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Not yet. There's a lot of (preliminary) interleaving of impure actions and pure functions remaining in the controller, even after this refactoring. </p> <p> A future article will tackle that question. One of the reasons I even started writing about monads, functor relationships, etcetera was to establish the foundations for what this requires. If it can be done without monads and traversals I don't know how. </p> <p> Even though the <code>Post</code> method isn't an impureim sandwich, I still consider the architecture <a href="https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell">functional core, imperative shell</a>, since I've kept all impure actions in the controllers. </p> <p> The reason I didn't go all the way to impureim sandwiches with the book's code is didactic. For complex logic, you'll need traversals, monads, sum types, and so on, and none of those things were in scope for the book. </p> </div> <div class="comment-date">2022-09-18 19:28 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The State pattern and the State monad https://blog.ploeh.dk/2022/09/05/the-state-pattern-and-the-state-monad 2022-09-05T12:48:00+00:00 Mark Seemann <div id="post"> <p> <em>The names are the same. Is there a connection? An article for object-oriented programmers.</em> </p> <p> This article is part of <a href="/2018/03/05/some-design-patterns-as-universal-abstractions">a series of articles about specific design patterns and their category theory counterparts</a>. In this article I compare the <a href="https://en.wikipedia.org/wiki/State_pattern">State design pattern</a> to the <a href="/2022/06/20/the-state-monad">State monad</a>. </p> <p> Since the design pattern and the monad share the name <em>State</em> you'd think that they might be <a href="/2018/01/08/software-design-isomorphisms">isomorphic</a>, but it's not quite that easy. I find it more likely that the name is an example of <a href="https://en.wikipedia.org/wiki/Parallel_evolution">parallel evolution</a>. Monads were discovered by <a href="https://en.wikipedia.org/wiki/Eugenio_Moggi">Eugenio Moggi</a> in the early nineties, and <a href="/ref/dp">Design Patterns</a> is from 1994. That's close enough in time that I find it more likely that whoever came up with the names found them independently. <em>State</em>, after all, is hardly an exotic word. </p> <p> Thus, it's possible that the choice of the same name is coincidental. If this is true (which is only my conjecture), does the State pattern have anything in common with the State monad? I find that the answer is a tentative <em>yes</em>. The State design pattern describes an open polymorphic stateful computation. That kind of computation can also be described with the State monad. </p> <p> This article contains a significant amount of code, and it's all quite abstract. It examines the abstract shape of the pattern, so there's little prior intuition on which to build an understanding. While later articles will show more concrete examples, if you want to follow along, you can use the <a href="https://github.com/ploeh/StatePatternAndMonad">GitHub repository</a>. </p> <h3 id="320763be103b49018debc64b45069e3c"> Shape <a href="#320763be103b49018debc64b45069e3c" title="permalink">#</a> </h3> <p> <a href="/ref/dp">Design Patterns</a> is a little vague when it comes to representing the essential form of the pattern. What one can deduce from the diagram in the <em>Structure</em> section describing the pattern, you have an abstract <code>State</code> class with a <code>Handle</code> method like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">virtual</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Handle</span>(Context&nbsp;<span style="color:#1f377f;">context</span>) { }</pre> </p> <p> This, however, doesn't capture all scenarios. What if you need to pass more arguments to the method? What if the method returns a result? What if there's more than one method? </p> <p> Taking into account all those concerns, you might arrive at a more generalised description of the State pattern where an abstract <code>State</code> class might define methods like these: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;Out1&nbsp;<span style="color:#74531f;">Handle1</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>); <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;Out2&nbsp;<span style="color:#74531f;">Handle2</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In2&nbsp;<span style="color:#1f377f;">in2</span>);</pre> </p> <p> There might be an arbitrary number of <code>Handle</code> methods, from <code>Handle1</code> to <code>HandleN</code>, each with their own input and return types. </p> <p> The idea behind the State pattern is that clients don't interact directly with <code>State</code> objects. Instead, they interact with a <code>Context</code> object that delegates operations to a <code>State</code> object, passing itself as an argument: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Out1&nbsp;<span style="color:#74531f;">Request1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;State.Handle1(<span style="color:blue;">this</span>,&nbsp;in1); } <span style="color:blue;">public</span>&nbsp;Out2&nbsp;<span style="color:#74531f;">Request2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;State.Handle2(<span style="color:blue;">this</span>,&nbsp;in2); }</pre> </p> <p> Classes that derive from the abstract <code>State</code> may then mutate <code>context.State</code>. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;Out2&nbsp;<span style="color:#74531f;">Handle2</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(in2&nbsp;==&nbsp;In2.Epsilon) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;context.State&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ConcreteStateB(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Out2.Eta; }</pre> </p> <p> Clients interact with the <code>Context</code> object and aren't aware of this internal machinery: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;ctx.Request2(in2);</pre> </p> <p> With such state mutation going on, is it possible to refactor to a design that uses immutable data and <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>? </p> <h3 id="463f29ae1f8240ec8de7ed18cd429b9f"> State pair <a href="#463f29ae1f8240ec8de7ed18cd429b9f" title="permalink">#</a> </h3> <p> When you have a <code>void</code> method that mutates state, you can refactor it to a pure function by leaving the existing state unchanged and instead returning the new state. What do you do, however, when the method in question already returns a value? </p> <p> This is the case with the generalised <code>HandleN</code> methods, above. </p> <p> One way to resolve this problem is to introduce a more complex type to return. To avoid too much duplication or boilerplate code, you could make it a generic type: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">StatePair</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">StatePair</span>(T&nbsp;<span style="color:#1f377f;">value</span>,&nbsp;State&nbsp;<span style="color:#1f377f;">state</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Value&nbsp;=&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;state; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;Value&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;State&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="color:#74531f;">Equals</span>(<span style="color:blue;">object</span>&nbsp;<span style="color:#1f377f;">obj</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;StatePair&lt;T&gt;&nbsp;<span style="color:#1f377f;">result</span>&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;T&gt;.Default.Equals(Value,&nbsp;result.Value)&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;State&gt;.Default.Equals(State,&nbsp;result.State); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#74531f;">GetHashCode</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;HashCode.Combine(Value,&nbsp;State); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This enables you to change the signatures of the <code>Handle</code> methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;Out1&gt;&nbsp;<span style="color:#74531f;">Handle1</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>); <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;Out2&gt;&nbsp;<span style="color:#74531f;">Handle2</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In2&nbsp;<span style="color:#1f377f;">in2</span>);</pre> </p> <p> This refactoring is always possible. Even if the original return type of a method was <code>void</code>, you can <a href="/2018/01/15/unit-isomorphisms">use a <em>unit</em> type as a replacement for <em>void</em></a>. While redundant but consistent, a method could return <code>StatePair&lt;Unit&gt;</code>. </p> <h3 id="e2ca8d320e684cf89407880d06256b64"> Generic pair <a href="#e2ca8d320e684cf89407880d06256b64" title="permalink">#</a> </h3> <p> The above <code>StatePair</code> type is so coupled to a particular <code>State</code> class that it's not reusable. If you had more than one implementation of the State pattern in your code base, you'd have to duplicate that effort. That seems wasteful, so why not make the type generic in the state dimension as well? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">StatePair</span>&lt;<span style="color:#2b91af;">TState</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">StatePair</span>(T&nbsp;<span style="color:#1f377f;">value</span>,&nbsp;TState&nbsp;<span style="color:#1f377f;">state</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Value&nbsp;=&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;state; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;Value&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TState&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="color:#74531f;">Equals</span>(<span style="color:blue;">object</span>&nbsp;<span style="color:#1f377f;">obj</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;StatePair&lt;TState,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">pair</span>&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;T&gt;.Default.Equals(Value,&nbsp;pair.Value)&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EqualityComparer&lt;TState&gt;.Default.Equals(State,&nbsp;pair.State); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#74531f;">GetHashCode</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;HashCode.Combine(Value,&nbsp;State); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> When you do that then clearly you'd also need to modify the <code>Handle</code> methods accordingly: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Handle1</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>); <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Handle2</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In2&nbsp;<span style="color:#1f377f;">in2</span>);</pre> </p> <p> Notice that, as is the case with <a href="/2021/07/19/the-state-functor">the State functor</a>, the <em>type</em> declares the type with <code>TState</code> before <code>T</code>, while the <em>constructor</em> takes <code>T</code> before <code>TState</code>. While odd and potentially confusing, I've done this to stay consistent with my previous articles, which again do this to stay consistent with prior art (mainly <a href="https://www.haskell.org/">Haskell</a>). </p> <p> With <code>StatePair</code> you can make the methods pure. </p> <h3 id="cacad33b630d4529b4f703337b9b0a61"> Pure functions <a href="#cacad33b630d4529b4f703337b9b0a61" title="permalink">#</a> </h3> <p> Since <code>Handle</code> methods can now return a new state instead of mutating objects, they can be pure functions. Here's an example: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Handle2</span>(Context&nbsp;<span style="color:#1f377f;">context</span>,&nbsp;In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(in2&nbsp;==&nbsp;In2.Epsilon) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;(Out2.Eta,&nbsp;<span style="color:blue;">new</span>&nbsp;ConcreteStateB()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;(Out2.Eta,&nbsp;<span style="color:blue;">this</span>); }</pre> </p> <p> The same is true for <code>Context</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">pair</span>&nbsp;=&nbsp;State.Handle1(<span style="color:blue;">this</span>,&nbsp;in1); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;StatePair&lt;Context,&nbsp;Out1&gt;(pair.Value,&nbsp;<span style="color:blue;">new</span>&nbsp;Context(pair.State)); } <span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Request2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">pair</span>&nbsp;=&nbsp;State.Handle2(<span style="color:blue;">this</span>,&nbsp;in2); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;StatePair&lt;Context,&nbsp;Out2&gt;(pair.Value,&nbsp;<span style="color:blue;">new</span>&nbsp;Context(pair.State)); }</pre> </p> <p> Does this begin to look familiar? </p> <h3 id="bf25630a0d324f628f400c73ddaa78fa"> Monad <a href="#bf25630a0d324f628f400c73ddaa78fa" title="permalink">#</a> </h3> <p> The <code>StatePair</code> class is nothing but a glorified tuple. Armed with that knowledge, you can introduce a variation of <a href="/2021/07/19/the-state-functor">the IState interface I used to introduce the State functor</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IState</span>&lt;<span style="color:#2b91af;">TState</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;StatePair&lt;TState,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">Run</span>(TState&nbsp;<span style="color:#1f377f;">state</span>); }</pre> </p> <p> This variation uses the explicit <code>StatePair</code> class as the return type of <code>Run</code>, rather than a more anonymous tuple. These representations are isomorphic. (That might be a good exercise: Write functions that convert from one to the other, and vice versa.) </p> <p> You can write the usual <code>Select</code> and <code>SelectMany</code> implementations to make <code>IState</code> a functor and monad. Since I have already shown these in previous articles, I'm also going to skip those. (Again, it might be a good exercise to implement them if you're in doubt of how they work.) </p> <p> You can now, for example, use C# query syntax to run the same computation multiple times: </p> <p> <pre>IState&lt;Context,&nbsp;(Out1&nbsp;a,&nbsp;Out1&nbsp;b)&gt;&nbsp;<span style="color:#1f377f;">s</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;a&nbsp;<span style="color:blue;">in</span>&nbsp;in1.Request1() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;b&nbsp;<span style="color:blue;">in</span>&nbsp;in1.Request1() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;(a,&nbsp;b); StatePair&lt;Context,&nbsp;(Out1&nbsp;a,&nbsp;Out1&nbsp;b)&gt;&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&nbsp;s.Run(ctx);</pre> </p> <p> This example calls <code>Request1</code> twice, and collects both return values in a tuple. Running the computation with a <code>Context</code> will produce both a result (the two outputs <code>a</code> and <code>b</code>) as well as the 'current' <code>Context</code> (state). </p> <p> <code>Request1</code> is a State-valued extension method on <code>In1</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(<span style="color:blue;">this</span>&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;ctx&nbsp;<span style="color:blue;">in</span>&nbsp;Get&lt;Context&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;p&nbsp;=&nbsp;ctx.Request1(in1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;Put(p.State) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;p.Value; }</pre> </p> <p> Notice the abstraction level in play. This extension method doesn't return a <code>StatePair</code>, but rather an <code>IState</code> computation, defined by using <a href="/2022/07/04/get-and-put-state">the State monad's Get and Put functions</a>. Since the computation is running with a <code>Context</code> state, the computation can <code>Get</code> a <code>ctx</code> object and call its <code>Request1</code> method. This method returns a pair <code>p</code>. The computation can then <code>Put</code> the pair's <code>State</code> (here, a <code>Context</code> object) and return the pair's <code>Value</code>. </p> <p> This stateful computation is composed from the building blocks of the State monad, including query syntax supported by <code>SelectMany</code>, <code>Get</code>, and <code>Put</code>. </p> <p> This does, however, still feel unsatisfactory. After all, you have to know enough of the details of the State monad to know that <code>ctx.Request1</code> returns a pair of which you must remember to <code>Put</code> the <code>State</code>. Would it be possible to also express the underlying <code>Handle</code> methods as stateful computations? </p> <h3 id="f8c4147c58924cd19f1d0bdae8fc05a5"> StatePair bifunctor <a href="#f8c4147c58924cd19f1d0bdae8fc05a5" title="permalink">#</a> </h3> <p> The <code>StatePair</code> class is isomorphic to a <em>pair</em> (a <em>two-tuple</em>), and we know that <a href="/2018/12/31/tuple-bifunctor">a pair gives rise to a bifunctor</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;TState1,&nbsp;T1&gt;&nbsp;<span style="color:#74531f;">SelectBoth</span>&lt;<span style="color:#2b91af;">TState1</span>,&nbsp;<span style="color:#2b91af;">T1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">selectValue</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;TState,&nbsp;TState1&gt;&nbsp;<span style="color:#1f377f;">selectState</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;StatePair&lt;TState1,&nbsp;T1&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selectValue(Value), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selectState(State)); }</pre> </p> <p> You can use <code>SelectBoth</code> to implement both <code>Select</code> and <code>SelectState</code>. In the following we're only going to need <code>SelectState</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;TState1,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">SelectState</span>&lt;<span style="color:#2b91af;">TState1</span>&gt;(Func&lt;TState,&nbsp;TState1&gt;&nbsp;<span style="color:#1f377f;">selectState</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;SelectBoth(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x,&nbsp;selectState); }</pre> </p> <p> This enables us to slightly simplify the <code>Context</code> methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;State.Handle1(<span style="color:blue;">this</span>,&nbsp;in1).SelectState(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s)); } <span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Request2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;State.Handle2(<span style="color:blue;">this</span>,&nbsp;in2).SelectState(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s)); }</pre> </p> <p> Keep in mind that <code>Handle1</code> returns a <code>StatePair&lt;State,&nbsp;Out1&gt;</code>, <code>Handle2</code> returns <code>StatePair&lt;State,&nbsp;Out2&gt;</code>, and so on. While <code>Request1</code> calls <code>Handle1</code>, it must return a <code>StatePair&lt;Context,&nbsp;Out1&gt;</code> rather than a <code>StatePair&lt;State,&nbsp;Out1&gt;</code>. Since <code>StatePair</code> is a bifunctor, the <code>Request1</code> method can use <code>SelectState</code> to map the <code>State</code> to a <code>Context</code>. </p> <p> Unfortunately, this doesn't seem to move us much closer to being able to express the underlying functions as stateful computations. It does, however, set up the code so that the next change is a little easier to follow. </p> <h3 id="21f08903569d4c1a898914092f2bf3f9"> State computations <a href="#21f08903569d4c1a898914092f2bf3f9" title="permalink">#</a> </h3> <p> Is it possible to express the <code>Handle</code> methods on <code>State</code> as <code>IState</code> computations? One option is to write another extension method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;State,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1S</span>(<span style="color:blue;">this</span>&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;s&nbsp;<span style="color:blue;">in</span>&nbsp;Get&lt;State&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;ctx&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;p&nbsp;=&nbsp;s.Handle1(ctx,&nbsp;in1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;Put(p.State) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;p.Value; }</pre> </p> <p> I had to add an <code>S</code> suffix to the name, since it only differs from the above <code>Request1</code> extension method on its return type, and C# doesn't allow method overloading on return types. </p> <p> You can add a similar <code>Request2S</code> extension method. It feels like boilerplate code, but enables us to express the <code>Context</code> methods in terms of running stateful computations: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;in1.Request1S().Run(State).SelectState(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s)); } <span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Request2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;in2.Request2S().Run(State).SelectState(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s)); }</pre> </p> <p> This still isn't entirely satisfactory, since the return types of these <code>Request</code> methods are state pairs, and not <code>IState</code> values. The above <code>Request1S</code> function, however, contains a clue about how to proceed. Notice how it can create a <code>Context</code> object from the underlying <code>State</code>, and convert that <code>Context</code> object back to a <code>State</code> object. That's a generalizable idea. </p> <h3 id="6ec1667fe3354e8bb9b5f7b3540b74f9"> Invariant functor <a href="#6ec1667fe3354e8bb9b5f7b3540b74f9" title="permalink">#</a> </h3> <p> While it's possible to map the <code>TState</code> dimension of the state pair, it seems harder to do it on <code><span style="color:#2b91af;">IState</span>&lt;<span style="color:#2b91af;">TState</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt;</code>. A tuple, after all, is covariant in both dimensions. The State monad, on the other hand, is neither co- nor contravariant in the state dimension. You can deduce this with positional variance analysis (which I've learned from <a href="https://thinkingwithtypes.com/">Thinking with Types</a>). In short, this is because <code>TState</code> appears as both input and output in <code>StatePair&lt;TState,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">Run</span>(TState&nbsp;<span style="color:#1f377f;">state</span>)</code> - it's neither co- nor contravariant, but rather <em>invariant</em>. </p> <p> What little option is left us, then, is to make <code>IState</code> an <a href="/2022/08/01/invariant-functors">invariant functor</a> in the state dimension: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;TState1,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">SelectState</span>&lt;<span style="color:#2b91af;">TState</span>,&nbsp;<span style="color:#2b91af;">TState1</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IState&lt;TState,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">state</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;TState,&nbsp;TState1&gt;&nbsp;<span style="color:#1f377f;">forward</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;TState1,&nbsp;TState&gt;&nbsp;<span style="color:#1f377f;">back</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;s1&nbsp;<span style="color:blue;">in</span>&nbsp;Get&lt;TState1&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;s&nbsp;=&nbsp;back(s1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;p&nbsp;=&nbsp;state.Run(s) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;Put(forward(p.State)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;p.Value; }</pre> </p> <p> Given an <code>IState&lt;TState,&nbsp;T&gt;</code> the <code>SelectState</code> function enables us to turn it into a <code>IState&lt;TState1,&nbsp;T&gt;</code>. This is, however, only possible if you can translate both <code>forward</code> and <code>back</code> between two representations. When we have two such translations, we can produce a new computation that runs in <code>TState1</code> by first using <code>Get</code> to retrieve a <code>TState1</code> value from the new environment, translate it <code>back</code> to <code>TState</code>, which enables the expression to <code>Run</code> the <code>state</code>. Then translate the resulting <code>p.State</code> <code>forward</code> and <code>Put</code> it. Finally, return the <code>Value</code>. </p> <p> As <a href="https://reasonablypolymorphic.com/">Sandy Maguire</a> explains: </p> <blockquote> <p> "... an invariant type <code>T</code> allows you to map from <code>a</code> to <code>b</code> if and only if <code>a</code> and <code>b</code> are isomorphic. [...] an isomorphism between <code>a</code> and <code>b</code> means they're already the same thing to begin with." </p> <footer><cite>Sandy Maguire, <a href="https://thinkingwithtypes.com/">Thinking with Types</a></cite></footer> </blockquote> <p> This may seem limiting, but is enough in this case. The <code>Context</code> class is only a wrapper of a <code>State</code> object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Context</span>(State&nbsp;<span style="color:#1f377f;">state</span>) { &nbsp;&nbsp;&nbsp;&nbsp;State&nbsp;=&nbsp;state; } <span style="color:blue;">public</span>&nbsp;State&nbsp;State&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;}</pre> </p> <p> If you have a <code>State</code> object, you can create a <code>Context</code> object via the <code>Context</code> constructor. On the other hand, if you have a <code>Context</code> object, you can get the wrapped <code>State</code> object by reading the <code>State</code> property. </p> <p> The first improvement this offers is simplification of the <code>Request1</code> extension method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(<span style="color:blue;">this</span>&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;in1.Request1S().SelectState(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Context(s),&nbsp;<span style="color:#1f377f;">ctx</span>&nbsp;=&gt;&nbsp;ctx.State); }</pre> </p> <p> Recall that <code>Request1S</code> returns a <code>IState&lt;State,&nbsp;Out1&gt;</code>. Since a two-way translation between <code>State</code> and <code>Context</code> exists, <code>SelectState</code> can translate <code>IState&lt;State,&nbsp;Out1&gt;</code> to <code>IState&lt;Context,&nbsp;Out1&gt;</code>. </p> <p> The same applies to the equivalent <code>Request2</code> extension method. </p> <p> This, again, enables us to rewrite the <code>Context</code> methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;in1.Request1().Run(<span style="color:blue;">this</span>); } <span style="color:blue;">public</span>&nbsp;StatePair&lt;Context,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Request2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;in2.Request2().Run(<span style="color:blue;">this</span>); }</pre> </p> <p> While this may seem like an insignificant change, one result has been gained: This last refactoring pushed the <code>Run</code> call to the right. It's now clear that each expression is a stateful computation, and that the only role that the <code>Request</code> methods play is to <code>Run</code> the computations. </p> <p> This illustrates that the <code>Request</code> methods can be decomposed into two decoupled steps: <ol> <li>A stateful computation expression</li> <li>Running the expression</li> </ol> The question now becomes: How useful is the <code>Context</code> wrapper class now? </p> <h3 id="4fc39c9d7bd647cb9773344082468051"> Eliminating the Context <a href="#4fc39c9d7bd647cb9773344082468051" title="permalink">#</a> </h3> <p> A reasonable next refactoring might be to remove the <code>context</code> parameter from each of the <code>Handle</code> methods. After all, this parameter is a remnant of the State design pattern. Its original purpose was to enable <code>State</code> implementers to mutate the <code>context</code> by changing its <code>State</code>. </p> <p> After refactoring to immutable functions, the <code>context</code> parameter no longer needs to be there - for that reason. Do we need it for other reasons? Does it carry other information that a <code>State</code> implementer might need? </p> <p> In the form that the code now has, it doesn't. Even if it did, we could consider moving that data to the other input parameter: <code>In1</code>, <code>In2</code>, etcetera. </p> <p> Therefore, it seems sensible to remove the <code>context</code> parameter from the <code>State</code> methods: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Handle1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>); <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Handle2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>);</pre> </p> <p> This also means that a function like <code>Request1S</code> becomes simpler: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;State,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Request1S</span>(<span style="color:blue;">this</span>&nbsp;In1&nbsp;<span style="color:#1f377f;">in1</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;s&nbsp;<span style="color:blue;">in</span>&nbsp;Get&lt;State&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;p&nbsp;=&nbsp;s.Handle1(in1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;Put(p.State) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;p.Value; }</pre> </p> <p> Since <code>Context</code> and <code>State</code> are isomorphic, you can rewrite all callers of <code>Context</code> to instead use <code>State</code>, like the above example: </p> <p> <pre>IState&lt;State,&nbsp;(Out1&nbsp;a,&nbsp;Out1&nbsp;b)&gt;&nbsp;<span style="color:#1f377f;">s</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;a&nbsp;<span style="color:blue;">in</span>&nbsp;in1.Request1() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;b&nbsp;<span style="color:blue;">in</span>&nbsp;in1.Request1() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;(a,&nbsp;b); <span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&nbsp;s.Run(csa);</pre> </p> <p> Do this consistently, and you can eventually delete the <code>Context</code> class. </p> <h3 id="c92292ad071c4b468fa223d764d89ab8"> Further possible refactorings <a href="#c92292ad071c4b468fa223d764d89ab8" title="permalink">#</a> </h3> <p> With the <code>Context</code> class gone, you're left with the abstract <code>State</code> class and its implementers: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">State</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out1&gt;&nbsp;<span style="color:#74531f;">Handle1</span>(In1&nbsp;<span style="color:#1f377f;">in1</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">abstract</span>&nbsp;StatePair&lt;State,&nbsp;Out2&gt;&nbsp;<span style="color:#74531f;">Handle2</span>(In2&nbsp;<span style="color:#1f377f;">in2</span>); }</pre> </p> <p> One further change worth considering might be to change the abstract base class to an interface. </p> <p> In this article, I've considered the general case where the <code>State</code> class supports an arbitrary number of independent state transitions, symbolised by the methods <code>Handle1</code> and <code>Handle2</code>. With an arbitrary number of such state transitions, you would have additional methods up to <code>HandleN</code> for <em>N</em> independent state transitions. </p> <p> At the other extreme, you may have just a single polymorphic state transition function. My intuition tells me that that's more likely to be the case than one would think at first. </p> <h3 id="c53b386a92b04575b675272d2dc85365"> Relationship between pattern and monad <a href="#c53b386a92b04575b675272d2dc85365" title="permalink">#</a> </h3> <p> You can view the State design pattern as a combination of two common practices in object-oriented programming: Mutation and polymorphism. </p> <p> <img src="/content/binary/state-pattern-venn.png" alt="Venn diagram with the two sets mutation and polymorphism. The intersection is labelled state."> </p> <p> The patterns in <em>Design Patterns</em> rely heavily on mutation of object state. Most other 'good' object-oriented code tends to do likewise. </p> <p> Proper object-oriented code also makes good use of polymorphism. Again, refer to <em>Design Patterns</em> or a book like <a href="https://blog.ploeh.dk/ref/refactoring">Refactoring</a> for copious examples. </p> <p> I view the State pattern as the intersection of these two common practices. The problem to solve is this: </p> <blockquote> <p> "Allow an object to alter its behavior when its internal state changes." </p> <footer><cite><a href="/ref/dp">Design Patterns</a></cite></footer> </blockquote> <p> The State pattern achieves that goal by having an inner polymorphic object (<code>State</code>) wrapped by an container object (<code>Context</code>). The <code>State</code> objects can mutate the <code>Context</code>, which enables them to replace themselves with other states. </p> <p> While functional programming also has notions of polymorphism, a pure function can't mutate state. Instead, a pure function must return a new state, leaving the old state unmodified. If there's nothing else to return, you can model such state-changing behaviour as an <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a>. The article <a href="/2021/05/31/from-state-tennis-to-endomorphism">From State tennis to endomorphism</a> gives a quite literal example of that. </p> <p> Sometimes, however, an object-oriented method does more than one thing: It both mutates state and returns a value. (This, by the way, violates <a href="https://en.wikipedia.org/wiki/Command%E2%80%93query_separation">the Command Query Separation principle</a>.) The State monad is the functional way of doing that: Return both the result and the new state. </p> <p> Essentially, you replace mutation with the State monad. </p> <p> <img src="/content/binary/state-monad-venn.png" alt="Venn diagram with the two sets state monad and polymorphism. The intersection is labelled state."> </p> <p> From a functional perspective, then, we can view the State pattern as the intersection of polymorphism and the State monad. </p> <h3 id="d31601e4146d4f48ae30228f697de0a0"> Examples <a href="#d31601e4146d4f48ae30228f697de0a0" title="permalink">#</a> </h3> <p> This article is both long and abstract. Some examples might be helpful, so I'll give a few in separate articles: <ul> <li><a href="/2022/09/26/refactoring-the-tcp-state-pattern-example-to-pure-functions">Refactoring the TCP State pattern example to pure functions</a></li> <li><a href="/2022/10/10/refactoring-a-saga-from-the-state-pattern-to-the-state-monad">Refactoring a saga from the State pattern to the State monad</a></li> </ul> The first one uses the example from <em>Design Patterns</em>. That example is, unfortunately not that illuminating, since none of the book's methods return data. Thus, the endomorphism-based refactoring is enough, and you don't need the State monad. Therefore, another example is warranted. </p> <h3 id="1b5dead7f5c84f3896d639837314f17d"> Conclusion <a href="#1b5dead7f5c84f3896d639837314f17d" title="permalink">#</a> </h3> <p> You can view the State design pattern as the intersection of polymorphism and mutation. Both are object-oriented staples. The pattern uses polymorphism to model state, and mutation to change from one polymorphic state to another. </p> <p> In functional programming pure functions can't mutate state. You can often design around that problem, but if all else fails, the State monad offers a general-purpose alternative to both return a value and change object state. Thus, you can view the functional equivalent of the State pattern as the intersection of polymorphism and the State monad. </p> <p> <strong>Next:</strong> <a href="/2022/09/26/refactoring-the-tcp-state-pattern-example-to-pure-functions">Refactoring the TCP State pattern example to pure functions</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Natural transformations as invariant functors https://blog.ploeh.dk/2022/08/29/natural-transformations-as-invariant-functors 2022-08-29T06:12:00+00:00 Mark Seemann <div id="post"> <p> <em>An article (also) for object-oriented programmers.</em> </p> <ins datetime="2022-09-04T06:40Z"> <p> <strong>Update 2022-09-04:</strong> <em>This article is most likely partially incorrect. What it describes works, but may not be a natural transformation. See <a href="#9336107b26fd48a8971e617d8ebd5159">the below comment</a> for more details.</em> </p> </ins> <p> This article is part of <a href="/2022/08/01/invariant-functors">a series of articles about invariant functors</a>. An invariant functor is a <a href="/2018/03/22/functors">functor</a> that is neither covariant nor contravariant. See the series introduction for more details. The <a href="/2022/08/08/endomorphism-as-an-invariant-functor">previous article</a> described how you can view an <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a> as an invariant functor. This article generalises that result. </p> <h3 id="f3a2cab08b004fe2bfa8e4b307317d88"> Endomorphism as a natural transformation <a href="#f3a2cab08b004fe2bfa8e4b307317d88" title="permalink">#</a> </h3> <p> An endomorphism is a function whose <a href="https://en.wikipedia.org/wiki/Domain_of_a_function">domain</a> and <a href="https://en.wikipedia.org/wiki/Codomain">codomain</a> is the same. In C# you'd denote the type as <code>Func&lt;T, T&gt;</code>, in <a href="https://fsharp.org/">F#</a> as <code>'a -&gt; 'a</code>, and in <a href="https://www.haskell.org/">Haskell</a> as <code>a -&gt; a</code>. <code>T</code>, <code>'a</code>, and <code>a</code> all symbolise generic types - the notation is just different, depending on the language. </p> <p> A 'naked' value is isomorphic to <a href="/2018/09/03/the-identity-functor">the Identity functor</a>. You can wrap a value of the type <code>a</code> in <code>Identity a</code>, and if you have an <code>Identity a</code>, you can extract the <code>a</code> value. </p> <p> An endomorphism is thus isomorphic to a function from Identity to Identity. In C#, you might denote that as <code>Func&lt;Identity&lt;T&gt;, Identity&lt;T&gt;&gt;</code>, and in Haskell as <code>Identity a -&gt; Identity a</code>. </p> <p> In fact, you can lift any function to an Identity-valued function: </p> <p> <pre>Prelude Data.Functor.Identity&gt; :t \f -&gt; Identity . f . runIdentity \f -&gt; Identity . f . runIdentity :: (b -&gt; a) -&gt; Identity b -&gt; Identity a</pre> </p> <p> While this is a general result that allows <code>a</code> and <code>b</code> to differ, when <code>a ~ b</code> this describes an endomorphism. </p> <p> Since Identity is a functor, a function <code>Identity a -&gt; Identity a</code> is a <a href="/2022/07/18/natural-transformations">natural transformation</a>. </p> <p> The identity function (<code>id</code> in F# and Haskell; <code>x =&gt; x</code> in C#) is the only one possible entirely general endomorphism. You can use the <a href="https://hackage.haskell.org/package/natural-transformation">natural-transformation</a> package to make it explicit that this is a natural transformation: </p> <p> <pre><span style="color:#2b91af;">idNT</span>&nbsp;::&nbsp;<span style="color:blue;">Identity</span>&nbsp;:~&gt;&nbsp;<span style="color:blue;">Identity</span> idNT&nbsp;=&nbsp;NT&nbsp;$&nbsp;Identity&nbsp;.&nbsp;<span style="color:blue;">id</span>&nbsp;.&nbsp;runIdentity</pre> </p> <p> The point, so far, is that you can view an endomorphism as a natural transformation. </p> <p> Since an endomorphism forms an invariant functor, this suggests a promising line of inquiry. </p> <h3 id="88842564359c40978adb687e27cc460d"> Natural transformations as invariant functors <a href="#88842564359c40978adb687e27cc460d" title="permalink">#</a> </h3> <p> Are <em>all</em> natural transformations invariant functors? </p> <p> Yes, they are. In Haskell, you can implement it like this: </p> <p> <pre><span style="color:blue;">instance</span>&nbsp;(<span style="color:blue;">Functor</span>&nbsp;f,&nbsp;<span style="color:blue;">Functor</span>&nbsp;g)&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;<span style="color:blue;">Invariant</span>&nbsp;(<span style="color:blue;">NT</span>&nbsp;f&nbsp;g)&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;invmap&nbsp;f&nbsp;g&nbsp;(NT&nbsp;h)&nbsp;=&nbsp;NT&nbsp;$&nbsp;<span style="color:blue;">fmap</span>&nbsp;f&nbsp;.&nbsp;h&nbsp;.&nbsp;<span style="color:blue;">fmap</span>&nbsp;g</pre> </p> <p> Here, I chose to define <code>NT</code> from scratch, rather than relying on the <em>natural-transformation</em> package. </p> <p> <pre><span style="color:blue;">newtype</span>&nbsp;NT&nbsp;f&nbsp;g&nbsp;a&nbsp;=&nbsp;NT&nbsp;{&nbsp;unNT&nbsp;::&nbsp;f&nbsp;a&nbsp;-&gt;&nbsp;g&nbsp;a&nbsp;}</pre> </p> <p> Notice how the implementation (<code><span style="color:blue;">fmap</span>&nbsp;f&nbsp;.&nbsp;h&nbsp;.&nbsp;<span style="color:blue;">fmap</span>&nbsp;g</code>) looks like a generalisation of the endomorphism implementation of <code>invmap</code> (<code>f . h . g</code>). Instead of pre-composing with <code>g</code>, the generalisation pre-composes with <code>fmap g</code>, and instead of post-composing with <code>f</code>, it post-composes with <code>fmap f</code>. </p> <p> Using the same kind of diagram as in the previous article, this composition now looks like this: </p> <p> <img src="/content/binary/invariant-natural-transformation-map-diagram.png" alt="Arrow diagram showing the mapping from a natural transformation in a to a natural transformation in b."> </p> <p> I've used thicker arrows to indicate that each one potentially involves 'more work'. Each is a mapping from a functor to a functor. For the <a href="/2022/04/19/the-list-monad">List functor</a>, for example, the arrow implies zero to many values being mapped. Thus, 'more data' moves 'through' each arrow, and for that reason I thought it made sense to depict them as being thicker. This 'more data' view is not always correct. For example, for <a href="/2018/03/26/the-maybe-functor">the Maybe functor</a>, the amount of data transported though each arrow is zero or one, which rather suggests a thinner arrow. For something like <a href="/2021/07/19/the-state-functor">the State functor</a> or <a href="/2021/08/30/the-reader-functor">the Reader functor</a>, there's really no <em>data</em> in the strictest sense moving through the arrows, but rather functions (which are also, however, a kind of data). Thus, don't take this metaphor of the thicker arrows literally. I did, however, wish to highlight that there's something 'more' going on. </p> <p> The diagram shows a natural transformation <code>h</code> from some functor <code>F</code> to another functor <code>G</code>. It transports objects of the type <code>a</code>. If <code>a</code> and <code>b</code> are isomorphic, you can map that natural transformation to one that transports objects of the type <code>b</code>. </p> <p> Compared to endomorphisms, where you need to, say, map <code>b</code> to <code>a</code>, you now need to map <code>F b</code> to <code>F a</code>. If <code>g</code> maps <code>b</code> to <code>a</code>, then <code>fmap g</code> maps <code>F b</code> to <code>F a</code>. The same line of argument applies to <code>fmap f</code>. </p> <p> In C# you can implement the same behaviour as follows. Assume that you have a natural transformation <code>H</code> from the functor <code>F</code> to the functor <code>G</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Func&lt;F&lt;A&gt;,&nbsp;G&lt;A&gt;&gt;&nbsp;H&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;}</pre> </p> <p> You can now implement a non-standard <code>Select</code> overload (as described in the introductory article) that maps a natural transformation <code>FToG&lt;A&gt;</code> to a natural transformation <code>FToG&lt;B&gt;</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;FToG&lt;B&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">B</span>&gt;(Func&lt;A,&nbsp;B&gt;&nbsp;<span style="color:#1f377f;">aToB</span>,&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">bToA</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;FToG&lt;B&gt;(<span style="color:#1f377f;">fb</span>&nbsp;=&gt;&nbsp;H(fb.Select(bToA)).Select(aToB)); }</pre> </p> <p> The implementation looks more imperative than in Haskell, but the idea is the same. First it uses <code>Select</code> on <code>F</code> in order to translate <code>fb</code> (of the type <code>F&lt;B&gt;</code>) to an <code>F&lt;A&gt;</code>. It then uses <code>H</code> to transform the <code>F&lt;A&gt;</code> to an <code>G&lt;A&gt;</code>. Finally, now that it has a <code>G&lt;A&gt;</code>, it can use <code>Select</code> on <em>that</em> functor to map to a <code>G&lt;B&gt;</code>. </p> <p> Note that there's two different functors (<code>F</code> and <code>G</code>) in play, so the two <code>Select</code> methods are different. This is also true in the Haskell code. <code>fmap g</code> need not be the same as <code>fmap f</code>. </p> <h3 id="70c39c1999864a82a73a4ce22f252110"> Identity law <a href="#70c39c1999864a82a73a4ce22f252110" title="permalink">#</a> </h3> <p> As in the previous article, I'll set out to <em>prove</em> the two laws for invariant functors, starting with the identity law. Again, I'll use equational reasoning with <a href="https://bartoszmilewski.com/2015/01/20/functors/">the notation that Bartosz Milewski uses</a>. Here's the proof that the <code>invmap</code> instance obeys the identity law: </p> <p> <pre>&nbsp;&nbsp;invmap&nbsp;id&nbsp;id&nbsp;(NT&nbsp;h) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;fmap&nbsp;id&nbsp;.&nbsp;h&nbsp;.&nbsp;fmap&nbsp;id =&nbsp;{&nbsp;first&nbsp;functor&nbsp;law&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;id&nbsp;.&nbsp;h&nbsp;.&nbsp;id =&nbsp;{&nbsp;eta&nbsp;expansion&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;(\x&nbsp;-&gt;&nbsp;(id&nbsp;.&nbsp;h&nbsp;.&nbsp;id)&nbsp;x) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;(.)&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;(\x&nbsp;-&gt;&nbsp;id(h(id(x)))) =&nbsp;{&nbsp;defintion&nbsp;of&nbsp;id&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;(\x&nbsp;-&gt;&nbsp;h(x)) =&nbsp;{&nbsp;eta&nbsp;reduction&nbsp;} &nbsp;&nbsp;NT&nbsp;h =&nbsp;{&nbsp;definition&nbsp;of&nbsp;id&nbsp;} &nbsp;&nbsp;id&nbsp;(NT&nbsp;h)</pre> </p> <p> I'll leave it here without further comment. The Haskell type system is so expressive and abstract that it makes little sense to try to translate these findings to C# or F# in the abstract. Instead, you'll see some more concrete examples later. </p> <h3 id="d54ff79951db44559583a200d20cf6d9"> Composition law <a href="#d54ff79951db44559583a200d20cf6d9" title="permalink">#</a> </h3> <p> As with the identity law, I'll offer a proof for the composition law for the Haskell instance: </p> <p> <pre>&nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;$&nbsp;invmap&nbsp;f1&nbsp;f1&#39;&nbsp;(NT&nbsp;h) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;$&nbsp;NT&nbsp;$&nbsp;fmap&nbsp;f1&nbsp;.&nbsp;h&nbsp;.&nbsp;fmap&nbsp;f1&#39; =&nbsp;{&nbsp;defintion&nbsp;of&nbsp;($)&nbsp;} &nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;(NT&nbsp;(fmap&nbsp;f1&nbsp;.&nbsp;h&nbsp;.&nbsp;fmap&nbsp;f1&#39;)) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;fmap&nbsp;f2&nbsp;.&nbsp;(fmap&nbsp;f1&nbsp;.&nbsp;h&nbsp;.&nbsp;fmap&nbsp;f1&#39;)&nbsp;.&nbsp;fmap&nbsp;f2&#39; =&nbsp;{&nbsp;associativity&nbsp;of&nbsp;composition&nbsp;(.)&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;(fmap&nbsp;f2&nbsp;.&nbsp;fmap&nbsp;f1)&nbsp;.&nbsp;h&nbsp;.&nbsp;(fmap&nbsp;f1&#39;&nbsp;.&nbsp;fmap&nbsp;f2&#39;) =&nbsp;{&nbsp;second&nbsp;functor&nbsp;law&nbsp;} &nbsp;&nbsp;NT&nbsp;$&nbsp;fmap&nbsp;(f2&nbsp;.&nbsp;f1)&nbsp;.&nbsp;h&nbsp;.&nbsp;fmap&nbsp;(f1&#39;&nbsp;.&nbsp;f2&#39;) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;invmap&nbsp;(f2&nbsp;.&nbsp;f1)&nbsp;(f1&#39;&nbsp;.&nbsp;f2&#39;)&nbsp;(NT&nbsp;h)</pre> </p> <p> Unless I've made a mistake, these two proofs should demonstrate that all natural transformations can be turned into an invariant functor - in Haskell, at least, but I'll conjecture that that result carries over to other languages like F# and C# as long as one stays within the confines of <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. </p> <h3 id="2afc7c35819c4d499f531a7229102404"> The State functor as a natural transformation <a href="#2afc7c35819c4d499f531a7229102404" title="permalink">#</a> </h3> <p> I'll be honest and admit that my motivation for embarking on this exegesis was because I'd come to the realisation that you can think about <a href="/2021/07/19/the-state-functor">the State functor</a> as a natural transformation. Recall that <code>State</code> is usually defined like this: </p> <p> <pre><span style="color:blue;">newtype</span>&nbsp;State&nbsp;s&nbsp;a&nbsp;=&nbsp;State&nbsp;{&nbsp;runState&nbsp;::&nbsp;s&nbsp;-&gt;&nbsp;(a,&nbsp;s)&nbsp;}</pre> </p> <p> You can easily establish that this definition of <code>State</code> is isomorphic with a natural transformation from <a href="/2018/09/03/the-identity-functor">the Identity functor</a> to <a href="/2018/12/31/tuple-bifunctor">the tuple functor</a>: </p> <p> <pre><span style="color:#2b91af;">stateToNT</span>&nbsp;::&nbsp;<span style="color:blue;">State</span>&nbsp;s&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">NT</span>&nbsp;<span style="color:blue;">Identity</span>&nbsp;((,)&nbsp;a)&nbsp;s stateToNT&nbsp;(State&nbsp;h)&nbsp;=&nbsp;NT&nbsp;$&nbsp;h&nbsp;.&nbsp;runIdentity <span style="color:#2b91af;">ntToState</span>&nbsp;::&nbsp;<span style="color:blue;">NT</span>&nbsp;<span style="color:blue;">Identity</span>&nbsp;((,)&nbsp;a)&nbsp;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">State</span>&nbsp;s&nbsp;a ntToState&nbsp;(NT&nbsp;h)&nbsp;=&nbsp;State&nbsp;$&nbsp;h&nbsp;.&nbsp;Identity</pre> </p> <p> Notice that this is a natural transformation in <code>s</code> - not in <code>a</code>. </p> <p> Since I've already established that natural transformations form invariant functors, this also applies to the State monad. </p> <h3 id="6c0025936a9949eca6191989f4d1b8c6"> State mapping <a href="#6c0025936a9949eca6191989f4d1b8c6" title="permalink">#</a> </h3> <p> My point with all of this isn't really to insist that anyone makes actual use of all this machinery, but rather that this line of reasoning helps to <em>identify a capability</em>. We now know that it's possible to translate a <code>State s a</code> value to a <code>State t a</code> value if <code>s</code> is isomorphic to <code>t</code>. </p> <p> As an example, imagine that you have some State-valued function that attempts to find the maximum value based on various criteria. Such a <code>pickMax</code> function may have the type <code><span style="color:blue;">State</span>&nbsp;(<span style="color:blue;">Max</span>&nbsp;<span style="color:#2b91af;">Integer</span>)&nbsp;<span style="color:#2b91af;">String</span></code> where the state type (<code><span style="color:blue;">Max</span>&nbsp;<span style="color:#2b91af;">Integer</span></code>) is used to keep track of the maximum value found while examining candidates. </p> <p> You could conceivably turn such a function around to instead look for the minimum by mapping the state to a <code>Min</code> value instead: </p> <p> <pre><span style="color:#2b91af;">pickMin</span>&nbsp;::&nbsp;<span style="color:blue;">State</span>&nbsp;(<span style="color:blue;">Min</span>&nbsp;<span style="color:#2b91af;">Integer</span>)&nbsp;<span style="color:#2b91af;">String</span> pickMin&nbsp;=&nbsp;ntToState&nbsp;$&nbsp;invmap&nbsp;(Min&nbsp;.&nbsp;getMax)&nbsp;(Max&nbsp;.&nbsp;getMin)&nbsp;$&nbsp;stateToNT&nbsp;pickMax</pre> </p> <p> You can use <code>getMax</code> to extract the underlying <code>Integer</code> from the <code><span style="color:blue;">Max</span>&nbsp;<span style="color:#2b91af;">Integer</span></code> and then <code>Min</code> to turn it into a <code><span style="color:blue;">Min</span>&nbsp;<span style="color:#2b91af;">Integer</span></code> value, and vice versa. <code><span style="color:blue;">Max</span>&nbsp;<span style="color:#2b91af;">Integer</span></code> and <code><span style="color:blue;">Min</span>&nbsp;<span style="color:#2b91af;">Integer</span></code> are isomorphic. </p> <p> In C#, you can implement a similar method. The code shown here extends the code shown in <a href="/2021/07/19/the-state-functor">The State functor</a>. I chose to call the method <code>SelectState</code> so as to not make things too confusing. The State functor already comes with a <code>Select</code> method that maps <code>T</code> to <code>T1</code> - that's the 'normal', covariant functor implementation. The new method is the invariant functor implementation that maps the state <code>S</code> to <code>S1</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;S1,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">SelectState</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">S</span>,&nbsp;<span style="color:#2b91af;">S1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IState&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">state</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;<span style="color:#1f377f;">sToS1</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S1,&nbsp;S&gt;&nbsp;<span style="color:#1f377f;">s1ToS</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;InvariantStateMapper&lt;T,&nbsp;S,&nbsp;S1&gt;(state,&nbsp;sToS1,&nbsp;s1ToS); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">InvariantStateMapper</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">S</span>,&nbsp;<span style="color:#2b91af;">S1</span>&gt;&nbsp;:&nbsp;IState&lt;S1,&nbsp;T&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;IState&lt;S,&nbsp;T&gt;&nbsp;state; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;sToS1; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;Func&lt;S1,&nbsp;S&gt;&nbsp;s1ToS; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">InvariantStateMapper</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IState&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">state</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;<span style="color:#1f377f;">sToS1</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S1,&nbsp;S&gt;&nbsp;<span style="color:#1f377f;">s1ToS</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.state&nbsp;=&nbsp;state; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.sToS1&nbsp;=&nbsp;sToS1; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.s1ToS&nbsp;=&nbsp;s1ToS; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;T,&nbsp;S1&gt;&nbsp;<span style="color:#74531f;">Run</span>(S1&nbsp;<span style="color:#1f377f;">s1</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;state.Run(s1ToS(s1)).Select(sToS1); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As usual when working in C# with interfaces instead of higher-order functions, <a href="/2019/12/16/zone-of-ceremony">there's some ceremony to be expected</a>. The only interesting line of code is the <code>Run</code> implementation. </p> <p> It starts by calling <code>s1ToS</code> in order to translate the <code>s1</code> parameter into an <code>S</code> value. This enables it to call <code>Run</code> on <code>state</code>. The result is a tuple with the type <code>Tuple&lt;T,&nbsp;S&gt;</code>. It's necessary to translate the <code>S</code> to <code>S1</code> with <code>sToS1</code>. You could do that by extracting the value from the tuple, mapping it, and returning a new tuple. Since a tuple gives rise to a functor (<a href="/2018/12/31/tuple-bifunctor">two, actually</a>) I instead used the <code>Select</code> method I'd already defined on it. </p> <p> Notice how similar the implementation is to the implementation of <a href="/2022/08/08/endomorphism-as-an-invariant-functor">the endomorphism invariant functor</a>. The only difference is that when translating back from <code>S</code> to <code>S1</code>, this happens inside a <code>Select</code> mapping. This is as predicted by the general implementation of invariant functors for natural transformations. </p> <p> In a future article, you'll see an example of <code>SelectState</code> in action. </p> <h3 id="5a148244488b4391be257181108072b3"> Other natural transformations <a href="#5a148244488b4391be257181108072b3" title="permalink">#</a> </h3> <p> As the <a href="/2022/07/18/natural-transformations">natural transformations</a> article outlines, there are infinitely many natural transformations. Each one gives rise to an invariant functor. </p> <p> It might be a good exercise to try to implement a few of them as invariant functors. If you want to do it in C#, you could, for example, start with the <em>safe head</em> natural transformation. </p> <p> If you want to stick to interfaces, you could define one like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">ISafeHead</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;Maybe&lt;T&gt;&nbsp;<span style="color:#74531f;">TryFirst</span>(IEnumerable&lt;T&gt;&nbsp;<span style="color:#1f377f;">ts</span>); }</pre> </p> <p> The exercise is now to define and implement a method like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;ISafeHead&lt;T1&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">T1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;ISafeHead&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">tToT1</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T1,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">t1ToT</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Implementation&nbsp;goes&nbsp;here...</span> }</pre> </p> <p> The implementation, once you get the handle of it, is entirely automatable. After all, in Haskell it's possible to do it once and for all, as shown above. </p> <h3 id="2c86c2f77e0a404da80858d855898843"> Conclusion <a href="#2c86c2f77e0a404da80858d855898843" title="permalink">#</a> </h3> <p> A natural transformation forms an invariant functor. This may not be the most exciting result ever, because invariant functors are limited in use. They only work when translating between types that are already isomorphic. Still, I did <a href="/2022/09/05/the-state-pattern-and-the-state-monad">find a use for this result</a> when I was working with the relationship between the State design pattern and the <a href="/2022/06/20/the-state-monad">State monad</a>. </p> <p> <strong>Next:</strong> <a href="/2022/12/26/functors-as-invariant-functors">Functors as invariant functors</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="9336107b26fd48a8971e617d8ebd5159"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Due to feedback that I've received, I have to face evidence that this article may be partially incorrect. While I've added that proviso at the top of the article, I've decided to use a comment to expand on the issue. </p> <p> On Twitter, the user <a href="https://twitter.com/Savlambda">@Savlambda</a> (<em>borar</em>) argued that my <code>newtype</code> isn't a natural transformation: </p> <blockquote> <p> "The newtype 'NT' in the article is not a natural transformation though. Quantification over 'a' is at the "wrong place": it is not allowed for a client module to instantiate the container element type of a natural transformation." </p> <footer><cite><a href="https://twitter.com/Savlambda/status/1564175654845030400">@Savlambda</a></cite></footer> </blockquote> <p> While I engaged with the tweet, I have to admit that it took me a while to understand the core of the criticism. Of course I'm not happy about being wrong, but initially I genuinely didn't understand what was the problem. On the other hand, it's not the first time @Savlambda has provided valuable insights, so I knew it'd behove me to pay attention. </p> <p> After a few tweets back and forth, @Savlambda finally supplied a counter-argument that I understood. </p> <blockquote> <p> "This is not being overly pedantic. Here is one practical implication:" </p> <footer><cite><a href="https://twitter.com/Savlambda/status/1564970422981890048">@Savlambda</a></cite></footer> </blockquote> <p> The practical implication shown in the tweet is a screen shot (in order to get around Twitter's character limitation), but I'll reproduce it as code here in order to <a href="https://meta.stackoverflow.com/q/285551/126014">not show images of code</a>. </p> <p> <pre><span style="color:blue;">type</span>&nbsp;<span style="color:#2b91af;">(~&gt;)</span>&nbsp;f&nbsp;g&nbsp;=&nbsp;forall&nbsp;a.&nbsp;f&nbsp;a&nbsp;-&gt;&nbsp;g&nbsp;a <span style="color:green;">--&nbsp;Use&nbsp;the&nbsp;natural&nbsp;transformation&nbsp;twice,&nbsp;for&nbsp;different&nbsp;types </span><span style="color:#2b91af;">convertLists</span>&nbsp;::&nbsp;([]&nbsp;~&gt;&nbsp;g)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(g&nbsp;<span style="color:#2b91af;">Int</span>,&nbsp;g&nbsp;<span style="color:#2b91af;">Bool</span>) convertLists&nbsp;nt&nbsp;=&nbsp;(nt&nbsp;[1,2],&nbsp;nt&nbsp;[True]) <span style="color:blue;">newtype</span>&nbsp;NT&nbsp;f&nbsp;g&nbsp;a&nbsp;=&nbsp;NT&nbsp;(f&nbsp;a&nbsp;-&gt;&nbsp;g&nbsp;a) <span style="color:green;">--&nbsp;Does&nbsp;not&nbsp;type&nbsp;check,&nbsp;does&nbsp;not&nbsp;work;&nbsp;not&nbsp;a&nbsp;natural&nbsp;transformation </span><span style="color:#2b91af;">convertLists2</span>&nbsp;::&nbsp;<span style="color:blue;">NT</span>&nbsp;[]&nbsp;g&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(g&nbsp;<span style="color:#2b91af;">Int</span>,&nbsp;g&nbsp;<span style="color:#2b91af;">Bool</span>) convertLists2&nbsp;(NT&nbsp;f)&nbsp;=&nbsp;(f&nbsp;[1,2],&nbsp;f&nbsp;[True])</pre> </p> <p> I've moved the code comments to prevent horizontal scrolling, but otherwise tried to stay faithful to @Savlambda's screen shot. </p> <p> This was the example that finally hit the nail on the head for me. A natural transformation is a mapping from one functor (<code>f</code>) to another functor (<code>g</code>). I knew that already, but hadn't realised the implications. In Haskell (and other languages with <a href="https://en.wikipedia.org/wiki/Parametric_polymorphism">parametric polymorphism</a>) a <code>Functor</code> is defined for all <code>a</code>. </p> <p> A natural transformation is a higher level of abstraction, mapping one functor to another. That mapping must be defined for all <code>a</code>, and it must be <em>reusable</em>. The second example provided by @Savlambda demonstrates that the function wrapped by <code>NT</code> isn't reusable for different contained types. </p> <p> If you try to compile that example, GHC emits this compiler error: </p> <p> <pre>* Couldn't match type `a' with `Int' `a' is a rigid type variable bound by the type signature for: convertLists2 :: forall (g :: * -&gt; *) a. NT [] g a -&gt; (g Int, g Bool) Expected type: g Int Actual type: g a * In the expression: f [1, 2] In the expression: (f [1, 2], f [True]) In an equation for `convertLists2': convertLists2 (NT f) = (f [1, 2], f [True])</pre> </p> <p> Even though it's never fun to be proven wrong, I want to thank @Savlambda for educating me. One reason I write blog posts like this one is that writing is a way to learn. By writing about topics like these, I educate myself. Occasionally, it turns out that I make a mistake, and <a href="/2018/12/03/set-is-not-a-functor">this isn't the first time that's happened</a>. I also wish to apologise if this article has now left any readers more confused. </p> <p> A remaining question is what practical implications this has? Only rarely do you need a programming construct like <code>convertLists2</code>. On the other hand, had I wanted a function with the type <code>NT [] g Int -&gt; (g Int, g Int)</code>, it would have type-checked just fine. </p> <p> I'm not claiming that this is generally useful either, but I actually wrote this article because I <em>did</em> have use for the result that <code>NT</code> (whatever it is) is an invariant functor. As far as I can tell, that result still holds. </p> <p> I could be wrong about that, too. If you think so, please leave a comment. </p> </div> <div class="comment-date">2022-09-04 7:53 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Can types replace validation? https://blog.ploeh.dk/2022/08/22/can-types-replace-validation 2022-08-22T05:57:00+00:00 Mark Seemann <div id="post"> <p> <em>With some examples in C#.</em> </p> <p> In a comment to my article on <a href="/2022/08/15/aspnet-validation-revisited">ASP.NET validation revisited</a> Maurice Johnson asks: </p> <blockquote> <p> "I was just wondering, is it possible to use the type system to do the validation instead ? </p> <p> "What I mean is, for example, to make all the ReservationDto's field a type with validation in the constructor (like a class name, a class email, and so on). Normally, when the framework will build ReservationDto, it will try to construct the fields using the type constructor, and if there is an explicit error thrown during the construction, the framework will send us back the error with the provided message. </p> <p> "Plus, I think types like "email", "name" and "at" are reusable. And I feel like we have more possibilities for validation with that way of doing than with the validation attributes. </p> <p> "What do you think ?" </p> <footer><cite><a href="/2022/08/15/aspnet-validation-revisited#f6deac18851d47c3b066f82a8be3847d">Maurice Johnson</a></cite></footer> </blockquote> <p> I started writing a response below the question, but it grew and grew so I decided to turn it into a separate article. I think the question is of general interest. </p> <h3 id="2492a444e49b49a09a4f7c03f38b29d5"> The halting problem <a href="#2492a444e49b49a09a4f7c03f38b29d5" title="permalink">#</a> </h3> <p> I'm all in favour of using the type system for encapsulation, but there are limits to what it can do. We know this because it follows from the <a href="https://en.wikipedia.org/wiki/Halting_problem">halting problem</a>. </p> <p> I'm basing my understanding of the halting problem on <a href="https://www.goodreads.com/review/show/1731926050">my reading</a> of <a href="/ref/annotated-turing">The Annotated Turing</a>. In short, given an arbitrary computer program in a Turing-complete language, there's no general algorithm that will determine whether or not the program will finish running. </p> <p> A compiler that performs type-checking is a program, but typical type systems aren't Turing-complete. It's possible to write type checkers that always finish, because the 'programming language' they are running on - the type system - isn't Turing-complete. </p> <p> Normal type systems (like C#'s) aren't Turing-complete. You expect the C# compiler to always arrive at a result (either compiled code or error) in finite time. As a counter-example, consider <a href="https://www.haskell.org/">Haskell</a>'s type system. By default it, too, isn't Turing-complete, but with sufficient language extensions, you <em>can</em> make it Turing-complete. Here's a fun example: <a href="https://aphyr.com/posts/342-typing-the-technical-interview">Typing the technical interview</a> by Kyle Kingsbury (Aphyr). When you make the type system Turing-complete, however, termination is no longer guaranteed. A program may now compile forever or, practically, until it times out or runs out of memory. That's what happened to me when I tried to compile Kyle Kingsbury's code example. </p> <p> How is this relevant? </p> <p> This matters because understanding that a normal type system is <em>not</em> Turing-complete means that there are truths it <em>can't</em> express. Thus, we shouldn't be surprised if we run into rules or policies that we can't express with the type system we're given. What exactly is inexpressible depends on the type system. There are policies you can express in Haskell that are impossible to express in C#, and so on. Let's stick with C#, though. Here are some examples of rules that are practically inexpressible: </p> <ul> <li>An integer must be positive.</li> <li>A string must be at most 100 characters long.</li> <li>A maximum value must be greater than a minimum value.</li> <li>A value must be a valid email address.</li> </ul> <p> <a href="https://www.hillelwayne.com/">Hillel Wayne</a> provides more compelling examples in the article <a href="https://buttondown.email/hillelwayne/archive/making-illegal-states-unrepresentable/">Making Illegal States Unrepresentable</a>. </p> <h3 id="34f9d2e5570842adbb79a20972f458a0"> Encapsulation <a href="#34f9d2e5570842adbb79a20972f458a0" title="permalink">#</a> </h3> <p> Depending on how many times you've been around the block, you may find the above list naive. You may, for example, say that it's possible to express that an integer is positive like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">struct</span>&nbsp;<span style="color:#2b91af;">NaturalNumber</span>&nbsp;:&nbsp;IEquatable&lt;NaturalNumber&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;i; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">NaturalNumber</span>(<span style="color:blue;">int</span>&nbsp;candidate) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(candidate&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentOutOfRangeException( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nameof(candidate), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;The&nbsp;value&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;(non-zero)&nbsp;number,&nbsp;but&nbsp;was:&nbsp;</span>{candidate}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.i&nbsp;=&nbsp;candidate; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Various&nbsp;other&nbsp;members&nbsp;follow...</span></pre> </p> <p> I like introducing wrapper types like this. To the inexperienced developer this may seem redundant, but using a wrapper like this has several advantages. For one, it makes preconditions explicit. Consider a constructor like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;Guid&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;DateTime&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;NaturalNumber&nbsp;quantity)</pre> </p> <p> What are the preconditions that you, as a client developer, has to fulfil before you can create a valid <code>Reservation</code> object? First, you must supply five arguments: <code>id</code>, <code>at</code>, <code>email</code>, <code>name</code>, and <code>quantity</code>. There is, however, more information than that. </p> <p> Consider, as an alternative, a constructor like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;Guid&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;DateTime&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity)</pre> </p> <p> This constructor requires you to supply the same five arguments. There is, however, less explicit information available. If that was the only available constructor, you might be wondering: <em>Can I pass zero as <code>quantity</code>? Can I pass <code>-1</code>?</em> </p> <p> When the only constructor available is the first of these two alternatives, you already have the answer: No, the <code>quantity</code> must be a natural number. </p> <p> Another advantage of creating wrapper types like <code>NaturalNumber</code> is that you centralise run-time checks in one place. Instead of sprinkling defensive code all over the code base, you have it in one place. Any code that receives a <code>NaturalNumber</code> object knows that the check has already been performed. </p> <p> There's a word for this: <a href="/encapsulation-and-solid">Encapsulation</a>. </p> <p> You gather a coherent set of invariants and collect it in a single type, making sure that the type always guarantees its invariants. Note that this is an important design technique in functional programming too. While you may not have to worry about state mutation preserving invariants, it's still important to guarantee that all values of a type are <em>valid</em>. </p> <h3 id="8c48a55a5db74686ab7a0c8fc40dd650"> Predicative and constructive data <a href="#8c48a55a5db74686ab7a0c8fc40dd650" title="permalink">#</a> </h3> <p> It's debatable whether the above <code>NaturalNumber</code> class <em>really</em> uses the type system to model what constitutes valid data. Since it relies on a run-time predicate, it falls in the category of types Hillel Wayne <a href="https://www.hillelwayne.com/post/constructive/">calls <em>predicative</em></a>. Such types are easy to create and compose well, but on the other hand fail to take full advantage of the type system. </p> <p> It's often worthwhile considering if a <em>constructive</em> design is possible and practical. In other words, is it possible to <a href="https://blog.janestreet.com/effective-ml-video/">make illegal states unrepresentable</a> (MISU)? </p> <p> What's wrong with <code>NaturalNumber</code>? Doesn't it do that? No, it doesn't, because this compiles: </p> <p> <pre><span style="color:blue;">new</span>&nbsp;NaturalNumber(-1)</pre> </p> <p> Surely it <em>will</em> fail at run time, but it compiles. Thus, it's <em>representable</em>. </p> <p> <a href="/2011/04/29/Feedbackmechanismsandtradeoffs">The compiler gives you feedback faster than tests</a>. Considering MISU is worthwhile. </p> <p> Can we model natural numbers in a constructive way? Yes, with <a href="https://en.wikipedia.org/wiki/Peano_axioms">Peano numbers</a>. This is even <a href="/2018/05/28/church-encoded-natural-numbers">possible in C#</a>, but I wouldn't consider it practical. On the other hand, while it's possible to represent any natural number, there is <em>no way</em> to express -1 as a Peano number. </p> <p> As Hillel Wayne describes, constructive data types are much harder and requires a considerable measure of creativity. Often, a constructive model can seem impossible until you get a good idea. </p> <blockquote> <p> "a list can only be of even length. Most languages will not be able to express such a thing in a reasonable way in the data type." </p> <footer><cite><a href="https://note89.github.io/state-of-emergency/">Nils Eriksson</a></cite></footer> </blockquote> <p> Such a requirement may look difficult until inspiration hits. Then one day you may realise that it'd be as simple as a list of pairs (two-tuples). In Haskell, it could be as simple as this: </p> <p> <pre>newtype EvenList a = EvenList [(a,a)] deriving (Eq, Show)</pre> </p> <p> With such a constructive data model, lists of uneven length are unrepresentable. This is a simple example of the kind of creative thinking you may need to engage in with constructive data modelling. </p> <p> If you feel the need to object that Haskell isn't 'most languages', then here's the same idea expressed in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">EvenCollection</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;:&nbsp;IEnumerable&lt;T&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;IEnumerable&lt;Tuple&lt;T,&nbsp;T&gt;&gt;&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">EvenCollection</span>(IEnumerable&lt;Tuple&lt;T,&nbsp;T&gt;&gt;&nbsp;<span style="color:#1f377f;">values</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.values&nbsp;=&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IEnumerator&lt;T&gt;&nbsp;<span style="color:#74531f;">GetEnumerator</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">foreach</span>&nbsp;(var&nbsp;<span style="color:#1f377f;">x</span>&nbsp;<span style="color:#8f08c4;">in</span>&nbsp;values) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;x.Item1; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;x.Item2; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;IEnumerator&nbsp;IEnumerable.<span style="color:#74531f;">GetEnumerator</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;GetEnumerator(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> You can create such a list like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">list</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;EvenCollection&lt;<span style="color:blue;">string</span>&gt;(<span style="color:blue;">new</span>[] { &nbsp;&nbsp;&nbsp;&nbsp;Tuple.Create(<span style="color:#a31515;">&quot;foo&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;Tuple.Create(<span style="color:#a31515;">&quot;baz&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;qux&quot;</span>) });</pre> </p> <p> On the other hand, this doesn't compile: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">list</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;EvenCollection&lt;<span style="color:blue;">string</span>&gt;(<span style="color:blue;">new</span>[] { &nbsp;&nbsp;&nbsp;&nbsp;Tuple.Create(<span style="color:#a31515;">&quot;foo&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;Tuple.Create(<span style="color:#a31515;">&quot;baz&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;qux&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;quux&quot;</span>) });</pre> </p> <p> Despite this digression, the point remains: Constructive data modelling may be impossible, unimagined, or impractical. </p> <p> Often, in languages like C# we resort to predicative data modelling. That's also what I did in the article <a href="/2022/08/15/aspnet-validation-revisited">ASP.NET validation revisited</a>. </p> <h3 id="830f3f7a663b459994cc585ef9604d11"> Validation as functions <a href="#830f3f7a663b459994cc585ef9604d11" title="permalink">#</a> </h3> <p> That was a long rambling detour inspired by a simple question: Is it possible to use types instead of validation? </p> <p> In order to address that question, it's only proper to explicitly state assumptions and definitions. What's the definition of <em>validation?</em> </p> <p> I'm not aware of a ubiquitous definition. While I could draw from <a href="https://en.wikipedia.org/wiki/Data_validation">the Wikipedia article on the topic</a>, at the time of writing it doesn't cite any sources when it sets out to define what it is. So I may as well paraphrase. It seems fair, though, to consider the stem of the word: <em>Valid</em>. </p> <p> Validation is the process of examining input to determine whether or not it's valid. I consider this a (mostly) self-contained operation: Given the data, is it well-formed and according to specification? If you have to query a database before making a decision, you're not validating the input. In that case, you're applying a business rule. As a rule of thumb I expect validations to be <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. </p> <p> Validation, then, seems to imply a process. Before you execute the process, you don't know if data is valid. After executing the process, you do know. </p> <p> Data types, whether predicative like <code>NaturalNumber</code> or constructive like <code>EvenCollection&lt;T&gt;</code>, aren't processes or functions. They are results. </p> <p> <img src="/content/binary/validation-as-a-function-from-data-to-type.png" alt="An arrow labelled 'validation' pointing from a document to the left labelled 'Data' to a box to the right labelled 'Type'."> </p> <p> Sometimes an algorithm can use a type to <em>infer</em> the validation function. This is common in statically typed languages, from C# over <a href="https://fsharp.org/">F#</a> to Haskell (which are the languages with which I'm most familiar). </p> <h3 id="4cc324c1919e45958542bb82c2ca73d4"> Data Transfer Object as a validation DSL <a href="#4cc324c1919e45958542bb82c2ca73d4" title="permalink">#</a> </h3> <p> In a way you can think of the type system as a <a href="https://en.wikipedia.org/wiki/Domain-specific_language">domain-specific language</a> (DSL) for defining validation functions. It's not perfectly suited for that task, but often good enough that many developers reach for it. </p> <p> Consider the <code>ReservationDto</code> class from the <a href="/2022/08/15/aspnet-validation-revisited">ASP.NET validation revisited</a> article where I eventually gave up on it: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;LinkDto[]?&nbsp;Links&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Guid?&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;[Required,&nbsp;NotNull] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;DateTime?&nbsp;At&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;[Required,&nbsp;NotNull] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Email&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Name&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;[NaturalNumber] &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Quantity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> It actually tries to do what Maurice Johnson suggests. Particularly, it defines <code>At</code> as a <code>DateTime?</code> value. </p> <p> <pre>&gt; <span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">json</span>&nbsp;=&nbsp;<span style="color:#a31515;">&quot;{&nbsp;\&quot;At\&quot;:&nbsp;\&quot;2022-10-11T19:30\&quot;,&nbsp;\&quot;Email\&quot;:&nbsp;\&quot;z@example.com\&quot;,&nbsp;\&quot;Quantity\&quot;:&nbsp;1}&quot;</span>; &gt; JsonSerializer.Deserialize&lt;ReservationDto&gt;(json) ReservationDto { At=[11.10.2022 19:30:00], Email="z@example.com", Id=null, Name=null, Quantity=1 }</pre> </p> <p> A JSON deserializer like this one uses run-time reflection to examine the type in question and then maps the incoming data onto an instance. Many XML deserializers work the same way. </p> <p> What happens if you supply malformed input? </p> <p> <pre>&gt; <span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">json</span>&nbsp;=&nbsp;<span style="color:#a31515;">&quot;{&nbsp;\&quot;At\&quot;:&nbsp;\&quot;foo\&quot;,&nbsp;\&quot;Email\&quot;:&nbsp;\&quot;z@example.com\&quot;,&nbsp;\&quot;Quantity\&quot;:&nbsp;1}&quot;</span>; &gt; JsonSerializer.Deserialize&lt;ReservationDto&gt;(json) <span style="color:red">System.Text.Json.JsonException:↩ The JSON value could not be converted to System.Nullable`1[System.DateTime].↩ Path: $.At | LineNumber: 0 | BytePositionInLine: 26.↩ [...]</span></pre> </p> <p> (I've wrapped the result over multiple lines for readability. The <code>↩</code> symbol indicates where I've wrapped the text. I've also omitted a stack trace, indicated by <code>[...]</code>. I'll do that repeatedly throughout this article.) </p> <p> What happens if we try to define <code>ReservationDto.Quantity</code> with <code>NaturalNumber</code>? </p> <p> <pre>&gt; var json = "{ \"At\": \"2022-10-11T19:30\", \"Email\": \"z@example.com\", \"Quantity\": 1}"; &gt; JsonSerializer.Deserialize&lt;ReservationDto&gt;(json) <span style="color:red">System.Text.Json.JsonException:↩ The JSON value could not be converted to NaturalNumber.↩ Path: $.Quantity | LineNumber: 0 | BytePositionInLine: 67.↩ [...]</span></pre> </p> <p> While <a href="https://docs.microsoft.com/dotnet/api/system.text.json.jsonserializer">JsonSerializer</a> is a sophisticated piece of software, it's not so sophisticated that it can automatically map <code>1</code> to a <code>NaturalNumber</code> value. </p> <p> I'm sure that you can configure the behaviour with one or more <a href="https://docs.microsoft.com/dotnet/api/system.text.json.serialization.jsonconverter">JsonConverter</a> objects, but this is exactly the kind of framework <a href="https://en.wikipedia.org/wiki/Whac-A-Mole">Whack-a-mole</a> that I consider costly. It also suggests a wider problem. </p> <h3 id="bdad63c97ad44f1199f4f06e2b2e3fff"> Error handling <a href="#bdad63c97ad44f1199f4f06e2b2e3fff" title="permalink">#</a> </h3> <p> What happens if input to a validation function is malformed? You may want to report the errors to the caller, and you may want to report all errors in one go. Consider the user experience if you don't: A user types in a big form and submits it. The system informs him or her that there's an error in the third field. Okay, correct the error and submit again. Now there's an error in the fifth field, and so on. </p> <p> It's often better to return all errors as one collection. </p> <p> The problem is that type-based validation doesn't <em>compose</em> well. What do I mean by that? </p> <p> It's fairly clear that if you take a <em>simple</em> (i.e. non-complex) type like <code>NaturalNumber</code>, if you fail to initialize a value it's because the input is at fault: </p> <p> <pre>&gt; <span style="color:blue;">new</span>&nbsp;NaturalNumber(-1) <span style="color:red">System.ArgumentOutOfRangeException: The value must be a positive (non-zero) number, but was: -1.↩ (Parameter 'candidate') + NaturalNumber..ctor(int)</span></pre> </p> <p> The problem is that for complex types (i.e. types made from other types), exceptions short-circuit. As soon as one exception is thrown, further data validation stops. The <a href="/2022/08/15/aspnet-validation-revisited">ASP.NET validation revisited</a> article shows examples of that particular problem. </p> <p> This happens when validation functions have no composable way to communicate errors. When throwing exceptions, you can return an exception message, but exceptions short-circuit rather than compose. The same is true for the <a href="/2022/05/09/an-either-monad">Either monad</a>: It short-circuits. Once you're on <a href="https://fsharpforfunandprofit.com/posts/recipe-part2/">the failure track</a> you stay there and no further processing takes place. Errors don't compose. </p> <h3 id="7a85a7339fc24eda8e922ba12fdb3d89"> Monoidal versus applicative validation <a href="#7a85a7339fc24eda8e922ba12fdb3d89" title="permalink">#</a> </h3> <p> The naive take on validation is to answer the question: <em>Is that data valid or invalid?</em> Notice the binary nature of the question. It's either-or. </p> <p> This is true for both predicative data and constructive data. </p> <p> For constructive data, the question is: Is a candidate value representable? For example, can you represent <em>-1</em> as a Peano number? The answer is either yes or no; true or false. </p> <p> This is even clearer for predicative data, which is defined by a <em>predicate</em>. (Here's another <a href="/2021/09/09/the-specification-contravariant-functor">example of a natural number specification</a>.) A predicate is a function that returns a Boolean value: True or false. </p> <p> It's possible to compose Boolean values. The composition that we need in this case is Boolean <em>and</em>, which is also known as the <em>all</em> <a href="/2017/10/06/monoids">monoid</a>: If all values are <em>true</em>, the composed value is <em>true</em>; if just one value is <em>false</em>, the composed value is <em>false</em>. </p> <p> The problem is that during composition, we lose information. While a single <em>false</em> value causes the entire aggregated value to be <em>false</em>, we don't know why. And we don't know if there was only a single <em>false</em> value, or if there were more than one. Boolean <em>all</em> short-circuits on the first <em>false</em> value it encounters, and stops processing subsequent predicates. </p> <p> In logic, that's all you need, but in data validation you often want to know <em>what's wrong with the data</em>. </p> <p> Fortunately, this is <a href="/2020/12/14/validation-a-solved-problem">a solved problem</a>. Use <a href="/2018/11/05/applicative-validation">applicative validation</a>, an example of which I supplied in the article <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">An applicative reservation validation example in C#</a>. </p> <p> This changes focus on validation. No longer is validation a <em>true/false</em> question. Validation is a function from less-structured data to more-structured data. <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">Parse, don't validate</a>. </p> <h3 id="96d5b6c4df424f79b9eec8b186a2b3de"> Conclusion <a href="#96d5b6c4df424f79b9eec8b186a2b3de" title="permalink">#</a> </h3> <p> Can types replace validation? </p> <p> In some cases they can, but I think that the general answer is <em>no</em>. Granted, this answer is partially based on capabilities of current deserialisers. <a href="https://docs.microsoft.com/dotnet/api/system.text.json.jsonserializer.deserialize">JsonSerializer.Deserialize</a> short-circuits on the first error it encounters, and the same does <a href="https://hackage.haskell.org/package/aeson/docs/Data-Aeson.html">aeson</a>'s <a href="https://hackage.haskell.org/package/aeson/docs/Data-Aeson.html#v:eitherDecode">eitherDecode</a>. </p> <p> While that's the current state of affairs, it may not have to stay like that forever. One might be able to derive an applicative parser from a desired destination type, but I haven't seen that done yet. </p> <p> It sounds like a worthwhile research project. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="4e4498fcf878443e96fad490569a8e4f"> <div class="comment-author"><a href="https://www.lloydatkinson.net">Lloyd Atkinson</a></div> <div class="comment-content"> <p> This slightly reminds me of <a href="https://github.com/colinhacks/zod">Zod</a> which is described as "TypeScript-first schema validation with static type inference". </p> <p> The library automatically infers a type that matches the validation - in a way it blurs this line between types and validation by making them become one. </p> <p> Of course, once you have that infered type there is nothing stopping you using it without the library, but that's something code reviews could catch. It's quite interesting though. </p> <pre> <code> import { z } from 'zod'; const User = z.object({ username: z.string(), age: z.number().positive({ message: 'Your age must be positive!', }), }); User.parse({ username: 'Ludwig', age: -1 }); // extract the inferred type type User = z.infer<typeof User>; // { username: string, age: number } </code> </pre> </div> <div class="comment-date">2022-08-28 00:53 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. ASP.NET validation revisited https://blog.ploeh.dk/2022/08/15/aspnet-validation-revisited 2022-08-15T05:48:00+00:00 Mark Seemann <div id="post"> <p> <em>Is the built-in validation framework better than applicative validation?</em> </p> <p> I recently published an article called <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">An applicative reservation validation example in C#</a> in which I describe how to use the universal abstractions of <a href="/2018/10/01/applicative-functors">applicative functors</a> and <a href="/2017/11/27/semigroups">semigroups</a> to implement reusable, composable validation. </p> <p> One reader reaction made me stop and think: </p> <blockquote> <p> "An exercise on how to reject 90% of the framework's existing services (*Validation) only to re implement them more poorly, by renouncing standardization, interoperability and globalization all for the glory of FP." </p> <footer><cite><a href="https://twitter.com/PopCatalin/status/1551478523981881349">PopCatalin</a></cite></footer> </blockquote> <p> (At the time of posting, the <a href="https://twitter.com/PopCatalin">PopCatalin Twitter account</a>'s display name was <em>Prime minister of truth™ カタリンポップ🇺🇦</em>, which I find unhelpful. The linked <a href="https://github.com/popcatalin81">GitHub account</a> locates the user in <a href="https://en.wikipedia.org/wiki/Cluj-Napoca">Cluj-Napoca</a>, a city I've <a href="/schedule">repeatedly visited for conferences</a> - the last time as recent as June 2022. I wouldn't be surprised if we've interacted, but if so, I'm sorry to say that I can't connect these accounts with one of the many wonderful people I've met there. In general, I'm getting a strong sarcastic vibe from that account, and I'm not sure whether or not to take <em>Pronouns kucf/fof</em> seriously. As the possibly clueless 51-year white male that I am, I will proceed with good intentions and to the best of my abilities.) </p> <p> That reply is an important reminder that I should once in a while check my assumptions. I'm aware that the ASP.NET framework comes with validation features, but I many years ago dismissed them because I found them inadequate. Perhaps, in the meantime, these built-in services have improved to the point that they are to be preferred over <a href="/2018/11/05/applicative-validation">applicative validation</a>. </p> <p> I decided to attempt to refactor the code to take advantage of the built-in ASP.NET validation to be able to compare the two approaches. This article is an experience report. </p> <h3 id="d4b2e6bdae494d0397866f683ddd64e9"> Requirements <a href="#d4b2e6bdae494d0397866f683ddd64e9" title="permalink">#</a> </h3> <p> In order to compare the two approaches, the ASP.NET-based validation should support the same validation features as the applicative validation example: </p> <ul> <li>The <code>At</code> property is required and should be a valid date and time. If it isn't, the validation message should report the problem and the offending input.</li> <li>The <code>Email</code> property should be required. If it's missing, the validation message should state so.</li> <li>The <code>Quantity</code> property is required and should be a natural number. If it isn't, the validation message should report the problem and the offending input.</li> </ul> <p> The <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">previous article</a> includes an interaction example that I'll repeat here for convenience: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 Content-Type: application/json {&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;large&quot;</span>,&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>,&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;-1&nbsp;} HTTP/1.1 400 Bad Request Invalid date or time: large. Email address is missing. Quantity must be a positive integer, but was: -1.</pre> </p> <p> ASP.NET validation formats the errors differently, as you'll see later in this article. That's not much of a concern, though: <a href="/2014/12/23/exception-messages-are-for-programmers">Error <em>messages</em> are for other developers</a>. They don't really have to be machine-readable or have a strict shape (as opposed to error <em>types</em>, which should be machine-readable). </p> <p> Reporting the offending values, as in <em>"Quantity must be a positive integer, but was: -1."</em> is part of the requirements. A REST API can make no assumptions about its clients. Perhaps one client is an unattended batch job that only logs errors. Logging offending values may be helpful to maintenance developers of such a batch job. </p> <h3 id="74d95aed3a5a40eb8f32ace05b0410a0"> Framework API <a href="#74d95aed3a5a40eb8f32ace05b0410a0" title="permalink">#</a> </h3> <p> The first observation to make about the ASP.NET validation API is that it's specific to ASP.NET. It's not a general-purpose API that you can use for other purposes. </p> <p> If, instead, you need to validate input to a console application, a background message handler, a batch job, or a desktop or phone app, you can't use that API. </p> <p> Perhaps each of these styles of software come with their own validation APIs, but even if so, that's a different API you'll have to learn. And in cases where there's no built-in validation API, then what do you do? </p> <p> The beauty and practicality of applicative validation is that it's <em>universal</em>. Since it's based on mathematical foundations, it's not tied to a particular framework, platform, or language. These concepts exist independently of technology. Once you understand the concepts, they're always there for you. </p> <p> The code example from the previous article, as well as here, build upon the code base that accompanies <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>. An example code base has to be written in <em>some</em> language, and I chose C# because I'm more familiar with it than I am with <a href="https://www.java.com/">Java</a>, <a href="https://isocpp.org/">C++</a>, or <a href="https://www.typescriptlang.org/">TypeScript</a>. While I wanted the code base to be realistic, I tried hard to include only coding techniques and patterns that you could use in more than one language. </p> <p> As I wrote the book, I ran into many interesting problems and solutions that were specific to C# and ASP.NET. While I found them too specific to include in the book, I wrote <a href="/2021/06/14/new-book-code-that-fits-in-your-head">a series of blog posts</a> about them. This article is now becoming one of those. </p> <p> The point about the previous article on <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">applicative reservation validation in C#</a> was to demonstrate how the general technique works. Not specifically in ASP.NET, or even C#, but in general. </p> <p> It just so happens that this example is situated in a context where an alternative solution presents itself. This is not always the case. Sometimes you have to solve this problem yourself, and when this happens, it's useful to know that <a href="/2020/12/14/validation-a-solved-problem">validation is a solved problem</a>. Even so, while a universal solution exists, it doesn't follow that the universal solution is the best. Perhaps there are specialised solutions that are better, each within their constrained contexts. </p> <p> Perhaps ASP.NET validation is an example of that. </p> <h3 id="0b80f4aa36954378be9278761b2f27ba"> Email validation <a href="#0b80f4aa36954378be9278761b2f27ba" title="permalink">#</a> </h3> <p> The following is a report on my experience refactoring validation to use the built-in ASP.NET validation API. </p> <p> I decided to start with the <code>Email</code> property, since the only requirement is that this value should be present. That seemed like an easy way to get started. </p> <p> I added the <a href="https://docs.microsoft.com/dotnet/api/system.componentmodel.dataannotations.requiredattribute">[Required]</a> attribute to the <code>ReservationDto</code> class' <code>Email</code> property. Since this code base also uses <a href="https://docs.microsoft.com/dotnet/csharp/nullable-references">nullable reference types</a>, it was necessary to also annotate the property with the <a href="https://docs.microsoft.com/dotnet/api/system.diagnostics.codeanalysis.notnullattribute">[NotNull]</a> attribute: </p> <p> <pre>[Required,&nbsp;NotNull] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Email&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> That's not too difficult, and seems to be working satisfactorily: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 > content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21&nbsp;19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;1 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|552ab5ff-494e1d1a9d4c6355.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;]&nbsp;} }</pre> </p> <p> As discussed above, the response body is formatted differently than in the applicative validation example, but I consider that inconsequential for the reasons I gave. </p> <p> So far, so good. </p> <h3 id="a10d6da09d064a96bc0022e9657b62e5"> Quantity validation <a href="#a10d6da09d064a96bc0022e9657b62e5" title="permalink">#</a> </h3> <p> The next property I decided to migrate was <code>Quantity</code>. This must be a natural number; that is, an integer greater than zero. </p> <p> Disappointingly, no such built-in validation attribute seems to exist. One <a href="https://stackoverflow.com/a/7419330/126014">highly voted Stack Overflow answer</a> suggested using the <a href="https://docs.microsoft.com/dotnet/api/system.componentmodel.dataannotations.rangeattribute">[Range]</a> attribute, so I tried that: </p> <p> <pre>[Range(1,&nbsp;<span style="color:blue;">int</span>.MaxValue,&nbsp;ErrorMessage&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;natural&nbsp;number.&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Quantity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> As a <em>declarative</em> approach to validation goes, I don't think this is off to a good start. I like declarative programming, but I'd prefer to be able to declare that <code>Quantity</code> must be a <em>natural number</em>, rather than in the range of <code>1</code> and <code>int.MaxValue</code>. </p> <p> Does it work, though? </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21&nbsp;19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|d9a6be38-4be82ede7c525913.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Quantity&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;natural&nbsp;number.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> While it does capture the intent that <code>Quantity</code> must be one or greater, it fails to echo back the offending value. </p> <p> In order to address that concern, I tried reading the documentation to find a way forward. Instead I found this: </p> <blockquote> <p> "Internally, the attributes call <a href="https://docs.microsoft.com/en-us/dotnet/api/system.string.format">String.Format</a> with a placeholder for the field name and sometimes additional placeholders. [...]" </p> <p> "To find out which parameters are passed to <code>String.Format</code> for a particular attribute's error message, see the <a href="https://github.com/dotnet/runtime/tree/main/src/libraries/System.ComponentModel.Annotations/src/System/ComponentModel/DataAnnotations">DataAnnotations source code</a>." </p> <footer><cite><a href="https://docs.microsoft.com/aspnet/core/mvc/models/validation">ASP.NET validation documentation</a></cite></footer> </blockquote> <p> Really?! </p> <p> If you have to read implementation code, <a href="/encapsulation-and-solid">encapsulation</a> is broken. </p> <p> Hardly impressed, I nonetheless found <a href="https://github.com/dotnet/runtime/blob/main/src/libraries/System.ComponentModel.Annotations/src/System/ComponentModel/DataAnnotations/RangeAttribute.cs">the RangeAttribute source code</a>. Alas, it only passes the property <code>name</code>, <code>Minimum</code>, and <code>Maximum</code> to <code>string.Format</code>, but not the offending value: </p> <p> <pre>return string.Format(CultureInfo.CurrentCulture, ErrorMessageString, name, Minimum, Maximum);</pre> </p> <p> This looked like a dead end, but at least it's possible to extend the ASP.NET validation API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">NaturalNumberAttribute</span>&nbsp;:&nbsp;ValidationAttribute { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">protected</span>&nbsp;<span style="color:blue;">override</span>&nbsp;ValidationResult&nbsp;IsValid( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">object</span>&nbsp;value, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ValidationContext&nbsp;validationContext) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(validationContext&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(validationContext)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;i&nbsp;=&nbsp;value&nbsp;<span style="color:blue;">as</span>&nbsp;<span style="color:blue;">int</span>?; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(i.HasValue&nbsp;&amp;&amp;&nbsp;0&nbsp;&lt;&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ValidationResult.Success; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ValidationResult( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;</span>{validationContext.MemberName}<span style="color:#a31515;">&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;</span>{value}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Adding this <code>NaturalNumberAttribute</code> class enabled me to change the annotation of the <code>Quantity</code> property: </p> <p> <pre>[NaturalNumber] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Quantity&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> This seems to get the job done: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21&nbsp;19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|bb45b60d-4bd255194871157d.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Quantity&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;0.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> The <code>[NaturalNumber]</code> attribute now correctly reports the offending value together with a useful error message. </p> <p> Compare, however, the above <code>NaturalNumberAttribute</code> class to the <code>TryParseQuantity</code> function, repeated here for convenience: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;Validated&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#74531f;">TryParseQuantity</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Fail&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;</span>{Quantity}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Succeed&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;(Quantity); }</pre> </p> <p> <code>TryParseQuantity</code> is shorter and has half the <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a> of <code>NaturalNumberAttribute</code>. In isolation, at least, I'd prefer the shorter, simpler alternative. </p> <h3 id="3c90c9ded0af42fd9017593c9a387e97"> Date and time validation <a href="#3c90c9ded0af42fd9017593c9a387e97" title="permalink">#</a> </h3> <p> Remaining is validation of the <code>At</code> property. As a first step, I converted the property to a <code>DateTime</code> value and added attributes: </p> <p> <pre>[Required,&nbsp;NotNull] <span style="color:blue;">public</span>&nbsp;DateTime?&nbsp;At&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;}</pre> </p> <p> I'd been a little apprehensive doing that, fearing that it'd break a lot of code (particularly tests), but that turned out not to be the case. In fact, it actually simplified a few of the tests. </p> <p> On the other hand, this doesn't really work as required: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21&nbsp;19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|1e1d600e-4098fb36635642f6.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;dto&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;dto&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;$.at&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;JSON&nbsp;value&nbsp;could&nbsp;not&nbsp;be&nbsp;converted&nbsp;to&nbsp;System.Nullable`1[System.DateTime].↩ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Path:&nbsp;$.at&nbsp;|&nbsp;LineNumber:&nbsp;0&nbsp;|&nbsp;BytePositionInLine:&nbsp;26.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> (I've wrapped the last error message over two lines for readability. The <code>↩</code> symbol indicates where I've wrapped the text.) </p> <p> There are several problems with this response. First, in addition to complaining about the missing <code>at</code> property, it should also have reported that there are problems with the <code>Quantity</code> and that the <code>Email</code> property is missing. Instead, the response implies that the <code>dto</code> field is missing. That's likely confusing to client developers, because <code>dto</code> is an implementation detail; it's the name of the C# parameter of the method that handles the request. Client developers can't and shouldn't know this. Instead, it looks as though the REST API somehow failed to receive the JSON document that the client posted. </p> <p> Second, the error message exposes other implementation details, here that the <code>at</code> field has the type <code>System.Nullable`1[System.DateTime]</code>. This is, at best, irrelevant. At worst, it could be a security issue, because it reveals to a would-be attacker that the system is implemented on .NET. </p> <p> Third, the framework rejects what looks like a perfectly good date and time: <code>2022-11-21 19:00</code>. This is a breaking change, since the API used to accept such values. </p> <p> What's wrong with <code>2022-11-21 19:00</code>? It's not a valid <a href="https://en.wikipedia.org/wiki/ISO_8601">ISO 8601</a> string. According to the ISO 8601 standard, the date and time must be separated by <code>T</code>: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21T19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|1e1d600f-4098fb36635642f6.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Quantity&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;0.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> Posting a valid ISO 8601 string does, indeed, enable the client to proceed - only to receive a new set of error messages. After I converted <code>At</code> to <code>DateTime?</code>, the ASP.NET validation framework fails to collect and report all errors. Instead it stops if it can't parse the <code>At</code> property. It doesn't report any other errors that might also be present. </p> <p> That is exactly the requirement that applicative validation so elegantly solves. </p> <h3 id="e9986e66f63c476ca529788afaa863e4"> Tolerant Reader <a href="#e9986e66f63c476ca529788afaa863e4" title="permalink">#</a> </h3> <p> While it's true that <code>2022-11-21 19:00</code> isn't valid ISO 8601, it's unambiguous. According to <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a> an API should be a <a href="https://martinfowler.com/bliki/TolerantReader.html">Tolerant Reader</a>. It's not. </p> <p> This problem, however, is solvable. First, add the Tolerant Reader: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">DateTimeConverter</span>&nbsp;:&nbsp;JsonConverter&lt;DateTime&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;DateTime&nbsp;Read( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">ref</span>&nbsp;Utf8JsonReader&nbsp;reader, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Type&nbsp;typeToConvert, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;JsonSerializerOptions&nbsp;options) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;DateTime.Parse( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reader.GetString(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;CultureInfo.InvariantCulture); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Write( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utf8JsonWriter&nbsp;writer, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;DateTime&nbsp;value, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;JsonSerializerOptions&nbsp;options) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(writer&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(writer)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;writer.WriteStringValue(value.ToString(<span style="color:#a31515;">&quot;s&quot;</span>)); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Then add it to the JSON serialiser's <a href="https://docs.microsoft.com/dotnet/api/system.text.json.jsonserializeroptions.converters">Converters</a>: </p> <p> <pre>opts.JsonSerializerOptions.Converters.Add(<span style="color:blue;">new</span>&nbsp;DateTimeConverter());</pre> </p> <p> This, at least, addresses the Tolerant Reader concern: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-11-21&nbsp;19:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|11576943-400dafd4b489c282.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Quantity&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;0.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> The API now accepts the slightly malformed <code>at</code> field. It also correctly handles if the field is entirely missing: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|11576944-400dafd4b489c282.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;At&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;At&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Email&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;Email&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Quantity&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;0.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> On the other hand, it <em>still</em> doesn't gracefully handle the case when the <code>at</code> field is unrecoverably malformed: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 content-type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;foo&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;0 } HTTP/1.1 400 Bad Request Content-Type: application/problem+json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;type&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;https://tools.ietf.org/html/rfc7231#section-6.5.1&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;title&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;One&nbsp;or&nbsp;more&nbsp;validation&nbsp;errors&nbsp;occurred.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;status&quot;</span>:&nbsp;400, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;traceId&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;|11576945-400dafd4b489c282.&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;errors&quot;</span>:&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;supplied&nbsp;value&nbsp;is&nbsp;invalid.&quot;</span>&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;dto&quot;</span>:&nbsp;[&nbsp;<span style="color:#a31515;">&quot;The&nbsp;dto&nbsp;field&nbsp;is&nbsp;required.&quot;</span>&nbsp;] &nbsp;&nbsp;} }</pre> </p> <p> <code>The supplied value is invalid.</code> and <code>The dto field is required.</code>? That's not really helpful. And what happened to <code>The Email field is required.</code> and <code>Quantity must be a positive integer, but was: 0.</code>? </p> <p> If there's a way to address this problem, I don't know how. I've tried adding another custom attribute, similar to the above <code>NaturalNumberAttribute</code> class, but that doesn't solve it - probably because the model binder (that deserialises the JSON document to a <code>ReservationDto</code> instance) runs before the validation. </p> <p> Perhaps there's a way to address this problem with yet another class that derives from a base class, but I think that I've already played enough <a href="https://en.wikipedia.org/wiki/Whac-A-Mole">Whack-a-mole</a> to arrive at a conclusion. </p> <h3 id="fc3e2119ed1c4a7a8cff0c0992a8b071"> Conclusion <a href="#fc3e2119ed1c4a7a8cff0c0992a8b071" title="permalink">#</a> </h3> <p> Your context may differ from mine, so the conclusion that I arrive at may not apply in your situation. For example, I'm given to understand that one benefit that the ASP.NET validation framework provides is that when used with ASP.NET MVC (instead of as a Web API), (some of) the validation logic can also run in <a href="https://www.javascript.com/">JavaScript</a> in browsers. This, ostensibly, reduces code duplication. </p> <blockquote> <p> "Yet in the case of validation, a Declarative model is far superior to a FP one. The declarative model allows various environments to implement validation as they need it (IE: Client side validation) while the FP one is strictly limited to the environment executing the code." </p> <footer><cite><a href="https://twitter.com/PopCatalin/status/1551478926005911553">PopCatalin</a></cite></footer> </blockquote> <p> On the other hand, using the ASP.NET validation framework requires more code, and more complex code, than with applicative validation. It's a particular set of APIs that you have to learn, and that knowledge doesn't transfer to other frameworks, platforms, or languages. </p> <p> Apart from client-side validation, I fail to see how applicative validation <em>"re implement[s validation] more poorly, by renouncing standardization, interoperability and globalization"</em>. </p> <p> I'm not aware that there's any <em>standard</em> for validation as such, so I think that @PopCatalin has the 'standard' ASP.NET validation API in mind. If so, I consider applicative validation a much more standardised solution than a specialised API. </p> <p> If by <em>interoperability</em> @PopCatalin means the transfer of logic from server side to client side, then it's true that the applicative validation I showed in the previous article runs exclusively on the server. I wonder, however, how much of such custom validation as <code>NaturalNumberAttribute</code> automatically transfers to the client side. </p> <p> When it comes to globalisation, I fail to see how applicative validation is less globalisable than the ASP.NET validation framework. One could easily replace the hard-coded strings in my examples with resource strings. </p> <p> It would seem, again, that <a href="https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule">any sufficiently complicated custom validation framework contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of applicative validation</a>. </p> <blockquote> <p> "I must admit I really liked the declarative OOP model using annotations when I first saw it in Java (EJB3.0, almost 20yrs ago) until I saw FP way of doing things. FP way is so much simpler and powerful, because it's just function composition, nothing more, no hidden "magic"." </p> <footer><cite><a href="https://twitter.com/witoldsz/status/1552429555503493120">Witold Szczerba</a></cite></footer> </blockquote> <p> I still find myself in the same camp as Witold Szczerba. It's easy to get started using validation annotations, but it doesn't follow that it's simpler or better in the long run. As <a href="https://en.wikipedia.org/wiki/Rich_Hickey">Rich Hickey</a> points out in <a href="https://www.infoq.com/presentations/Simple-Made-Easy/">Simple Made Easy</a>, <em>simple</em> and <em>easy</em> isn't the same. If I have to maintain code, I'll usually choose the simple solution over the easy solution. That means choosing applicative validation over a framework-specific validation API. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="f6deac18851d47c3b066f82a8be3847d"> <div class="comment-author">Maurice Johnson</div> <div class="comment-content"> <p> Hello Mark. I was just wondering, is it possible to use the type system to do the validation instead ? </p> <p> What I mean is, for example, to make all the ReservationDto's field a type with validation in the constructor (like a class name, a class email, and so on). Normally, when the framework will build ReservationDto, it will try to construct the fields using the type constructor, and if there is an explicit error thrown during the construction, the framework will send us back the error with the provided message. </p> <p> Plus, I think types like "email", "name" and "at" are reusable. And I feel like we have more possibilities for validation with that way of doing than with the validation attributes. </p> <p> What do you think ? </p> <p> Regards. </p> </div> <div class="comment-date">2022-08-16 08:30 UTC</div> </div> <div class="comment" id="89a9ef6d57f645d7a1e848aaf20b8b72"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Maurice, thank you for writing. I started writing a reply, but it grew, so I'm going to turn it into a blog post. I'll post an update here once I've published it, but expect it to take a few weeks. </p> </div> <div class="comment-date">2022-08-18 7:50 UTC</div> </div> <div class="comment" id="e932eb2fd0804a049314872d5fe6a358"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> I've published the article: <a href="/2022/08/22/can-types-replace-validation">Can types replace validation?</a>. </p> </div> <div class="comment-date">2022-08-22 6:00 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Endomorphism as an invariant functor https://blog.ploeh.dk/2022/08/08/endomorphism-as-an-invariant-functor 2022-08-08T04:43:00+00:00 Mark Seemann <div id="post"> <p> <em>An article (also) for object-oriented programmers.</em> </p> <p> This article is part of <a href="/2022/08/01/invariant-functors">a series of articles about invariant functors</a>. An invariant functor is a <a href="/2018/03/22/functors">functor</a> that is neither covariant nor contravariant. See the series introduction for more details. </p> <p> An <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a> is a function where the return type is the same as the input type. </p> <p> In <a href="https://www.haskell.org/">Haskell</a> we denote an endomorphism as <code>a -&gt; a</code>, in <a href="http://fsharp.org/">F#</a> we have to add an apostrophe: <code>'a -&gt; 'a</code>, while in C# such a function corresponds to the delegate <code>Func&lt;T, T&gt;</code> or, alternatively, to a method that has the same return type as input type. </p> <p> In Haskell you can treat an endomorphism like a <a href="/2017/10/06/monoids">monoid</a> by wrapping it in a <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> called <a href="https://hackage.haskell.org/package/base/docs/Data-Monoid.html#t:Endo">Endo</a>: <code>Endo a</code>. In C#, we <a href="/2021/05/31/from-state-tennis-to-endomorphism">might model it as an interface</a> called <code><span style="color:#2b91af;">IEndomorphism</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code>. </p> <p> That looks enough like a functor that you might wonder if it is one, but it turns out that it's neither co- nor contravariant. You can deduce this with positional variance analysis (which I've learned from <a href="https://thinkingwithtypes.com/">Thinking with Types</a>). In short, this is because <code>T</code> appears as both input and output - it's neither co- nor contravariant, but rather <em>invariant</em>. </p> <h3 id="39fa705060f143aba4c18dd2c8ff7f2d"> Explicit endomorphism interface in C# <a href="#39fa705060f143aba4c18dd2c8ff7f2d" title="permalink">#</a> </h3> <p> Consider an <code><span style="color:#2b91af;">IEndomorphism</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> interface in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IEndomorphism</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;<span style="color:#74531f;">Run</span>(T&nbsp;<span style="color:#1f377f;">x</span>); }</pre> </p> <p> I've borrowed this interface from the article <a href="/2021/05/31/from-state-tennis-to-endomorphism">From State tennis to endomorphism</a>. In that article I explain that I only introduce this interface for educational reasons. I don't expect you to use something like this in production code bases. On the other hand, everything that applies to <code><span style="color:#2b91af;">IEndomorphism</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> also applies to 'naked' functions, as you'll see later in the article. </p> <p> As outlined in the introduction, you can make a container an invariant functor by implementing a non-standard version of <code>Select</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IEndomorphism&lt;B&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">A</span>,&nbsp;<span style="color:#2b91af;">B</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IEndomorphism&lt;A&gt;&nbsp;<span style="color:#1f377f;">endomorphism</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;A,&nbsp;B&gt;&nbsp;<span style="color:#1f377f;">aToB</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">bToA</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;SelectEndomorphism&lt;A,&nbsp;B&gt;(endomorphism,&nbsp;aToB,&nbsp;bToA); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SelectEndomorphism</span>&lt;<span style="color:#2b91af;">A</span>,&nbsp;<span style="color:#2b91af;">B</span>&gt;&nbsp;:&nbsp;IEndomorphism&lt;B&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;IEndomorphism&lt;A&gt;&nbsp;endomorphism; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;Func&lt;A,&nbsp;B&gt;&nbsp;aToB; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;bToA; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">SelectEndomorphism</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;A&gt;&nbsp;<span style="color:#1f377f;">endomorphism</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;A,&nbsp;B&gt;&nbsp;<span style="color:#1f377f;">aToB</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">bToA</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.endomorphism&nbsp;=&nbsp;endomorphism; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.aToB&nbsp;=&nbsp;aToB; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.bToA&nbsp;=&nbsp;bToA; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;B&nbsp;<span style="color:#74531f;">Run</span>(B&nbsp;<span style="color:#1f377f;">x</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;aToB(endomorphism.Run(bToA(x))); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Since the <code>Select</code> method has to return an <code>IEndomorphism&lt;B&gt;</code> implementation, one option is to use a private, nested class. Most of this is <a href="/2019/12/16/zone-of-ceremony">ceremony</a> required because it's working with interfaces. The interesting part is the nested class' <code>Run</code> implementation. </p> <p> In order to translate an <code>IEndomorphism&lt;A&gt;</code> to an <code>IEndomorphism&lt;B&gt;</code>, the <code>Run</code> method first uses <code>bToA</code> to translate <code>x</code> to an <code>A</code> value. Once it has the <code>A</code> value, it can <code>Run</code> the <code>endomorphism</code>, which returns another <code>A</code> value. Finally, the method can use <code>aToB</code> to convert the returned <code>A</code> value to a <code>B</code> value that it can return. </p> <p> Here's a simple example. Imagine that you have an endomorphism like this one: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Incrementer</span>&nbsp;:&nbsp;IEndomorphism&lt;BigInteger&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;BigInteger&nbsp;<span style="color:#74531f;">Run</span>(BigInteger&nbsp;<span style="color:#1f377f;">x</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;x&nbsp;+&nbsp;1; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This one simply increments a <a href="https://docs.microsoft.com/dotnet/api/system.numerics.biginteger">BigInteger</a> value. Since <code>BigInteger</code> is isomorphic to a byte array, it's possible to transform this <code>BigInteger</code> endomorphism to a byte array endomorphism: </p> <p> <pre>[Theory] [InlineData(<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[0],&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;{&nbsp;1&nbsp;})] [InlineData(<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;{&nbsp;1&nbsp;},&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;{&nbsp;2&nbsp;})] [InlineData(<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;{&nbsp;255,&nbsp;0&nbsp;},&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;{&nbsp;0,&nbsp;1&nbsp;})] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">InvariantSelection</span>(<span style="color:blue;">byte</span>[]&nbsp;<span style="color:#1f377f;">bs</span>,&nbsp;<span style="color:blue;">byte</span>[]&nbsp;<span style="color:#1f377f;">expected</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;BigInteger&gt;&nbsp;<span style="color:#1f377f;">source</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Incrementer(); &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;<span style="color:blue;">byte</span>[]&gt;&nbsp;<span style="color:#1f377f;">destination</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;source.Select(<span style="color:#1f377f;">bi</span>&nbsp;=&gt;&nbsp;bi.ToByteArray(),&nbsp;<span style="color:#1f377f;">arr</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;BigInteger(arr)); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;destination.Run(bs)); }</pre> </p> <p> You can convert a <code>BigInteger</code> to a byte array with the <code>ToByteArray</code> method, and convert such a byte array back to a <code>BigInteger</code> using one of its constructor overloads. Since this is possible, the example test can convert this <code>IEndomorphism&lt;BigInteger&gt;</code> to an <code>IEndomorphism&lt;<span style="color:blue;">byte</span>[]&gt;</code> and later <code>Run</code> it. </p> <h3 id="f9afa75c0b014df68d29ddf1244f329f"> Mapping functions in F# <a href="#f9afa75c0b014df68d29ddf1244f329f" title="permalink">#</a> </h3> <p> You don't need an interface in order to turn an endomorphism into an invariant functor. An endomorphism is just a function that has the same input and output type. In C# such a function has the type <code>Func&lt;T, T&gt;</code>, while in F# it's written <code>&#39;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;&#39;a</code>. </p> <p> You could write an F# module that defines an <code>invmap</code> function, which would be equivalent to the above <code>Select</code> method: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;Endo&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;b)&nbsp;-&gt;&nbsp;(&#39;b&nbsp;-&gt;&nbsp;&#39;a)&nbsp;-&gt;&nbsp;(&#39;a&nbsp;-&gt;&nbsp;&#39;a)&nbsp;-&gt;&nbsp;(&#39;b&nbsp;-&gt;&nbsp;&#39;b)</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;invmap&nbsp;(f&nbsp;:&nbsp;&#39;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;&#39;b)&nbsp;(g&nbsp;:&nbsp;&#39;b&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;&#39;a)&nbsp;(h&nbsp;:&nbsp;&#39;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;&#39;a)&nbsp;=&nbsp;g&nbsp;&gt;&gt;&nbsp;h&nbsp;&gt;&gt;&nbsp;f</pre> </p> <p> Since this function doesn't have to deal with the ceremony of interfaces, the implementation is simple function composition: For any input, first apply it to the <code>g</code> function, then apply the output to the <code>h</code> function, and again apply the output of that function to the <code>f</code> function. </p> <p> Here's the same example as above: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;increment&nbsp;(bi&nbsp;:&nbsp;BigInteger)&nbsp;=&nbsp;bi&nbsp;+&nbsp;BigInteger.One <span style="color:green;">//&nbsp;byte&nbsp;[]&nbsp;-&gt;&nbsp;byte&nbsp;[]</span> <span style="color:blue;">let</span>&nbsp;bArrInc&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Endo.invmap&nbsp;(<span style="color:blue;">fun</span>&nbsp;(bi&nbsp;:&nbsp;BigInteger)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;bi.ToByteArray&nbsp;())&nbsp;BigInteger&nbsp;increment</pre> </p> <p> Here's a simple sanity check of the <code>bArrInc</code> function executed in F# Interactive: </p> <p> <pre>&gt; let bArr = bArrInc [| 255uy; 255uy; 0uy |];; val bArr : byte [] = [|0uy; 0uy; 1uy|]</pre> </p> <p> If you are wondering about that particular output value, I'll refer you to <a href="https://docs.microsoft.com/dotnet/api/system.numerics.biginteger">the BigInteger documentation</a>. </p> <h3 id="76c32ad7688b41fc9e083a51584e4a6a"> Function composition <a href="#76c32ad7688b41fc9e083a51584e4a6a" title="permalink">#</a> </h3> <p> The F# implementation of <code>invmap</code> (<code>g&nbsp;&gt;&gt;&nbsp;h&nbsp;&gt;&gt;&nbsp;f</code>) makes it apparent that an endomorphism is an invariant functor via function composition. In F#, though, that fact almost disappears in all the type declaration ceremony. In the Haskell instance from the <a href="https://hackage.haskell.org/package/invariant">invariant</a> package it's even clearer: </p> <p> <pre><span style="color:blue;">instance</span>&nbsp;<span style="color:blue;">Invariant</span>&nbsp;<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;invmap&nbsp;f&nbsp;g&nbsp;(Endo&nbsp;h)&nbsp;=&nbsp;Endo&nbsp;(f&nbsp;.&nbsp;h&nbsp;.&nbsp;g)</pre> </p> <p> Perhaps a diagram is helpful: </p> <p> <img src="/content/binary/invariant-map-diagram.png" alt="Arrow diagram showing the mapping from an endomorphism in a to an endomorphism in b."> </p> <p> If you have a function <code>h</code> from the type <code>a</code> to <code>a</code> and you need a function <code>b -&gt; b</code>, you can produce it by putting <code>g</code> in front of <code>h</code>, and <code>f</code> after. That's also what the above C# implementation does. In F#, you can express such a composition as <code>g&nbsp;&gt;&gt;&nbsp;h&nbsp;&gt;&gt;&nbsp;f</code>, which seems natural to most westerners, since it goes from left to right. In Haskell, most expressions are instead expressed from right to left, so it becomes: <code>f&nbsp;.&nbsp;h&nbsp;.&nbsp;g</code>. In any case, the result is the desired function that takes a <code>b</code> value as input and returns a <code>b</code> value as output. That composed function is indicated by a dashed arrow in the above diagram. </p> <h3 id="3934e941d51f47c698d8b803b7adb97c"> Identity law <a href="#3934e941d51f47c698d8b803b7adb97c" title="permalink">#</a> </h3> <p> Contrary to my usual habit, I'm going to <em>prove</em> that both invariant functor laws hold for this implementation. I'll use equational reasoning with <a href="https://bartoszmilewski.com/2015/01/20/functors/">the notation that Bartosz Milewski uses</a>. Here's the proof that the <code>invmap</code> instance obeys the identity law: </p> <p> <pre>&nbsp;&nbsp;invmap&nbsp;id&nbsp;id&nbsp;(Endo&nbsp;h) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;Endo&nbsp;(id&nbsp;.&nbsp;h&nbsp;.&nbsp;id) =&nbsp;{&nbsp;eta&nbsp;expansion&nbsp;} &nbsp;&nbsp;Endo&nbsp;(\x&nbsp;-&gt;&nbsp;(id&nbsp;.&nbsp;h&nbsp;.&nbsp;id)&nbsp;x) =&nbsp;{&nbsp;defintion&nbsp;of&nbsp;composition&nbsp;(.)&nbsp;} &nbsp;&nbsp;Endo&nbsp;(\x&nbsp;-&gt;&nbsp;id&nbsp;(h&nbsp;(id&nbsp;x))) =&nbsp;{&nbsp;defintion&nbsp;of&nbsp;id&nbsp;} &nbsp;&nbsp;Endo&nbsp;(\x&nbsp;-&gt;&nbsp;h&nbsp;x) =&nbsp;{&nbsp;eta&nbsp;reduction&nbsp;} &nbsp;&nbsp;Endo&nbsp;h =&nbsp;{&nbsp;definition&nbsp;of&nbsp;id&nbsp;} &nbsp;&nbsp;id&nbsp;(Endo&nbsp;h)</pre> </p> <p> While I'm not going to comment further on that, I can show you what the identity law looks like in C#: </p> <p> <pre>[Theory] [InlineData(0)] [InlineData(1)] [InlineData(9)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">IdentityLaw</span>(<span style="color:blue;">long</span>&nbsp;<span style="color:#1f377f;">l</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;BigInteger&gt;&nbsp;<span style="color:#1f377f;">e</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Incrementer(); &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;BigInteger&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;e.Select(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x,&nbsp;<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(e.Run(l),&nbsp;actual.Run(l)); }</pre> </p> <p> In C#, you typically write the identity function (<code>id</code> in F# and Haskell) as the lambda expression <code><span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x</code>, since the identity function isn't 'built in' for C# like it is for F# and Haskell. (You can define it yourself, but it's not as <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a>.) </p> <h3 id="e47f375aafe441949e057787a5d35178"> Composition law <a href="#e47f375aafe441949e057787a5d35178" title="permalink">#</a> </h3> <p> As with the identity law, I'll start by suggesting a proof for the composition law for the Haskell instance: </p> <p> <pre>&nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;$&nbsp;invmap&nbsp;f1&nbsp;f1&#39;&nbsp;(Endo&nbsp;h) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;$&nbsp;Endo&nbsp;(f1&nbsp;.&nbsp;h&nbsp;.&nbsp;f1&#39;) =&nbsp;{&nbsp;defintion&nbsp;of&nbsp;($)&nbsp;} &nbsp;&nbsp;invmap&nbsp;f2&nbsp;f2&#39;&nbsp;(Endo&nbsp;(f1&nbsp;.&nbsp;h&nbsp;.&nbsp;f1&#39;)) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;Endo&nbsp;(f2&nbsp;.&nbsp;(f1&nbsp;.&nbsp;h&nbsp;.&nbsp;f1&#39;)&nbsp;.&nbsp;f2&#39;) =&nbsp;{&nbsp;associativity&nbsp;of&nbsp;composition&nbsp;(.)&nbsp;} &nbsp;&nbsp;Endo&nbsp;((f2&nbsp;.&nbsp;f1)&nbsp;.&nbsp;h&nbsp;.&nbsp;(f1&#39;&nbsp;.&nbsp;f2&#39;)) =&nbsp;{&nbsp;definition&nbsp;of&nbsp;invmap&nbsp;} &nbsp;&nbsp;invmap&nbsp;(f2&nbsp;.&nbsp;f1)&nbsp;(f1&#39;&nbsp;.&nbsp;f2&#39;)&nbsp;(Endo&nbsp;h)</pre> </p> <p> As above, a C# example may also help. First, assume that you have some endomorphism like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SecondIncrementer</span>&nbsp;:&nbsp;IEndomorphism&lt;TimeSpan&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;TimeSpan&nbsp;<span style="color:#74531f;">Run</span>(TimeSpan&nbsp;<span style="color:#1f377f;">x</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;x&nbsp;+&nbsp;TimeSpan.FromSeconds(1); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> A test then demonstrates the composition law in action: </p> <p> <pre>[Theory] [InlineData(-3)] [InlineData(0)] [InlineData(11)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">CompositionLaw</span>(<span style="color:blue;">long</span>&nbsp;<span style="color:#1f377f;">x</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;TimeSpan&gt;&nbsp;<span style="color:#1f377f;">i</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;SecondIncrementer(); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;TimeSpan,&nbsp;<span style="color:blue;">long</span>&gt;&nbsp;<span style="color:#1f377f;">f1</span>&nbsp;=&nbsp;<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;ts.Ticks; &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">long</span>,&nbsp;TimeSpan&gt;&nbsp;<span style="color:#1f377f;">f1p</span>&nbsp;=&nbsp;<span style="color:#1f377f;">l</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(l); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">long</span>,&nbsp;IntPtr&gt;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&nbsp;<span style="color:#1f377f;">l</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;IntPtr(l); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;IntPtr,&nbsp;<span style="color:blue;">long</span>&gt;&nbsp;<span style="color:#1f377f;">f2p</span>&nbsp;=&nbsp;<span style="color:#1f377f;">ip</span>&nbsp;=&gt;&nbsp;ip.ToInt64(); &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;IntPtr&gt;&nbsp;<span style="color:#1f377f;">left</span>&nbsp;=&nbsp;i.Select(f1,&nbsp;f1p).Select(f2,&nbsp;f2p); &nbsp;&nbsp;&nbsp;&nbsp;IEndomorphism&lt;IntPtr&gt;&nbsp;<span style="color:#1f377f;">right</span>&nbsp;=&nbsp;i.Select(<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;f2(f1(ts)),&nbsp;<span style="color:#1f377f;">ip</span>&nbsp;=&gt;&nbsp;f1p(f2p(ip))); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(left.Run(<span style="color:blue;">new</span>&nbsp;IntPtr(x)),&nbsp;right.Run(<span style="color:blue;">new</span>&nbsp;IntPtr(x))); }</pre> </p> <p> Don't try to make any sense of this. As outlined in the introductory article, in order to use an invariant functor, you're going to need an isomorphism. In order to demonstrate the composition law, you need <em>three</em> types that are isomorphic. Since you can convert back and forth between <code>TimeSpan</code> and <code>IntPtr</code> via <code>long</code>, this requirement is formally fulfilled. It doesn't make any sense to add a second to a value and then turn it into a function that changes a pointer. It sounds more like a security problem waiting to happen... Don't try this at home, kids. </p> <h3 id="ac1e8d3a75b54f9691c616755750ff77"> Conclusion <a href="#ac1e8d3a75b54f9691c616755750ff77" title="permalink">#</a> </h3> <p> Since an endomorphism can be modelled as a 'generic type', it may look like a candidate for a functor or <a href="/2021/09/02/contravariant-functors">contravariant functor</a>, but alas, neither is possible. The best we can get (apart from <a href="/2017/11/13/endomorphism-monoid">a monoid instance</a>) is an invariant functor. </p> <p> The invariant functor instance for an endomorphism turns out to be simple function composition. That's not how all invariant functors, work, though. </p> <p> <strong>Next:</strong> <a href="/2022/08/29/natural-transformations-as-invariant-functors">Natural transformations as invariant functors</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Invariant functors https://blog.ploeh.dk/2022/08/01/invariant-functors 2022-08-01T05:49:00+00:00 Mark Seemann <div id="post"> <p> <em>Containers that support mapping isomorphic values.</em> </p> <p> This article series is part of <a href="/2018/03/19/functors-applicatives-and-friends">a larger series of articles about functors, applicatives, and other mappable containers</a>. So far, you've seen examples of both co- and <a href="/2021/09/02/contravariant-functors">contravariant functors</a>, including <a href="/2021/11/01/profunctors">profunctors</a>. You've also seen a few examples of <a href="/2020/10/19/monomorphic-functors">monomorphic functors</a> - mappable containers where there's no variance at all. </p> <p> What happens, on the other hand, if you have a container of (generic) values, but it's neither co- nor contravariant? An <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a> is an example - it's neither co- nor contravariant. You'll see a treatment of that in a later article. </p> <p> Even if neither co- nor contravariant mappings exists for a container, all may not be lost. It may still be an <em>invariant functor</em>. </p> <h3 id="40d4c4a6deed4af593b3cf002563f085"> Invariance <a href="#40d4c4a6deed4af593b3cf002563f085" title="permalink">#</a> </h3> <p> Consider a <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> <code>f</code> (for <em>functor</em>). Depending on its variance, we call it <em>covariant</em>, <em>contravariant</em>, or <em>invariant</em>: </p> <ul> <li><em>Covariance</em> means that any function <code>a -&gt; b</code> can be lifted into a function <code>f a -&gt; f b</code>.</li> <li><em>Contravariance</em> means that any function <code>a -&gt; b</code> can be lifted into a function <code>f b -&gt; f a</code>.</li> <li><em>Invariance</em> means that in general, no function <code>a -&gt; b</code> can be lifted into a function over <code>f a</code>.</li> </ul> <p> <em>In general</em>, that is. A limited escape hatch exists: </p> <blockquote> <p> "an invariant type [...] allows you to map from <code>a</code> to <code>b</code> if and only if <code>a</code> and <code>b</code> are isomorphic. In a very real sense, this isn't an interesting property - an isomorphism between <code>a</code> and <code>b</code> means they're already the same thing to begin with." </p> <footer><cite>Sandy Maguire, <a href="https://thinkingwithtypes.com/">Thinking with Types</a></cite></footer> </blockquote> <p> In <a href="https://www.haskell.org/">Haskell</a> we may define an invariant functor (AKA <a href="http://comonad.com/reader/2008/rotten-bananas/">exponential functor</a>) as in the <a href="https://hackage.haskell.org/package/invariant">invariant</a> package: </p> <p> <pre><span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Invariant</span>&nbsp;f&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;<span style="color:#2b91af;">invmap</span>&nbsp;::&nbsp;(a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;b)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;(b&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;a)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;f&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;f&nbsp;b</pre> </p> <p> This means that an <em>invariant functor</em> <code>f</code> is a container of values where a translation from <code>f a</code> to <code>f b</code> exists if it's possible to translate contained values both ways: From <code>a</code> to <code>b</code>, and from <code>b</code> to <code>a</code>. Callers of the <code>invmap</code> function must supply translations that go both ways. </p> <h3 id="36e114d8871c46c9addb2e38936a2872"> Invariant functor in C# <a href="#36e114d8871c46c9addb2e38936a2872" title="permalink">#</a> </h3> <p> It's possible to translate the concept to a language like C#. Since C# doesn't have higher-kinded types, we have to examine the abstraction as a set of patterns or templates. For <a href="/2018/03/22/functors">functors</a> and <a href="/2022/03/28/monads">monads</a>, the C# compiler can perform 'compile-time duck typing' to recognise these motifs to enable query syntax. For more advanced or exotic universal abstractions, such as <a href="/2018/12/24/bifunctors">bifunctors</a>, <a href="/2021/11/01/profunctors">profunctors</a>, or invariant functors, we have to use a concrete container type as a stand-in for 'any' functor. In this article, I'll call it <code><span style="color:#2b91af;">Invariant</span>&lt;<span style="color:#2b91af;">A</span>&gt;</code>. </p> <p> Such a generic class must have a mapping function that corresponds to the above <code>invmap</code>. In C# it has this signature: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Invariant&lt;B&gt;&nbsp;<span style="color:#74531f;">InvMap</span>&lt;<span style="color:#2b91af;">B</span>&gt;(Func&lt;A,&nbsp;B&gt;&nbsp;<span style="color:#1f377f;">aToB</span>,&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">bToA</span>)</pre> </p> <p> In this example, <code>InvMap</code> is an instance method on <code><span style="color:#2b91af;">Invariant</span>&lt;<span style="color:#2b91af;">A</span>&gt;</code>. You may use it like this: </p> <p> <pre>Invariant&lt;<span style="color:blue;">long</span>&gt;&nbsp;<span style="color:#1f377f;">il</span>&nbsp;=&nbsp;createInvariant(); Invariant&lt;TimeSpan&gt;&nbsp;<span style="color:#1f377f;">its</span>&nbsp;=&nbsp;il.InvMap(<span style="color:#1f377f;">l</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(l),&nbsp;<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;ts.Ticks);</pre> </p> <p> It's not that easy to find good examples of truly isomorphic primitives, but <a href="https://docs.microsoft.com/dotnet/api/system.timespan">TimeSpan</a> is just a useful wrapper of <code>long</code>, so it's possible to translate back and forth without loss of information. To create a <code>TimeSpan</code> from a <code>long</code>, you can use the suitable constructor overload. To get a <code>long</code> from a <code>TimeSpan</code>, you can read the <a href="https://docs.microsoft.com/dotnet/api/system.timespan.ticks">Ticks</a> property. </p> <p> Perhaps you find a method name like <code>InvMap</code> non-idiomatic in C#. Perhaps a more <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> name might be <code>Select</code>? That's not a problem: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Invariant&lt;B&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">B</span>&gt;(Func&lt;A,&nbsp;B&gt;&nbsp;<span style="color:#1f377f;">aToB</span>,&nbsp;Func&lt;B,&nbsp;A&gt;&nbsp;<span style="color:#1f377f;">bToA</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;InvMap(aToB,&nbsp;bToA); }</pre> </p> <p> In that case, usage would look like this: </p> <p> <pre>Invariant&lt;<span style="color:blue;">long</span>&gt;&nbsp;<span style="color:#1f377f;">il</span>&nbsp;=&nbsp;createInvariant(); Invariant&lt;TimeSpan&gt;&nbsp;<span style="color:#1f377f;">its</span>&nbsp;=&nbsp;il.Select(<span style="color:#1f377f;">l</span>&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(l),&nbsp;<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;ts.Ticks);</pre> </p> <p> In this article, I'll use <code>Select</code> in order to be consistent with C# naming conventions. Using that name, however, will not make query syntax light up. While the name is fine, the signature is not one that the C# compiler will recognise as enabling special syntax. The name does, however, suggest a kinship with a normal functor, where the mapping in C# is called <code>Select</code>. </p> <h3 id="4a5c0a55258d4e398931d2880df059fb"> Laws <a href="#4a5c0a55258d4e398931d2880df059fb" title="permalink">#</a> </h3> <p> As is usual with these kinds of universal abstractions, an invariant functor must satisfy a few laws. </p> <p> The first one we might call the <em>identity law:</em> </p> <p> <pre>invmap id id = id</pre> </p> <p> This law corresponds to the first functor law. When performing the mapping operation, if the values in the invariant functor are mapped to themselves, the result will be an unmodified functor. </p> <p> In C# such a mapping might look like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;i.Select(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x,&nbsp;<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x);</pre> </p> <p> The law then says that <code>actual</code> should be equal to <code>i</code>. </p> <p> The second law we might call the <em>composition law:</em> </p> <p> <pre>invmap f2 f2' . invmap f1 f1' = invmap (f2 . f1) (f1' . f2')</pre> </p> <p> Granted, this looks more complicated, but also directly corresponds to the second functor law. If two sequential mapping operations are performed one after the other, the result should be the same as a single mapping operation where the functions are composed. </p> <p> In C# the left-hand side might look like this: </p> <p> <pre>Invariant&lt;IntPtr&gt;&nbsp;<span style="color:#1f377f;">left</span>&nbsp;=&nbsp;i.Select(f1,&nbsp;f1p).Select(f2,&nbsp;f2p);</pre> </p> <p> In C# you can't name functions or variables with a quotation mark (like the Haskell code's <code>f1'</code> and <code>f2'</code>), so instead I named them <code>f1p</code> and <code>f2p</code> (with a <em>p</em> for <em>prime</em>). </p> <p> Likewise, the right-hand side might look like this: </p> <p> <pre>Invariant&lt;IntPtr&gt;&nbsp;<span style="color:#1f377f;">right</span>&nbsp;=&nbsp;i.Select(<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;f2(f1(ts)),&nbsp;<span style="color:#1f377f;">ip</span>&nbsp;=&gt;&nbsp;f1p(f2p(ip)));</pre> </p> <p> The composition law says that the <code>left</code> and <code>right</code> values must be equal. </p> <p> You'll see some more detailed examples in later articles. </p> <h3 id="14c58dc57f9744dfbc955015d03b871e"> Examples <a href="#14c58dc57f9744dfbc955015d03b871e" title="permalink">#</a> </h3> <p> This is all too abstract to seem useful in itself, so example are warranted. You'll be able to peruse examples of specific invariant functors in separate articles: </p> <ul> <li><a href="/2022/08/08/endomorphism-as-an-invariant-functor">Endomorphism as an invariant functor</a></li> <li><a href="/2022/08/29/natural-transformations-as-invariant-functors">Natural transformations as invariant functors</a></li> <li><a href="/2022/12/26/functors-as-invariant-functors">Functors as invariant functors</a></li> <li>Contravariant functors as invariant functors</li> </ul> <p> As two of the titles suggest, all functors are also invariant functors, and the same goes for contravariant functors: </p> <p> <img src="/content/binary/invariant-functor-set-diagram.png" alt="Set diagram. The biggest set labelled invariant functos contains two other sets labelled functors and invariant functors."> </p> <p> To be honest, invariant functors are exotic, and you are unlikely to need them in all but the rarest cases. Still, I <em>did</em> run into a scenario where I needed an invariant functor instance to be able to perform <a href="/2022/09/05/the-state-pattern-and-the-state-monad">a particular sleight of hand</a>. The rabbit holes we sometimes fall into... </p> <h3 id="ffa0e597f09c40e7bcd1cabe808aa0d5"> Conclusion <a href="#ffa0e597f09c40e7bcd1cabe808aa0d5" title="permalink">#</a> </h3> <p> Invariant functors form a set that contains both co- and contravariant functors, as well as some data structures that are neither. This is an exotic abstraction that you may never need. It did, however, get me out of a bind at one time. </p> <strong>Next:</strong> <a href="/2022/08/08/endomorphism-as-an-invariant-functor">Endomorphism as an invariant functor</a>. </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="faae4a8cbe294be8ac7596488d675483"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> For <a href="/2018/03/22/functors">functors</a> and <a href="/2022/03/28/monads">monads</a>, the C# compiler can perform 'compile-time duck typing' to recognise these motifs to enable query syntax. </blockquote> <p> Instead of 'compile-time duck typing', I think a better phrase to describe this is <a href="https://en.wikipedia.org/wiki/Structural_type_system">structural typing</a>. </p> </div> <div class="comment-date">2022-09-17 16:20 UTC</div> </div> <div class="comment" id="98be00b677484b11a51d8e583c25baa5"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I wasn't aware of the term <em>structural typing</em>, so thank you for the link. I've now read that Wikipedia article, but all I know is what's there. Based on it, though, it looks as though <a href="https://fsharp.org">F#</a>'s <a href="https://learn.microsoft.com/dotnet/fsharp/language-reference/generics/statically-resolved-type-parameters">Statically Resolved Type Parameters</a> are another example of structural typing, in addition to the <a href="https://ocaml.org/">OCaml</a> example given in the article. </p> <p> IIRC, <a href="https://www.purescript.org/">PureScript</a>'s <em>row polymorphism</em> may be another example, but it's been many years since <a href="/2017/06/06/fractal-trees-with-purescript">I played with it</a>. In other words, I could be mistaken. </p> <p> Based on the Wikipedia article, it looks as though structural typing is more concerned with polymorphism, but granted, so is duck typing. Given how wrong 'compile-time duck typing' actually is in the above context, 'structural typing' seems more correct. </p> <p> I may still stick with 'compile-time duck typing' as a loose metaphor, though, because most people know what duck typing is, whereas I'm not sure as many people know of structural typing. The purpose of the metaphor is, after all, to be helpful. </p> </div> <div class="comment-date">2022-09-19 14:46 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An applicative reservation validation example in C# https://blog.ploeh.dk/2022/07/25/an-applicative-reservation-validation-example-in-c 2022-07-25T06:56:00+00:00 Mark Seemann <div id="post"> <p> <em>How to return all relevant error messages in a composable way.</em> </p> <p> I've previously suggested that <a href="/2020/12/14/validation-a-solved-problem">I consider validation a solved problem</a>. I still do, until someone disproves me with a counterexample. Here's a fairly straightforward <a href="/2018/11/05/applicative-validation">applicative validation</a> example in C#. </p> <p> After corresponding and speaking with readers of <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a> I've learned that some readers have objections to the following lines of code: </p> <p> <pre>Reservation?&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;=&nbsp;dto.Validate(id); <span style="color:#8f08c4;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;BadRequestResult();</pre> </p> <p> This code snippet demonstrates how to <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">parse, not validate</a>, an incoming Data Transfer Object (DTO). This code base uses C#'s <a href="https://docs.microsoft.com/dotnet/csharp/nullable-references">nullable reference types</a> feature to distinguish between null and non-null objects. Other languages (and earlier versions of C#) can instead use <a href="/2022/04/25/the-maybe-monad">the Maybe monad</a>. Nothing in this article or the book hinges on the <em>nullable reference types</em> feature. </p> <p> If the <code>Validate</code> method (which I really should have called <code>TryParse</code> instead) returns a null value, the Controller from which this code snippet is taken returns a <code>400 Bad Request</code> response. </p> <p> The <code>Validate</code> method is an instance method on the DTO class: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Reservation?&nbsp;<span style="color:#74531f;">Validate</span>(Guid&nbsp;<span style="color:#1f377f;">id</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!DateTime.TryParse(At,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">d</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Email&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">null</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;d, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Email(Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Name(Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity); }</pre> </p> <p> What irks some readers is the loss of information. While <code>Validate</code> 'knows' why it's rejecting a candidate, that information is lost and no error message is communicated to unfortunate HTTP clients. </p> <p> One email from a reader went on about this for quite some time and I got the impression that the sender considered this such a grave flaw that it invalidates the entire book. </p> <p> That's not the case. </p> <h3 id="26803ee560dd42be825ec266694d0211"> Rabbit hole, evaded <a href="#26803ee560dd42be825ec266694d0211" title="permalink">#</a> </h3> <p> When I wrote the code like above, I was fully aware of trade-offs and priorities. I understood that this particular design would mean that clients get no information about <em>why</em> a particular reservation JSON document is rejected - only that it is. </p> <p> This was a simplification that I explicitly decided to make for educational reasons. </p> <p> The above design is based on something as simple as a null check. I expect all my readers to be able to follow that code. As hinted above, you could also model a method like <code>Validate</code> with the Maybe monad, but while Maybe preserves success cases, it throws away all information about errors. In a production system, this is rarely acceptable, but I found it acceptable for the example code in the book, since this isn't the main topic. </p> <p> Instead of basing the design on nullable reference types or the Maybe monad, you can instead base parsing on applicative validation. In order to explain that, I'd first need to explain <a href="/2018/03/22/functors">functors</a>, <a href="/2018/10/01/applicative-functors">applicative functors</a>, and applicative validation. It might also prove helpful to the reader to explain <a href="/2018/05/22/church-encoding">Church encodings</a>, <a href="/2018/12/24/bifunctors">bifunctors</a>, and <a href="/2017/11/27/semigroups">semigroups</a>. That's quite a rabbit hole to fall into, and I felt that it would be such a big digression from the themes of the book that I decided not to go there. </p> <p> On this blog, however, I have all the space and time I'd like. I can digress as much as I'd like. Most of that digression has already happened. Those articles are already on the blog. I'm going to assume that you've read all of the articles I just linked, or that you understand these concepts. </p> <p> In this article, I'm going to rewrite the DTO parser to also return error messages. It's an entirely local change that breaks no existing tests. </p> <h3 id="5f6a7326fb484c1d8b4402b24e49d13b"> Validated <a href="#5f6a7326fb484c1d8b4402b24e49d13b" title="permalink">#</a> </h3> <p> Most functional programmers are already aware of the <a href="/2022/05/09/an-either-monad">Either monad</a>. They often reach for it when they need to expand the Maybe monad with <a href="https://fsharpforfunandprofit.com/posts/recipe-part2/">an error track</a>. </p> <p> The problem with the Either monad is, however, that it short-circuits error handling. It's like throwing exceptions. As soon as an Either composition hits the first error, it stops processing the rest of the data. As a caller, you only get one error message, even if there's more than one thing wrong with your input value. </p> <p> In a distributed system where a client posts a document to a service, you'd like to respond with a collection of errors. </p> <p> You can do this with a data type that's isomorphic with Either, but behaves differently as an applicative functor. Instead of short-circuiting on the first error, it collects them. This, however, turns out to be incompatible to the Either monad's short-circuiting behaviour, so this data structure is usually not given monadic features. </p> <p> This data type is usually called <code>Validation</code>, but when I translated that to C# various static code analysis rules lit up, claiming that there was already a referenced namespace called <code>Validation</code>. Instead, I decided to call the type <code><span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;</code>, which I like better anyway. </p> <p> The type arguments are <code>F</code> for <em>failure</em> and <code>S</code> for <em>success</em>. I've put <code>F</code> before <code>S</code> because by convention that's how Either works. </p> <p> I'm using an encapsulated variation of a Church encoding and a series of <code>Apply</code> overloads as described in the article <a href="/2018/10/15/an-applicative-password-list">An applicative password list</a>. There's quite a bit of boilerplate, so I'll just dump the entire contents of the file here instead of tiring you with a detailed walk-through: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IValidation</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;<span style="color:#74531f;">Match</span>&lt;<span style="color:#2b91af;">T</span>&gt;(Func&lt;F,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onFailure</span>,&nbsp;Func&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onSuccess</span>); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;IValidation&nbsp;imp; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">Validated</span>(IValidation&nbsp;<span style="color:#1f377f;">imp</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.imp&nbsp;=&nbsp;imp; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Succeed</span>(S&nbsp;<span style="color:#1f377f;">success</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Validated&lt;F,&nbsp;S&gt;(<span style="color:blue;">new</span>&nbsp;Success(success)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Fail</span>(F&nbsp;<span style="color:#1f377f;">failure</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Validated&lt;F,&nbsp;S&gt;(<span style="color:blue;">new</span>&nbsp;Failure(failure)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;<span style="color:#74531f;">Match</span>&lt;<span style="color:#2b91af;">T</span>&gt;(Func&lt;F,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onFailure</span>,&nbsp;Func&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onSuccess</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;imp.Match(onFailure,&nbsp;onSuccess); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Validated&lt;F1,&nbsp;S1&gt;&nbsp;<span style="color:#74531f;">SelectBoth</span>&lt;<span style="color:#2b91af;">F1</span>,&nbsp;<span style="color:#2b91af;">S1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F1&gt;&nbsp;<span style="color:#1f377f;">selectFailure</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;<span style="color:#1f377f;">selectSuccess</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f</span>&nbsp;=&gt;&nbsp;Validated.Fail&lt;F1,&nbsp;S1&gt;(selectFailure(f)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;Validated.Succeed&lt;F1,&nbsp;S1&gt;(selectSuccess(s))); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Validated&lt;F1,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">SelectFailure</span>&lt;<span style="color:#2b91af;">F1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F1&gt;&nbsp;<span style="color:#1f377f;">selectFailure</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;SelectBoth(selectFailure,&nbsp;<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;s); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Validated&lt;F,&nbsp;S1&gt;&nbsp;<span style="color:#74531f;">SelectSuccess</span>&lt;<span style="color:#2b91af;">S1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;<span style="color:#1f377f;">selectSuccess</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;SelectBoth(<span style="color:#1f377f;">f</span>&nbsp;=&gt;&nbsp;f,&nbsp;selectSuccess); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Validated&lt;F,&nbsp;S1&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">S1</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;S1&gt;&nbsp;<span style="color:#1f377f;">selector</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;SelectSuccess(selector); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Success</span>&nbsp;:&nbsp;IValidation &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;S&nbsp;success; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Success</span>(S&nbsp;<span style="color:#1f377f;">success</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.success&nbsp;=&nbsp;success; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;<span style="color:#74531f;">Match</span>&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onFailure</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onSuccess</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;onSuccess(success); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Failure</span>&nbsp;:&nbsp;IValidation &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;F&nbsp;failure; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Failure</span>(F&nbsp;<span style="color:#1f377f;">failure</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.failure&nbsp;=&nbsp;failure; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;<span style="color:#74531f;">Match</span>&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onFailure</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">onSuccess</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;onFailure(failure); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Validated</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Succeed</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;S&nbsp;<span style="color:#1f377f;">success</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated&lt;F,&nbsp;S&gt;.Succeed(success); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Fail</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;F&nbsp;<span style="color:#1f377f;">failure</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated&lt;F,&nbsp;S&gt;.Fail(failure); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Apply</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T,&nbsp;S&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;F,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F,&nbsp;F&gt;&nbsp;<span style="color:#1f377f;">combine</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(selector&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(selector)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;selector.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f1</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;S&gt;(combine(f1,&nbsp;f2)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">_</span>&nbsp;&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;S&gt;(f1)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">map</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;S&gt;(f2), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">x</span>&nbsp;&nbsp;=&gt;&nbsp;Succeed&lt;F,&nbsp;S&gt;(map(x)))); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T2,&nbsp;S&gt;&gt;&nbsp;<span style="color:#74531f;">Apply</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">T1</span>,&nbsp;<span style="color:#2b91af;">T2</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T1,&nbsp;T2,&nbsp;S&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;F,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F,&nbsp;F&gt;&nbsp;<span style="color:#1f377f;">combine</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(selector&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(selector)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;selector.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f1</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;S&gt;&gt;(combine(f1,&nbsp;f2)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">_</span>&nbsp;&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;S&gt;&gt;(f1)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">map</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;S&gt;&gt;(f2), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">x</span>&nbsp;&nbsp;=&gt;&nbsp;Succeed&lt;F,&nbsp;Func&lt;T2,&nbsp;S&gt;&gt;(<span style="color:#1f377f;">y</span>&nbsp;=&gt;&nbsp;map(x,&nbsp;y)))); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;&nbsp;<span style="color:#74531f;">Apply</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">T1</span>,&nbsp;<span style="color:#2b91af;">T2</span>,&nbsp;<span style="color:#2b91af;">T3</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T1,&nbsp;T2,&nbsp;T3,&nbsp;S&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;F,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F,&nbsp;F&gt;&nbsp;<span style="color:#1f377f;">combine</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(selector&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(selector)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;selector.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f1</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;(combine(f1,&nbsp;f2)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">_</span>&nbsp;&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;(f1)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">map</span>&nbsp;=&gt;&nbsp;source.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">f2</span>&nbsp;=&gt;&nbsp;Fail&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;(f2), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">x</span>&nbsp;&nbsp;=&gt;&nbsp;Succeed&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;((<span style="color:#1f377f;">y</span>,&nbsp;<span style="color:#1f377f;">z</span>)&nbsp;=&gt;&nbsp;map(x,&nbsp;y,&nbsp;z)))); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;F,&nbsp;Func&lt;T2,&nbsp;T3,&nbsp;S&gt;&gt;&nbsp;<span style="color:#74531f;">Apply</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">T1</span>,&nbsp;<span style="color:#2b91af;">T2</span>,&nbsp;<span style="color:#2b91af;">T3</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;Func&lt;T1,&nbsp;T2,&nbsp;T3,&nbsp;S&gt;&nbsp;<span style="color:#1f377f;">map</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;F,&nbsp;T1&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Func&lt;F,&nbsp;F,&nbsp;F&gt;&nbsp;<span style="color:#1f377f;">combine</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Apply( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Succeed&lt;F,&nbsp;Func&lt;T1,&nbsp;T2,&nbsp;T3,&nbsp;S&gt;&gt;((<span style="color:#1f377f;">x</span>,&nbsp;<span style="color:#1f377f;">y</span>,&nbsp;<span style="color:#1f377f;">z</span>)&nbsp;=&gt;&nbsp;map(x,&nbsp;y,&nbsp;z)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;source, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;combine); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I only added the <code>Apply</code> overloads that I needed for the following demo code. As stated above, I'm not going to launch into a detailed walk-through, since the code follows the concepts lined out in the various articles I've already mentioned. If there's something that you'd like me to explain then please <a href="https://github.com/ploeh/ploeh.github.com#comments">leave a comment</a>. </p> <p> Notice that <code><span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;</code> has no <code>SelectMany</code> method. It's deliberately not a <a href="/2022/03/28/monads">monad</a>, because monadic <em>bind</em> (<code>SelectMany</code>) would conflict with the applicative functor implementation. </p> <h3 id="533e728019834b22b323ffab10f4cae8"> Individual parsers <a href="#533e728019834b22b323ffab10f4cae8" title="permalink">#</a> </h3> <p> An essential quality of applicative validation is that it's composable. This means that you can compose a larger, more complex parser from smaller ones. Parsing a <code><span style="color:#2b91af;">ReservationDto</span></code> object, for example, involves parsing the date and time of the reservation, the email address, and the quantity. Here's how to parse the date and time: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;Validated&lt;<span style="color:blue;">string</span>,&nbsp;DateTime&gt;&nbsp;<span style="color:#74531f;">TryParseAt</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(!DateTime.TryParse(At,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">d</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Fail&lt;<span style="color:blue;">string</span>,&nbsp;DateTime&gt;(<span style="color:#a31515;">$&quot;Invalid&nbsp;date&nbsp;or&nbsp;time:&nbsp;</span>{At}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Succeed&lt;<span style="color:blue;">string</span>,&nbsp;DateTime&gt;(d); }</pre> </p> <p> In order to keep things simple I'm going to use strings for error messages. You could instead decide to encode error conditions as a <a href="https://en.wikipedia.org/wiki/Tagged_union">sum type</a> or other polymorphic type. This would be appropriate if you also need to be able to make programmatic decisions based on individual error conditions, or if you need to translate the error messages to more than one language. </p> <p> The <code><span style="color:#74531f;">TryParseAt</span></code> function only attempts to parse the <code>At</code> property to a <code>DateTime</code> value. If parsing fails, it returns a <code><span style="color:#2b91af;">Failure</span></code> value with a helpful error message; otherwise, it wraps the parsed date and time in a <code><span style="color:#2b91af;">Success</span></code> value. </p> <p> Parsing the email address is similar: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;Validated&lt;<span style="color:blue;">string</span>,&nbsp;Email&gt;&nbsp;<span style="color:#74531f;">TryParseEmail</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Email&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Fail&lt;<span style="color:blue;">string</span>,&nbsp;Email&gt;(<span style="color:#a31515;">$&quot;Email&nbsp;address&nbsp;is&nbsp;missing.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Succeed&lt;<span style="color:blue;">string</span>,&nbsp;Email&gt;(<span style="color:blue;">new</span>&nbsp;Email(Email)); }</pre> </p> <p> As is parsing the quantity: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;Validated&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#74531f;">TryParseQuantity</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(Quantity&nbsp;&lt;&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Fail&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Quantity&nbsp;must&nbsp;be&nbsp;a&nbsp;positive&nbsp;integer,&nbsp;but&nbsp;was:&nbsp;</span>{Quantity}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Validated.Succeed&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;(Quantity); }</pre> </p> <p> There's no reason to create a parser for the reservation name, because if the name doesn't exist, instead use the empty string. That operation can't fail. </p> <h3 id="48f0650702f1490392838bf3c5c88cd1"> Composition <a href="#48f0650702f1490392838bf3c5c88cd1" title="permalink">#</a> </h3> <p> You can now use applicative composition to reuse those individual parsers in a more complex parser: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;Validated&lt;<span style="color:blue;">string</span>,&nbsp;Reservation&gt;&nbsp;<span style="color:#74531f;">TryParse</span>(Guid&nbsp;<span style="color:#1f377f;">id</span>) { &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;DateTime,&nbsp;Email,&nbsp;<span style="color:blue;">int</span>,&nbsp;Reservation&gt;&nbsp;<span style="color:#1f377f;">createReservation</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#1f377f;">at</span>,&nbsp;<span style="color:#1f377f;">email</span>,&nbsp;<span style="color:#1f377f;">quantity</span>)&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Reservation(id,&nbsp;at,&nbsp;email,&nbsp;<span style="color:blue;">new</span>&nbsp;Name(Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>),&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">combine</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:#1f377f;">x</span>,&nbsp;<span style="color:#1f377f;">y</span>)&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;x,&nbsp;y); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;createReservation &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(TryParseAt(),&nbsp;combine) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(TryParseEmail(),&nbsp;combine) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(TryParseQuantity(),&nbsp;combine); }</pre> </p> <p> <code><span style="color:#1f377f;">createReservation</span></code> is a local function that closes over <code>id</code> and <code>Name</code>. Specifically, it uses the null coalescing operator (<code>??</code>) to turn a null name into the empty string. On the other hand, it takes <code>at</code>, <code>email</code>, and <code>quantity</code> as inputs, since these are the values that must first be parsed. </p> <p> A type like <code><span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;</code> is only an applicative functor when the failure dimension (<code>F</code>) gives rise to a semigroup. The way I've modelled it here is as a binary operation that you need to pass as a parameter to each <code>Apply</code> overload. This seems awkward, but is good enough for a proof of concept. </p> <p> The <code><span style="color:#1f377f;">combine</span></code> function joins two strings together, separated by a line break. </p> <p> The <code><span style="color:#74531f;">TryParse</span></code> function composes <code><span style="color:#1f377f;">createReservation</span></code> with <code>TryParseAt</code>, <code>TryParseEmail</code>, and <code>TryParseQuantity</code> using the various <code>Apply</code> overloads. The combination is a <code>Validated</code> value that's either a failure string or a properly encapsulated <code>Reservation</code> object. </p> <h3 id="04091776345e4f4b8ba4d446d8d80376"> Using the parser <a href="#04091776345e4f4b8ba4d446d8d80376" title="permalink">#</a> </h3> <p> Client code can now invoke the <code>TryParse</code> function on the DTO. Here is the code inside the <code><span style="color:#74531f;">Post</span></code> method on the <code><span style="color:#2b91af;">ReservationsController</span></code> class: </p> <p> <pre>[HttpPost(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/reservations&quot;</span>)] <span style="color:blue;">public</span>&nbsp;Task&lt;ActionResult&gt;&nbsp;<span style="color:#74531f;">Post</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">restaurantId</span>,&nbsp;ReservationDto&nbsp;<span style="color:#1f377f;">dto</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">id</span>&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;Guid.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">parseResult</span>&nbsp;=&nbsp;dto.TryParse(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;parseResult.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">msgs</span>&nbsp;=&gt;&nbsp;Task.FromResult&lt;ActionResult&gt;(<span style="color:blue;">new</span>&nbsp;BadRequestObjectResult(msgs)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;=&gt;&nbsp;TryCreate(restaurantId,&nbsp;reservation)); }</pre> </p> <p> When the <code>parseResult</code> matches a failure, it returns a <code><span style="color:blue;">new</span>&nbsp;BadRequestObjectResult</code> with all collected error messages. When, on the other hand, it matches a success, it invokes the <code><span style="color:#74531f;">TryCreate</span></code> helper method with the parsed <code>reservation</code>. </p> <h3 id="d260c38d1dd444e291868e377732fd24"> HTTP request and response <a href="#d260c38d1dd444e291868e377732fd24" title="permalink">#</a> </h3> <p> A client will now receive all relevant error messages if it posts a malformed reservation: </p> <p> <pre>POST /restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D HTTP/1.1 Content-Type: application/json {&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;large&quot;</span>,&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Kerry&nbsp;Onn&quot;</span>,&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;-1&nbsp;} HTTP/1.1 400 Bad Request Invalid date or time: large. Email address is missing. Quantity must be a positive integer, but was: -1.</pre> </p> <p> Of course, if only a single element is wrong, only that error message will appear. </p> <h3 id="fada4ef8e64648b5a2d90e47082acfde"> Conclusion <a href="#fada4ef8e64648b5a2d90e47082acfde" title="permalink">#</a> </h3> <p> The changes described in this article were entirely local to the two involved types: <code><span style="color:#2b91af;">ReservationsController</span></code> and <code><span style="color:#2b91af;">ReservationDto</span></code>. Once I'd expanded <code><span style="color:#2b91af;">ReservationDto</span></code> with the <code><span style="color:#74531f;">TryParse</span></code> function and its helper functions, and changed <code><span style="color:#2b91af;">ReservationsController</span></code> accordingly, the rest of the code base compiled and all tests passed. The point is that this isn't a big change, and that's why I believe that the original design (returning null or non-null) doesn't invalidate anything else I had to say in the book. </p> <p> The change did, however, take quite a bit of boilerplate code, as witnessed by the <code>Validated</code> code dump. That API is, on the other hand, completely reusable, and you can find packages on the internet that already implement this functionality. It's not much of a burden in terms of extra code, but it would have taken a couple of extra chapters to explain in the book. It could easily have been double the size if I had to include material about functors, applicative functors, semigroups, Church encoding, etcetera. </p> <p> To fix two lines of code, I didn't think that was warranted. After all, it's not a major blocker. On the contrary, validation is a solved problem. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="22b961d6ecb84e6f8af7232b09a445fc"> <div class="comment-author">Dan Carter</div> <div class="comment-content"> <blockquote>you can find packages on the internet that already implement this functionality</blockquote> <p> Do you have any recommendations for a library that implements the <code>Validated<F, S></code> type? </p> </div> <div class="comment-date">2022-08-15 11:15 UTC</div> </div> <div class="comment" id="03273eecd6e749188106e8caef68d82e"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Dan, thank you for writing. The following is not a recommendation, but the most comprehensive C# library for functional programming currently seems to be <a href="https://github.com/louthy/language-ext">LanguageExt</a>, which includes a <a href="https://louthy.github.io/language-ext/LanguageExt.Core/Monads/Alternative%20Value%20Monads/Validation/index.html">Validation</a> functor. </p> <p> I'm neither recommending nor arguing against LanguageExt. </p> <ul> <li>I've never used it in a real-world code base.</li> <li>I've been answering questions about it on <a href="https://stackoverflow.com">Stack Overflow</a>. In general, it seems to stump C# developers, since it's very Haskellish and quite advanced.</li> <li>Today is just a point in time. Libraries come and go.</li> </ul> <p> Since all the ideas presented in these articles are universal abstractions, you can safely and easily implement them yourself, instead of taking a dependency on a third-party library. If you stick with lawful implementations, the only variation possible is with naming. Do you call a functor like this one <code>Validation</code>, <code>Validated</code>, or something else? Do you call monadic <em>bind</em> <code>SelectMany</code> or <code>Bind</code>? Will you have a <code>Flatten</code> or a <code>Join</code> function? </p> <p> When working with teams that are new to these things, I usually start by adding these concepts as source code as they become useful. If a type like <code>Maybe</code> or <code>Validated</code> starts to proliferate, sooner or later you'll need to move it to a shared library so that multiple in-house libraries can use the type to communicate results across library boundaries. Eventually, you may decide to move such a dependency to a NuGet package. You can, at such time, decide to use an existing library instead of your own. </p> <p> The maintenance burden for these kinds of libraries is low, since the APIs and behaviour are defined and locked in advance by mathematics. </p> </div> <div class="comment-date">2022-08-16 5:54 UTC</div> </div> <div class="comment" id="fa7fc6662aee4889bcc9f84bc5db1b39"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> If you stick with lawful implementations, the only variation possible is with naming. </blockquote> <p> There are also language-specific choices that can vary. </p> <p> One example involves applicative functors in C#. The "standard" API for applicative functors works well in <a href="https://blog.ploeh.dk/2018/10/01/applicative-functors/#6173a96aedaa4e97ad868c159e6d06fa">Haskell</a> and <a href="https://blog.ploeh.dk/2018/10/01/applicative-functors/#6f9b2f13951e475d8c4fc9682ad96a94">F#</a> because it is designed to be used with curried functions, and both of those languages curry their functions by default. In contrast, <a href="https://blog.ploeh.dk/2018/10/01/applicative-functors/#cef395ee19644f30bfd1ad7a84b6f912:~:text=Applicative%20functors%20push%20the%20limits%20of%20what%20you%20can%20express%20in%20C%23">applicative functors push the limits of what you can express in C#</a>. I am impressed with <a href="https://github.com/louthy/language-ext/blob/ba16d97d4909067222c8b134a80bfd6b7e54c424/LanguageExt.Tests/ValidationTests.cs#L277">the design that Language Ext uses for applicative functors</a>, which is an extension method on a (value) tuple of applicative functor instances that accepts a lambda expression that is given all the "unwrapped" values "inside" the applicative functors. </p> <p> Another example involves monads in TypeScript. To avoid the <a href="https://en.wikipedia.org/wiki/Pyramid_of_doom_(programming)">Pyramid of doom</a> when performing a sequence of monadic operations, Haskell has <a href="https://en.wikibooks.org/wiki/Haskell/do_notation">do notation</a> and F# has <a href="https://learn.microsoft.com/en-us/dotnet/fsharp/language-reference/computation-expressions">computation expressions</a>. There is no equivalent language feature in TypeScript, but it has <a href="https://en.wikipedia.org/wiki/Row_polymorphism">row polymorphism</a>, which <a href="https://gcanti.github.io/fp-ts/guides/do-notation.html">pf-ts uses to effectively implement do notation</a>. </p> <p> A related dimension is how to approximate high-kinded types in a language that lacks them. Language Ext passes in the monad as a type parameter as well as the "lower-kinded" type parameter and then constrains the monad type parameter to implement a monad interface parametereized by the lower type parameter as well as being a struct. I find that second constraint very intersting. Since the type parameter has a struct constraint, it has a default constructor that can be used to get an instance, which then implements methods according to the interface constraint. For more infomration, see <a href="https://github.com/louthy/language-ext/wiki/Does-C%23-Dream-Of-Electric-Monads%3F">this wiki article</a> for a gentle introduction and <a href="https://github.com/louthy/language-ext/blob/ba16d97d4909067222c8b134a80bfd6b7e54c424/LanguageExt.Core/Class%20Instances/Trans/Trans.cs#L74-L80">Trans.cs</a> for how Language Ext uses this approach to only implement traverse once. Similarly, F#+ has a feature called <a href="https://fsprojects.github.io/FSharpPlus/generic-doc.html">generic functions</a> that enable one to write F# like <code>map aFoo</code> instead of the typical <code>Foo.map aFoo</code>. </p> </div> <div class="comment-date">2022-09-20 02:00 UTC</div> </div> <div class="comment" id="5619079f7c994fc0bd0dbbdbfeec9e4a"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I agree that details differ. Clearly, this is true across languages, where, say, <a href="https://www.haskell.org/">Haskell</a>'s <code>fmap</code> has a name different from C#'s <code>SelectMany</code>. To state the obvious, the syntax is also different. </p> <p> Even within the same language, you can have variations. Functor mapping in Haskell is generally called <code>fmap</code>, but you can also use <code>map</code> explicitly for lists. The same could be true in C#. I've seen functor and monad implementations in C# that use method names like <code>Map</code> and <code>Bind</code> rather than <code>Select</code> and <code>SelectMany</code>. </p> <p> To expand on this idea, one may also observe that what one language calls <em>Option</em>, another language calls <em>Maybe</em>. The same goes for <code>Result</code> versus <code>Either</code>. </p> <p> As you know, the names <code>Select</code> and <code>SelectMany</code> are special because they enable C# query syntax. While methods named <code>Map</code> and <code>Bind</code> are 'the same' functions, they don't light up that language feature. Another way to enable syntactic sugar for monads in C# is via <code>async</code> and <code>await</code>, as <a href="https://eiriktsarpalis.wordpress.com/2020/07/20/effect-programming-in-csharp/">shown by Eirik Tsarpalis and Nick Palladinos</a>. </p> <p> I do agree with you that there are various options available to an implementer. The point I was trying to make is that while implementation details differ, the concepts are the same. Thus, as a <em>user</em> of one of these APIs (monads, monoids, etc.) you only have to learn the mental model once. You still have to learn the implementation details. </p> <p> I recently heard a professor at <a href="https://en.wikipedia.org/wiki/UCPH_Department_of_Computer_Science">DIKU</a> state that once you know one programming language, you should be able to learn another one in a week. That's the same general idea. </p> <p> (I do, however, have issues with that statement about programming languages as a universal assertion, but I agree that it tends to hold for mainstream languages. When I read <a href="/ref/mazes-for-programmers">Mazes for Programmers</a> I'd never programmed in <a href="https://www.ruby-lang.org/en/">Ruby</a> before, but I had little trouble picking it up for the exercises. On the other hand, most people don't learn Haskell in a week.) </p> </div> <div class="comment-date">2022-09-20 17:42 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Natural transformations https://blog.ploeh.dk/2022/07/18/natural-transformations 2022-07-18T08:12:00+00:00 Mark Seemann <div id="post"> <p> <em>Mappings between functors, with some examples in C#.</em> </p> <p> This article is part of <a href="/2022/07/11/functor-relationships">a series of articles about functor relationships</a>. In this one you'll learn about natural transformations, which are simply mappings between two <a href="/2018/03/22/functors">functors</a>. It's probably the easiest relationship to understand. In fact, it may be so obvious that your reaction is: <em>Is that it?</em> </p> <p> In programming, a natural transformation is just a function from one functor to another. A common example is a function that tries to extract a value from a collection. You'll see specific examples a little later in this article. </p> <h3 id="05a07e6563da4ba39c83bd92250d3056"> Laws <a href="#05a07e6563da4ba39c83bd92250d3056" title="permalink">#</a> </h3> <p> In this, the dreaded section on <em>laws</em>, I have a nice surprise for you: There aren't any (that we need worry about)! </p> <p> In the broader context of <a href="https://en.wikipedia.org/wiki/Category_theory">category theory</a> there are, in fact, rules that a natural transformation must follow. </p> <blockquote> <p> "Haskell's parametric polymorphism has an unexpected consequence: any polymorphic function of the type: </p> <p> <pre>alpha :: F a -&gt; G a</pre> </p> <p> "where <code>F</code> and <code>G</code> are functors, automatically satisfies the naturality condition." </p> <footer><cite><a href="https://bartoszmilewski.com/2015/04/07/natural-transformations/">Natural Transformations</a>, Bartosz Milewski</cite></footer> </blockquote> <p> While C# isn't <a href="https://www.haskell.org">Haskell</a>, .NET generics are similar enough to Haskell <a href="https://en.wikipedia.org/wiki/Parametric_polymorphism">parametric polymorphism</a> that the result, as far as I can tell, carry over. (Again, however, we have to keep in mind that C# doesn't distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. The knowledge that I infer translates for pure functions. For impure actions, there are no guarantees.) </p> <p> The C# equivalent of the above <code>alpha</code> function would be a method like this: </p> <p> <pre>G&lt;T&gt;&nbsp;<span style="color:#74531f;">Alpha</span>&lt;<span style="color:#2b91af;">T</span>&gt;(F&lt;T&gt;&nbsp;<span style="color:#1f377f;">f</span>)</pre> </p> <p> where both <code>F</code> and <code>G</code> are functors. </p> <h3 id="643a53f5758f485f8a7000f9e7825c5c"> Safe head <a href="#643a53f5758f485f8a7000f9e7825c5c" title="permalink">#</a> </h3> <p> Natural transformations easily occur in normal programming. You've probably written some yourself, without being aware of it. Here are some examples. </p> <p> It's common to attempt to get the first element of a collection. Collections, however, may be empty, so this is not always possible. In Haskell, you'd model that as a function that takes a list as input and returns a <code>Maybe</code> as output: </p> <p> <pre>Prelude Data.Maybe&gt; :t listToMaybe listToMaybe :: [a] -&gt; Maybe a Prelude Data.Maybe&gt; listToMaybe [] Nothing Prelude Data.Maybe&gt; listToMaybe [7] Just 7 Prelude Data.Maybe&gt; listToMaybe [3,9] Just 3 Prelude Data.Maybe&gt; listToMaybe [5,9,2,4,4] Just 5</pre> </p> <p> In many tutorials such a function is often called <code>safeHead</code>, because it returns the <em>head</em> of a list (i.e. the first item) in a safe manner. It returns <code>Nothing</code> if the list is empty. In <a href="https://fsharp.org">F#</a> this function is called <a href="https://fsharp.github.io/fsharp-core-docs/reference/fsharp-collections-seqmodule.html#tryHead">tryHead</a>. </p> <p> In C# you could write a similar function like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Maybe&lt;T&gt;&nbsp;<span style="color:#74531f;">TryFirst</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;IEnumerable&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(source.Any()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Maybe&lt;T&gt;(source.First()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Maybe.Empty&lt;T&gt;(); }</pre> </p> <p> This extension method (which is really a pure function) is a natural transformation between two functors. The source functor is the <a href="/2022/04/19/the-list-monad">list functor</a> and the destination is <a href="/2018/03/26/the-maybe-functor">the Maybe functor</a>. </p> <p> Here are some unit tests that demonstrate how it works: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">TryFirstWhenEmpty</span>() { &nbsp;&nbsp;&nbsp;&nbsp;Maybe&lt;Guid&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;Enumerable.Empty&lt;Guid&gt;().TryFirst(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(Maybe.Empty&lt;Guid&gt;(),&nbsp;actual); } [Theory] [InlineData(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;foo&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;foo&quot;</span>)] [InlineData(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;baz&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>)] [InlineData(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;qux&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;quux&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;quuz&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;corge&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;corge&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;qux&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">TryFirstWhenNotEmpty</span>(<span style="color:blue;">string</span>[]&nbsp;<span style="color:#1f377f;">arr</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">expected</span>) { &nbsp;&nbsp;&nbsp;&nbsp;Maybe&lt;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;arr.TryFirst(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(<span style="color:blue;">new</span>&nbsp;Maybe&lt;<span style="color:blue;">string</span>&gt;(expected),&nbsp;actual); }</pre> </p> <p> All these tests pass. </p> <h3 id="6d899e8f2a7948ed898eaaefc3064dc6"> Safe index <a href="#6d899e8f2a7948ed898eaaefc3064dc6" title="permalink">#</a> </h3> <p> The above <em>safe head</em> natural transformation is just one example. Even for a particular combination of functors like <em>List to Maybe</em> many natural transformations may exist. For this particular combination, there are infinitely many natural transformations. </p> <p> You can view the <em>safe head</em> example as a special case of a more general set of <em>safe indexing</em>. With a collection of values, you can attempt to retrieve the value at a particular index. Since a collection can contain an arbitrary number of elements, however, there's no guarantee that there's an element at the requested index. </p> <p> In order to avoid exceptions, then, you can try to retrieve the value at an index, getting a Maybe value as a result. </p> <p> The F# <code>Seq</code> module defines a function called <a href="https://fsharp.github.io/fsharp-core-docs/reference/fsharp-collections-seqmodule.html#tryItem">tryItem</a>. This function takes an index and a sequence (<code>IEnumerable&lt;T&gt;</code>) and returns an <code>option</code> (F#'s name for Maybe): </p> <p> <pre>&gt; Seq.tryItem 2 [2;5;3;5];; val it : int option = Some 3</pre> </p> <p> The <code>tryItem</code> function itself is <em>not</em> a natural transformation, but because of currying, it's a function that <em>returns</em> a natural transformation. When you partially apply it with an index, it becomes a natural transformation: <code>Seq.tryItem 3</code> is a natural transformation <code>seq&lt;'a&gt; -&gt; 'a option</code>, as is <code>Seq.tryItem 4</code>, <code>Seq.tryItem 5</code>, and so on ad infinitum. Thus, there are infinitely many natural transformations from the List functor to the Maybe functor, and <em>safe head</em> is simply <code>Seq.tryItem 0</code>. </p> <p> In C# you can use the various <code>Func</code> delegates to implement currying, but if you want something that looks a little more object-oriented, you could write code like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Index</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;index; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Index</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">index</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.index&nbsp;=&nbsp;index; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Maybe&lt;T&gt;&nbsp;<span style="color:#74531f;">TryItem</span>&lt;<span style="color:#2b91af;">T</span>&gt;(IEnumerable&lt;T&gt;&nbsp;<span style="color:#1f377f;">values</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">candidate</span>&nbsp;=&nbsp;values.Skip(index).Take(1); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(candidate.Any()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Maybe&lt;T&gt;(candidate.First()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Maybe.Empty&lt;T&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This <code>Index</code> class captures an index value for potential use against any <code>IEnumerable&lt;T&gt;</code>. Thus, the <code>TryItem</code> method is a natural transformation from the List functor to the Maybe functor. Here are some examples: </p> <p> <pre>[Theory] [InlineData(0,&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>[0])] [InlineData(1,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;bee&quot;</span>&nbsp;})] [InlineData(2,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;nig&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;fev&quot;</span>&nbsp;})] [InlineData(4,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;sta&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;ali&quot;</span>&nbsp;})] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">MissItem</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">i</span>,&nbsp;<span style="color:blue;">string</span>[]&nbsp;<span style="color:#1f377f;">xs</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">idx</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Index(i); &nbsp;&nbsp;&nbsp;&nbsp;Maybe&lt;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;idx.TryItem(xs); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(Maybe.Empty&lt;<span style="color:blue;">string</span>&gt;(),&nbsp;actual); } [Theory] [InlineData(0,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;foo&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;foo&quot;</span>)] [InlineData(1,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;baz&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;baz&quot;</span>)] [InlineData(1,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;qux&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;quux&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;quuz&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;quux&quot;</span>)] [InlineData(2,&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#a31515;">&quot;corge&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;grault&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;fred&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;garply&quot;</span>&nbsp;},&nbsp;<span style="color:#a31515;">&quot;fred&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">FindItem</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">i</span>,&nbsp;<span style="color:blue;">string</span>[]&nbsp;<span style="color:#1f377f;">xs</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">expected</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">idx</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Index(i); &nbsp;&nbsp;&nbsp;&nbsp;Maybe&lt;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;idx.TryItem(xs); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(<span style="color:blue;">new</span>&nbsp;Maybe&lt;<span style="color:blue;">string</span>&gt;(expected),&nbsp;actual); }</pre> </p> <p> Since there are infinitely many integers, there are infinitely many such natural transformations. (This is strictly not true for the above code, since there's a finite number of 32-bit integers. Exercise: Is it possible to rewrite the above <code>Index</code> class to instead work with <a href="https://docs.microsoft.com/dotnet/api/system.numerics.biginteger">BigInteger</a>?) </p> <p> The Haskell <a href="https://hackage.haskell.org/package/natural-transformation">natural-transformation</a> package offers an even more explicit way to present the same example: </p> <p> <pre><span style="color:blue;">import</span>&nbsp;Control.Natural <span style="color:#2b91af;">tryItem</span>&nbsp;::&nbsp;(<span style="color:blue;">Eq</span>&nbsp;a,&nbsp;<span style="color:blue;">Num</span>&nbsp;a,&nbsp;<span style="color:blue;">Enum</span>&nbsp;a)&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[]&nbsp;:~&gt;&nbsp;<span style="color:#2b91af;">Maybe</span> tryItem&nbsp;i&nbsp;=&nbsp;NT&nbsp;$&nbsp;<span style="color:blue;">lookup</span>&nbsp;i&nbsp;.&nbsp;<span style="color:blue;">zip</span>&nbsp;[0..]</pre> </p> <p> You can view this <code>tryItem</code> function as a function that takes a number and returns a particular natural transformation. For example you can define a value called <code>tryThird</code>, which is a natural transformation from <code>[]</code> to <code>Maybe</code>: </p> <p> <pre>λ tryThird = tryItem 2 λ :t tryThird tryThird :: [] :~&gt; Maybe</pre> </p> <p> Here are some usage examples: </p> <p> <pre>λ tryThird # [] Nothing λ tryThird # [1] Nothing λ tryThird # [2,3] Nothing λ tryThird # [4,5,6] Just 6 λ tryThird # [7,8,9,10] Just 9</pre> </p> <p> In all three languages (F#, C#, Haskell), <em>safe head</em> is really just a special case of <em>safe index</em>: <code>Seq.tryItem 0</code> in F#, <code><span style="color:blue;">new</span>&nbsp;Index(0)</code> in C#, and <code>tryItem 0</code> in Haskell. </p> <h3 id="a8e50cd940974b88b5564c2939900702"> Maybe to List <a href="#a8e50cd940974b88b5564c2939900702" title="permalink">#</a> </h3> <p> You can also move in the opposite direction: From Maybe to List. In F#, I can't find a function that translates from <code>option 'a</code> to <code>seq 'a</code> (<code>IEnumerable&lt;T&gt;</code>), but there are both <a href="https://fsharp.github.io/fsharp-core-docs/reference/fsharp-core-optionmodule.html#toArray">Option.toArray</a> and <a href="https://fsharp.github.io/fsharp-core-docs/reference/fsharp-core-optionmodule.html#toList">Option.toList</a>. I'll use <code>Option.toList</code> for a few examples: </p> <p> <pre>&gt; Option.toList (None : string option);; val it : string list = [] &gt; Option.toList (Some "foo");; val it : string list = ["foo"]</pre> </p> <p> Contrary to translating from List to Maybe, going the other way there aren't a lot of options: <code>None</code> translates to an empty list, and <code>Some</code> translates to a singleton list. </p> <p> Using <a href="/2018/06/25/visitor-as-a-sum-type">a Visitor-based</a> Maybe in C#, you can implement the natural transformation like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IEnumerable&lt;T&gt;&nbsp;<span style="color:#74531f;">ToList</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;IMaybe&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.Accept(<span style="color:blue;">new</span>&nbsp;ToListVisitor&lt;T&gt;()); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ToListVisitor</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;:&nbsp;IMaybeVisitor&lt;T,&nbsp;IEnumerable&lt;T&gt;&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IEnumerable&lt;T&gt;&nbsp;VisitNothing &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span>&nbsp;{&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Enumerable.Empty&lt;T&gt;();&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IEnumerable&lt;T&gt;&nbsp;<span style="color:#74531f;">VisitJust</span>(T&nbsp;<span style="color:#1f377f;">just</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;just&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Here are some examples: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">NothingToList</span>() { &nbsp;&nbsp;&nbsp;&nbsp;IMaybe&lt;<span style="color:blue;">double</span>&gt;&nbsp;<span style="color:#1f377f;">maybe</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Nothing&lt;<span style="color:blue;">double</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;IEnumerable&lt;<span style="color:blue;">double</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;maybe.ToList(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Empty(actual); } [Theory] [InlineData(-1)] [InlineData(&nbsp;0)] [InlineData(15)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">JustToList</span>(<span style="color:blue;">double</span>&nbsp;<span style="color:#1f377f;">d</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IMaybe&lt;<span style="color:blue;">double</span>&gt;&nbsp;<span style="color:#1f377f;">maybe</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Just&lt;<span style="color:blue;">double</span>&gt;(d); &nbsp;&nbsp;&nbsp;&nbsp;IEnumerable&lt;<span style="color:blue;">double</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;maybe.ToList(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Single(actual,&nbsp;d); }</pre> </p> <p> In Haskell this natural transformation is called <a href="https://hackage.haskell.org/package/base/docs/Data-Maybe.html#v:maybeToList">maybeToList</a> - just when you think that Haskell names are always <a href="/2021/06/07/abstruse-nomenclature">abstruse</a>, you learn that some are very explicit and self-explanatory. </p> <p> If we wanted, we could use the <em>natural-transformation</em> package to demonstrate that this is, indeed, a natural transformation: </p> <p> <pre>λ :t NT maybeToList NT maybeToList :: Maybe :~&gt; []</pre> </p> <p> There would be little point in doing so, since we'd need to unwrap it again to use it. Using the function directly, on the other hand, looks like this: </p> <p> <pre>λ maybeToList Nothing [] λ maybeToList $ Just 2 [2] λ maybeToList $ Just "fon" ["fon"]</pre> </p> <p> A <code>Nothing</code> value is always translated to the empty list, and a <code>Just</code> value to a singleton list, exactly as in the other languages. </p> <p> Exercise: Is this the only possible natural transformation from Maybe to List? </p> <h3 id="5dfc02dd10514c2288721ace174c7229"> Maybe-Either relationships <a href="#5dfc02dd10514c2288721ace174c7229" title="permalink">#</a> </h3> <p> The Maybe functor is isomorphic to <a href="/2019/01/14/an-either-functor">Either</a> where the <em>left</em> (or <em>error</em>) dimension is <a href="/2018/01/15/unit-isomorphisms">unit</a>. Here are the two natural transformations in F#: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;Option&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;&#39;a&nbsp;option&nbsp;-&gt;&nbsp;Result&lt;&#39;a,unit&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;toResult&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;&#39;a,unit&gt;&nbsp;-&gt;&nbsp;&#39;a&nbsp;option</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;ofResult&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Some&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;None</pre> </p> <p> In F#, Maybe is called <code>option</code> and Either is called <code>Result</code>. Be aware that the F# <code>Result</code> discriminated union puts the <code>Error</code> dimension to the right of the <code>Ok</code>, which is opposite of Either, where <em>left</em> is usually used for errors, and <em>right</em> for successes (because what is correct is right). </p> <p> Here are some examples: </p> <p> <pre>&gt; Some "epi" |&gt; Option.toResult;; val it : Result&lt;string,unit&gt; = Ok "epi" &gt; Ok "epi" |&gt; Option.ofResult;; val it : string option = Some "epi"</pre> </p> <p> Notice that the natural transformation from <code>Result</code> to <code>Option</code> is only defined for <code>Result</code> values where the <code>Error</code> type is <code>unit</code>. You could also define a natural transformation from <em>any</em> <code>Result</code> to <code>option</code>: </p> <p> <pre><span style="color:green;">//&nbsp;Result&lt;&#39;a,&#39;b&gt;&nbsp;-&gt;&nbsp;&#39;a&nbsp;option</span> <span style="color:blue;">let</span>&nbsp;ignoreErrorValue&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;x&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Some&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;None</pre> </p> <p> That's still a natural transformation, but no longer part of an isomorphism due to the loss of information: </p> <p> <pre>&gt; (Error "Catastrophic failure" |&gt; ignoreErrorValue : int option);; val it : int option = None</pre> </p> <p> Just like above, when examining the infinitely many natural transformations from List to Maybe, we can use the Haskell <em>natural-transformation</em> package to make this more explicit: </p> <p> <pre><span style="color:#2b91af;">ignoreLeft</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Either</span>&nbsp;b&nbsp;:~&gt;&nbsp;<span style="color:#2b91af;">Maybe</span> ignoreLeft&nbsp;=&nbsp;NT&nbsp;$&nbsp;either&nbsp;(<span style="color:blue;">const</span>&nbsp;Nothing)&nbsp;Just</pre> </p> <p> <code>ignoreLeft</code> is a natural transformation from the <code>Either b</code> functor to the <code>Maybe</code> functor. </p> <p> Using a Visitor-based Either implementation (refactored from <a href="/2018/06/11/church-encoded-either">Church-encoded Either</a>), you can implement an equivalent <code>IgnoreLeft</code> natural transformation in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IMaybe&lt;R&gt;&nbsp;<span style="color:#74531f;">IgnoreLeft</span>&lt;<span style="color:#2b91af;">L</span>,&nbsp;<span style="color:#2b91af;">R</span>&gt;(<span style="color:blue;">this</span>&nbsp;IEither&lt;L,&nbsp;R&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.Accept(<span style="color:blue;">new</span>&nbsp;IgnoreLeftVisitor&lt;L,&nbsp;R&gt;()); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IgnoreLeftVisitor</span>&lt;<span style="color:#2b91af;">L</span>,&nbsp;<span style="color:#2b91af;">R</span>&gt;&nbsp;:&nbsp;IEitherVisitor&lt;L,&nbsp;R,&nbsp;IMaybe&lt;R&gt;&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IMaybe&lt;R&gt;&nbsp;<span style="color:#74531f;">VisitLeft</span>(L&nbsp;<span style="color:#1f377f;">left</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Nothing&lt;R&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IMaybe&lt;R&gt;&nbsp;<span style="color:#74531f;">VisitRight</span>(R&nbsp;<span style="color:#1f377f;">right</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Just&lt;R&gt;(right); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Here are some examples: </p> <p> <pre>[Theory] [InlineData(<span style="color:#a31515;">&quot;OMG!&quot;</span>)] [InlineData(<span style="color:#a31515;">&quot;Catastrophic&nbsp;failure&quot;</span>)] [InlineData(<span style="color:#a31515;">&quot;Important&nbsp;information!&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">IgnoreLeftOfLeft</span>(<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">msg</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IEither&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#1f377f;">e</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Left&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;(msg); &nbsp;&nbsp;&nbsp;&nbsp;IMaybe&lt;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;e.IgnoreLeft(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(<span style="color:blue;">new</span>&nbsp;Nothing&lt;<span style="color:blue;">int</span>&gt;(),&nbsp;actual); } [Theory] [InlineData(0)] [InlineData(1)] [InlineData(2)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">IgnoreLeftOfRight</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">i</span>) { &nbsp;&nbsp;&nbsp;&nbsp;IEither&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#1f377f;">e</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Right&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;(i); &nbsp;&nbsp;&nbsp;&nbsp;IMaybe&lt;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;e.IgnoreLeft(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(<span style="color:blue;">new</span>&nbsp;Just&lt;<span style="color:blue;">int</span>&gt;(i),&nbsp;actual); }</pre> </p> <p> I'm not insisting that this natural transformation is always useful, but I've occasionally found myself in situations were it came in handy. </p> <h3 id="cc17fdc2940948978ff8bea222503965"> Natural transformations to or from Identity <a href="#cc17fdc2940948978ff8bea222503965" title="permalink">#</a> </h3> <p> Some natural transformations are a little less obvious. If you have a <code><span style="color:#2b91af;">NotEmptyCollection</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> class as shown in my article <a href="/2017/12/11/semigroups-accumulate">Semigroups accumulate</a>, you could consider the <code>Head</code> property a natural transformation. It translates a <code><span style="color:#2b91af;">NotEmptyCollection</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> object to a <code>T</code> object. </p> <p> This function also exists in Haskell, where it's simply called <a href="https://hackage.haskell.org/package/base/docs/Data-List-NonEmpty.html#v:head">head</a>. </p> <p> The input type (<code><span style="color:#2b91af;">NotEmptyCollection</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> in C#, <code>NonEmpty a</code> in Haskell) is a functor, but the return type is a 'naked' value. That doesn't look like a functor. </p> <p> True, a naked value isn't a functor, but it's isomorphic to <a href="/2018/09/03/the-identity-functor">the Identity functor</a>. In Haskell, you can make that relationship quite explicit: </p> <p> <pre><span style="color:#2b91af;">headNT</span>&nbsp;::&nbsp;<span style="color:blue;">NonEmpty</span>&nbsp;:~&gt;&nbsp;<span style="color:blue;">Identity</span> headNT&nbsp;=&nbsp;NT&nbsp;$&nbsp;Identity&nbsp;.&nbsp;NonEmpty.<span style="color:blue;">head</span></pre> </p> <p> While not particularly useful in itself, this demonstrates that it's possible to think of the <code>head</code> function as a natural transformation from <code>NonEmpty</code> to <code>Identity</code>. </p> <p> Can you go the other way, too? </p> <p> Yes, indeed. Consider <a href="/2022/03/28/monads">monadic return</a>. This is a function that takes a 'naked' value and wraps it in a particular monad (which is also, always, a functor). Again, you may consider the 'naked' value as isomorphic with the Identity functor, and thus <em>return</em> as a natural transformation: </p> <p> <pre><span style="color:#2b91af;">returnNT</span>&nbsp;::&nbsp;<span style="color:blue;">Monad</span>&nbsp;m&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;<span style="color:blue;">Identity</span>&nbsp;:~&gt;&nbsp;m returnNT&nbsp;=&nbsp;NT&nbsp;$&nbsp;<span style="color:blue;">return</span>&nbsp;.&nbsp;runIdentity</pre> </p> <p> We might even consider if a function <code>a -&gt; a</code> (in Haskell syntax) or <code>Func&lt;T, T&gt;</code> (in C# syntax) might actually be a natural transformation from Identity to Identity... (It is, but only one such function exists.) </p> <h3 id="335f008f8f65439e8253bf96968af169"> Not all natural transformations are useful <a href="#335f008f8f65439e8253bf96968af169" title="permalink">#</a> </h3> <p> Are are all functor combinations possible as natural transformations? Can you take any two functors and define one or more natural transformations? I'm not sure, but it seems clear that even if it is so, not all natural transformations are useful. </p> <p> Famously, for example, you can't <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">get the value out of</a> the <a href="/2020/06/22/the-io-functor">IO functor</a>. Thus, at first glance it seems impossible to define a natural transformation <em>from</em> <code>IO</code> to some other functor. After all, how would you implement a natural transformation from <code>IO</code> to, say, the Identity functor. That seems impossible. </p> <p> On the other hand, <em>this</em> is possible: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IEnumerable&lt;T&gt;&nbsp;<span style="color:#74531f;">Collapse</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">break</span>; }</pre> </p> <p> That's a natural transformation from <code>IO&lt;T&gt;</code> to <code>IEnumerable&lt;T&gt;</code>. It's possible to ignore the input value and <em>always</em> return an empty sequence. This natural transformation collapses all values to a single return value. </p> <p> You can repeat this exercise with the Haskell <em>natural-transformation</em> package: </p> <p> <pre><span style="color:#2b91af;">collapse</span>&nbsp;::&nbsp;f&nbsp;:~&gt;&nbsp;[] collapse&nbsp;=&nbsp;NT&nbsp;$&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">[]</span></pre> </p> <p> This one collapses <em>any</em> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers/">container</a> <code>f</code> to a List (<code>[]</code>), including <code>IO</code>: </p> <p> <pre>λ collapse # (return 10 :: IO Integer) [] λ collapse # putStrLn "ploeh" []</pre> </p> <p> Notice that in the second example, the <code>IO</code> action is <code>putStrLn "ploeh"</code>, which ought to produce the side effect of writing to the console. This is effectively prevented - instead the <code>collapse</code> natural transformation simply produces the empty list as output. </p> <p> You can define a similar natural transformation from any functor (including <code>IO</code>) to Maybe. Try it as an exercise, in either C#, Haskell, or another language. If you want a Haskell-specific exercise, also define a natural transformation of this type: <code><span style="color:blue;">Alternative</span>&nbsp;g&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;f&nbsp;:~&gt;&nbsp;g</code>. </p> <p> These natural transformations are possible, but hardly useful. </p> <h3 id="103675f9845b4aa1b32f59a510280a53"> Conclusion <a href="#103675f9845b4aa1b32f59a510280a53" title="permalink">#</a> </h3> <p> A natural transformation is a function that translates one functor into another. Useful examples are safe or total collection indexing, including retrieving the first element from a collection. These natural transformations return a populated Maybe value if the element exists, and an empty Maybe value otherwise. </p> <p> Other examples include translating Maybe values into Either values or Lists. </p> <p> A natural transformation can easily involve loss of information. Even if you're able to retrieve the first element in a collection, the return value includes only that value, and not the rest of the collection. </p> <p> A few natural transformations may be true isomorphisms, but in general, being able to go in both directions isn't required. In degenerate cases, a natural transformation may throw away all information and map to a general empty value like the empty List or an empty Maybe value. </p> <p> <strong>Next:</strong> Traversals. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Functor relationships https://blog.ploeh.dk/2022/07/11/functor-relationships 2022-07-11T08:09:00+00:00 Mark Seemann <div id="post"> <p> <em>Sometimes you need to use more than one functor together.</em> </p> <p> This article series is part of <a href="/2018/03/19/functors-applicatives-and-friends">a larger series of articles about functors, applicatives, and other mappable containers</a>. Particularly, you've seen examples of both <a href="/2018/03/22/functors">functors</a> and <a href="/2018/10/01/applicative-functors">applicative functors</a>. </p> <p> There are situations where you can get by with a single functor. Many languages come with list comprehensions or other features to work with collections of values (C#, for instance, has <em>language-integrated query</em>, or: LINQ). The <a href="/2022/04/19/the-list-monad">list functor (and monad)</a> gives you a comprehensive API to manipulate multiple values. Likewise, you may write some parsing (<a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">or validation</a>) that exclusively uses the <a href="/2019/01/14/an-either-functor">Either functor</a>. </p> <p> At other times, however, you may find yourself having to juggle more than one functor at once. Perhaps you are working with Either values, but one existing API returns <a href="/2018/03/26/the-maybe-functor">Maybe</a> values instead. Or perhaps you need to deal with Either values, but you're already working within <a href="/2018/09/24/asynchronous-functors">an asynchronous functor</a>. </p> <p> There are several standard ways you can combine or transform combinations of functors. </p> <h3 id="70383da35af346f6b2dc3095a1bf7273"> A partial catalogue <a href="#70383da35af346f6b2dc3095a1bf7273" title="permalink">#</a> </h3> <p> The following relationships often come in handy - particularly those that top this list: </p> <ul> <li><a href="/2022/07/18/natural-transformations">Natural transformations</a></li> <li>Traversals</li> <li>Functor compositions</li> <li>Functor products</li> <li>Functor sums</li> </ul> <p> This list is hardly complete, and I may add to it in the future. Compared to some of the other subtopics of <a href="/2017/10/04/from-design-patterns-to-category-theory">the larger articles series on universal abstractions</a>, this catalogue is more heterogeneous. It collects various ways that functors can relate to each other, but uses disparate concepts and abstractions, rather than a single general idea (like a <a href="/2018/12/24/bifunctors">bifunctor</a>, <a href="/2017/10/06/monoids">monoid</a>, or <a href="/2019/04/29/catamorphisms">catamorphism</a>). </p> <p> Keep in mind when reading these articles that all <a href="/2022/03/28/monads">monads</a> are also functors and applicative functors, so what applies to functors also applies to monads. </p> <h3 id="1a4f0850f8974b59ab44eed352b0daf7"> Conclusion <a href="#1a4f0850f8974b59ab44eed352b0daf7" title="permalink">#</a> </h3> <p> You can use a single functor in isolation, or you can combine more than one. Most of the relationships described in this articles series work for all (lawful) functors, but traversals require applicative functors and functors that are 'foldable' (i.e. a catamorphism exists). </p> <p> <strong>Next:</strong> <a href="/2022/07/18/natural-transformations">Natural transformations</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Get and Put State https://blog.ploeh.dk/2022/07/04/get-and-put-state 2022-07-04T09:15:00+00:00 Mark Seemann <div id="post"> <p> <em>A pair of standard helper functions for the State monad. An article for object-oriented programmers.</em> </p> <p> The <a href="/2022/06/20/the-state-monad">State monad</a> is completely defined by <a href="/2022/03/28/monads">its two defining functions</a> (<code>SelectMany</code> and <code>Return</code>). While you can get by without them, two additional helper functions (<em>get</em> and <em>put</em>) are so convenient that they're typically included. To be clear, they're not part of the State <em>monad</em> - rather, you can consider them part of what we may term a <em>standard State API</em>. </p> <p> In short, <em>get</em> is a function that, as the name implies, gets the state while inside the State monad, and <em>put</em> replaces the state with a new value. </p> <p> Later in this article, I'll show how to implement these two functions, as well as a usage example. Before we get to that, however, I want to show a motivating example. In other words, an example that doesn't use <em>get</em> and <em>put</em>. </p> <p> The code shown in this article uses the C# State implementation from <a href="/2022/06/20/the-state-monad">the State monad article</a>. </p> <h3 id="62936af6492e474cb1f6afd919bc2cde"> Aggregator <a href="#62936af6492e474cb1f6afd919bc2cde" title="permalink">#</a> </h3> <p> Imagine that you have to implement a simple <a href="https://www.enterpriseintegrationpatterns.com/Aggregator.html">Aggregator</a>. </p> <blockquote> <p> "How do we combine the results of individual but related messages so that they can be processed as a whole?" </p> <p> [...] "Use a stateful filter, an <em>Aggregator</em>, to collect and store individual messages until it receives a complete set of related messages. Then, the <em>Aggregator</em> publishes a single message distilled from the individual messages." </p> <footer><cite><a href="/ref/eip">Enterprise Integration Patterns</a></cite></footer> </blockquote> <p> The example that I'll give here is simplified and mostly focuses on how to use the State monad to implement the desired behaviour. The book Enterprise Integration Patterns starts with a simple example where messages arrive with a <a href="https://www.enterpriseintegrationpatterns.com/CorrelationIdentifier.html">correlation ID</a> as an integer. The message payload is also a an integer, just to keep things simple. The Aggregator should only publish an aggregated message once it has received three correlated messages. </p> <p> Using the State monad, you could implement an Aggregator like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Aggregator</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;IState&lt;IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;,&nbsp;Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;correlationId; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Aggregator</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">correlationId</span>,&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">value</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.correlationId&nbsp;=&nbsp;correlationId; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.value&nbsp;=&nbsp;value; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;,&nbsp;IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Run</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;&nbsp;<span style="color:#1f377f;">state</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(state.TryGetValue(correlationId,&nbsp;<span style="color:blue;">out</span>&nbsp;var&nbsp;<span style="color:#1f377f;">coll</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(coll.Count&nbsp;==&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">retVal</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tuple.Create(coll.ElementAt(0),&nbsp;coll.ElementAt(1),&nbsp;value); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">newState</span>&nbsp;=&nbsp;state.Remove(correlationId); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Tuple.Create(retVal.ToMaybe(),&nbsp;newState); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">newColl</span>&nbsp;=&nbsp;coll.Append(value); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">newState</span>&nbsp;=&nbsp;state.Replace(correlationId,&nbsp;newColl); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Tuple.Create(<span style="color:blue;">new</span>&nbsp;Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;(),&nbsp;newState); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">newColl</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;value&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">newState</span>&nbsp;=&nbsp;state.Add(correlationId,&nbsp;newColl); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Tuple.Create(<span style="color:blue;">new</span>&nbsp;Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;(),&nbsp;newState); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The <code>Aggregator</code> class implements the <code>IState&lt;S, T&gt;</code> interface. The full generic type is something of a mouthful, though. </p> <p> The state type (<code>S</code>) is <code>IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;</code> - in other words, a dictionary of collections. Each entry in the dictionary is keyed by a correlation ID. Each value is a collection of messages that belong to that ID. Keep in mind that, in order to keep the example simple, each message is just a number (an <code>int</code>). </p> <p> The value to produce (<code>T</code>) is <code>Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;</code>. This code example uses <a href="/2022/04/25/the-maybe-monad">this implementation of the Maybe monad</a>. The value produced may or may not be empty, depending on whether the Aggregator has received all three required messages in order to produce an aggregated message. Again, for simplicity, the aggregated message is just a triple (a three-tuple). </p> <p> The <code>Run</code> method starts by querying the <code>state</code> dictionary for an entry that corresponds to the <code>correlationId</code>. This entry may or may not be present. If the message is the first in a series of three, there will be no entry, but if it's the second or third message, the entry will be present. </p> <p> In that case, the <code>Run</code> method checks the <code>Count</code> of the collection. If the <code>Count</code> is <code>2</code>, it means that two other messages with the same <code>correlationId</code> was already received. This means that the <code>Aggregator</code> is now handling the third and final message. Thus, it creates the <code>retVal</code> tuple, removes the entry from the dictionary to create the <code>newState</code>, and returns both. </p> <p> If the <code>state</code> contains an entry for the <code>correlationId</code>, but the <code>Count</code> isn't <code>2</code>, the <code>Run</code> method updates the entry by appending the <code>value</code>, updating the state to <code>newState</code>, and returns that together with an empty Maybe value. </p> <p> Finally, if there is no entry for the <code>correlationId</code>, the <code>Run</code> method creates a new collection containing only the <code>value</code>, adds it to the <code>state</code> dictionary, and returns the <code>newState</code> together with an empty Maybe value. </p> <h3 id="f8ed5ee8f1eb4f1499e283f2383962e9"> Message handler <a href="#f8ed5ee8f1eb4f1499e283f2383962e9" title="permalink">#</a> </h3> <p> A message handler could be a background service that receives messages from a durable queue, a REST endpoint, or based on some other technology. </p> <p> After it receives a message, a message handler would create a new instance of the <code>Aggregator</code>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">a</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Aggregator(msg.CorrelationId,&nbsp;msg.Value);</pre> </p> <p> Since <code>Aggregator</code> implements the <code>IState&lt;S, T&gt;</code> interface, the object <code>a</code> represents a stateful computation. A message handler might keep the current state in memory, or rehydrate it from some persistent storage system. Keep in mind that the state must be of the type <code>IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;</code>. Wherever it comes from, assume that this state is a variable called <code>s</code> (for <em>state</em>). </p> <p> The message handler can now <code>Run</code> the stateful computation by supplying <code>s</code>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&nbsp;a.Run(s);</pre> </p> <p> The result is a tuple where the first item is a Maybe value, and the second item is the new state. </p> <p> The message handler can now publish the triple if the Maybe value is populated. In any case, it can update the 'current' state with the new state. That's a nice little <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>. </p> <p> Notice how this design is different from a typical object-oriented solution. In object-oriented programming, you'd typically have an object than contains the state and then receives the run-time value as input to a method that might then mutate the state. <em>Data with behaviour</em>, as it's sometimes characterised. </p> <p> The State-based computation turns such a design on its head. The computation closes over the run-time values, and the state is supplied as an argument to the <code>Run</code> method. This is an example of the shift of perspective often required to think functionally, rather than object-oriented. <em>That's</em> why it takes time learning Functional Programming (FP); it's not about syntax. It's a different way to think. </p> <p> An object like the above <code>a</code> seems almost frivolous, since it's going to have a short lifetime. Calling code will create it only to call its <code>Run</code> method and then let it go out of scope to be garbage-collected. </p> <p> Of course, in a language more attuned to FP like <a href="https://www.haskell.org">Haskell</a>, it's a different story: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;h&nbsp;=&nbsp;handle&nbsp;(corrId&nbsp;msg)&nbsp;(val&nbsp;msg)</pre> </p> <p> Instead of creating an object using a constructor, you only pass the message values to a function called <code>handle</code>. The return value <code>h</code> is a State value that an overall message handler can then later run with a state <code>s</code>: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;(m,&nbsp;ns)&nbsp;=&nbsp;runState&nbsp;h&nbsp;s</pre> </p> <p> The return value is a tuple where <code>m</code> is the Maybe value that may or may not contain the aggregated message; <code>ns</code> is the new state. </p> <h3 id="6904a24cf4a242c98b6732b5fcb8a3d1"> Is this better? <a href="#6904a24cf4a242c98b6732b5fcb8a3d1" title="permalink">#</a> </h3> <p> Is this approach to state mutation better than the default kind of state mutation possible with most languages (including C#)? Why make things so complicated? </p> <p> There's more than one answer. First, in a language like Haskell, state mutation is in general not possible. While you <em>can</em> do state mutation with <a href="/2020/06/08/the-io-container">the IO container</a> in Haskell, this sets you completely free. You don't want to be free, because with freedom comes innumerable ways to shoot yourself in the foot. <a href="https://www.dotnetrocks.com/?show=1542">Constraints liberate</a>. </p> <p> While the IO monad allows uncontrolled state mutation (together with all sorts of other impure actions), the State monad constrains itself and callers to only one type of apparent mutation. The type of the state being 'mutated' is visible in the type system, and that's the only type of value you can 'mutate' (in Haskell, that is). </p> <p> The State monad uses the type system to clearly communicate what the type of state is. Given a language like Haskell, or otherwise given sufficient programming discipline, you can tell from an object's type exactly what to expect. </p> <p> This also goes a long way to explain why monads are such an important concept in Functional Programming. When discussing FP, a common question is: <em>How do you perform side effects?</em> The answer, as may be already implied by this article, is that you use <a href="/2022/03/28/monads">monads</a>. The State monad for local state mutation, and the IO monad for 'global' side effects. </p> <h3 id="598094a7efc64e2685cf6196415bca01"> Get <a href="#598094a7efc64e2685cf6196415bca01" title="permalink">#</a> </h3> <p> Clearly you can write an implementation of <code>IState&lt;S, T&gt;</code> like the above <code>Aggregator</code> class. Must we always write a class that implements the interface in order to work within the State monad? </p> <p> Monads are all about composition. Usually, you can compose even complex behaviour from smaller building blocks. Just consider the <a href="/2022/04/19/the-list-monad">list monad</a>, which in C# is epitomised by the <code>IEnumerable&lt;T&gt;</code> interface. You can write quite complex logic using the building blocks of <a href="https://docs.microsoft.com/dotnet/api/system.linq.enumerable.where">Where</a>, <a href="https://docs.microsoft.com/dotnet/api/system.linq.enumerable.select">Select</a>, <a href="https://docs.microsoft.com/dotnet/api/system.linq.enumerable.aggregate">Aggregate</a>, <a href="https://docs.microsoft.com/dotnet/api/system.linq.enumerable.zip">Zip</a>, etcetera. </p> <p> Likewise, we should expect that to be the case with the State monad, and it is so. The useful extra combinators are <em>get</em> and <em>put</em>. </p> <p> The <em>get</em> function enables a composition to retrieve the current state. Given the <code>IState&lt;S, T&gt;</code> interface, you can implement it like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;S,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Get</span>&lt;<span style="color:#2b91af;">S</span>&gt;() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;GetState&lt;S&gt;(); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">GetState</span>&lt;<span style="color:#2b91af;">S</span>&gt;&nbsp;:&nbsp;IState&lt;S,&nbsp;S&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;S,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Run</span>(S&nbsp;<span style="color:#1f377f;">state</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Tuple.Create(state,&nbsp;state); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The <code>Get</code> function represents a stateful computation that copies the <code>state</code> over to the 'value' dimension, so to speak. Notice that the return type is <code>IState&lt;S,&nbsp;S&gt;</code>. Copying the state over to the position of the <code>T</code> generic type means that it becomes accessible to the expressions that run inside of <code>Select</code> and <code>SelectMany</code>. </p> <p> You'll see an example once I rewrite the above <code>Aggregator</code> to be entirely based on composition, but in order to do that, I also need the <em>put</em> function. </p> <h3 id="224229b040ea417097a9de2ed1247953"> Put <a href="#224229b040ea417097a9de2ed1247953" title="permalink">#</a> </h3> <p> The <em>put</em> function enables you to write a new state value to the underlying state dimension. The implementation in the current code base looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;S,&nbsp;Unit&gt;&nbsp;<span style="color:#74531f;">Put</span>&lt;<span style="color:#2b91af;">S</span>&gt;(S&nbsp;<span style="color:#1f377f;">s</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;PutState&lt;S&gt;(s); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PutState</span>&lt;<span style="color:#2b91af;">S</span>&gt;&nbsp;:&nbsp;IState&lt;S,&nbsp;Unit&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;S&nbsp;s; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">PutState</span>(S&nbsp;<span style="color:#1f377f;">s</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.s&nbsp;=&nbsp;s; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Tuple&lt;Unit,&nbsp;S&gt;&nbsp;<span style="color:#74531f;">Run</span>(S&nbsp;<span style="color:#1f377f;">state</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Tuple.Create(Unit.Default,&nbsp;s); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This implementation uses a <code>Unit</code> value to represent <code>void</code>. As usual, we have the problem in C-based languages that <code>void</code> isn't a value, but fortunately, <a href="/2018/01/15/unit-isomorphisms">unit is isomorphic to void</a>. </p> <p> Notice that the <code>Run</code> method ignores the current <code>state</code> and instead replaces it with the new state <code>s</code>. </p> <h3 id="c347a208b9524a1b85756a66cf53999c"> Look, no classes! <a href="#c347a208b9524a1b85756a66cf53999c" title="permalink">#</a> </h3> <p> The <code>Get</code> and <code>Put</code> functions are enough that we can now rewrite the functionality currently locked up in the <code>Aggregator</code> class. Instead of having to define a new <code>class</code> for that purpose, it's possible to compose our way to the same functionality by writing a function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;,&nbsp;Maybe&lt;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">int</span>&gt;&gt;&gt; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#74531f;">Aggregate</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">correlationId</span>,&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">value</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;state&nbsp;<span style="color:blue;">in</span>&nbsp;State.Get&lt;IReadOnlyDictionary&lt;<span style="color:blue;">int</span>,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">int</span>&gt;&gt;&gt;() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;mcoll&nbsp;=&nbsp;state.TryGetValue(correlationId) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;retVal&nbsp;=&nbsp;<span style="color:blue;">from</span>&nbsp;coll&nbsp;<span style="color:blue;">in</span>&nbsp;mcoll.Where(<span style="color:#1f377f;">c</span>&nbsp;=&gt;&nbsp;c.Count&nbsp;==&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;Tuple.Create(coll.ElementAt(0),&nbsp;coll.ElementAt(1),&nbsp;value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;newState&nbsp;=&nbsp;retVal &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Select(<span style="color:#1f377f;">_</span>&nbsp;=&gt;&nbsp;state.Remove(correlationId)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.GetValueOrFallback( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;state.Replace( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;correlationId, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mcoll &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Select(<span style="color:#1f377f;">coll</span>&nbsp;=&gt;&nbsp;coll.Append(value)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.GetValueOrFallback(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;value&nbsp;}))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;_&nbsp;<span style="color:blue;">in</span>&nbsp;State.Put(newState) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;retVal; }</pre> </p> <p> Okay, I admit that there's a hint of <a href="https://en.wikipedia.org/wiki/Code_golf">code golf</a> over this. It's certainly not <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C#. To be clear, I'm not endorsing this style of C#; I'm only showing it to explain the abstraction offered by the State monad. <a href="/2019/03/18/the-programmer-as-decision-maker">Adopt such code at your own peril</a>. </p> <p> The first observation to be made about this code example is that it's written entirely in query syntax. There's a good reason for that. Query syntax is syntactic sugar on top of <code>SelectMany</code>, so you could, conceivably, also write the above expression using method call syntax. However, in order to make early values available to later expressions, you'd have to pass a lot of tuples around. For example, the above expression makes repeated use of <code>mcoll</code>, so had you been using method call syntax instead of query syntax, you would have had to pass that value on to subsequent computations as one item in a tuple. Not impossible, but awkward. With query syntax, all values remain in scope so that you can refer to them later. </p> <p> The expression starts by using <code>Get</code> to get the current state. The <code>state</code> variable is now available in the rest of the expression. </p> <p> The <code>state</code> is a dictionary, so the next step is to query it for an entry that corresponds to the <code>correlationId</code>. I've used an overload of <code>TryGetValue</code> that returns a Maybe value, which also explains (I hope) the <code>m</code> prefix of <code>mcoll</code>. </p> <p> Next, the expression filters <code>mcoll</code> and creates a triple if the <code>coll</code> has a <code>Count</code> of two. Notice that the nested query syntax expression (<code>from...select</code>) isn't running in the State monad, but rather in the <a href="/2022/04/25/the-maybe-monad">Maybe monad</a>. The result, <code>retVal</code>, is another Maybe value. </p> <p> That takes care of the 'return value', but we also need to calculate the new state. This happens in a somewhat roundabout way. The reason that it's not more straightforward is that C# query syntax doesn't allow branching (apart from the ternary <code>?..:</code> operator) and (this version of the language, at least) has weak pattern-matching abilities. </p> <p> Instead, it uses <code>retVal</code> and <code>mcoll</code> as indicators of how to update the state. If <code>retVal</code> is populated, it means that the <code>Aggregate</code> computation will return a triple, in which case it must <code>Remove</code> the entry from the state dictionary. On the other hand, if that's not the case, it must update the entry. Again, there are two options. If <code>mcoll</code> was already populated, it should be updated by appending the <code>value</code>. If not, a new entry containing only the <code>value</code> should be added. </p> <p> Finally, the expression uses <code>Put</code> to save the new state, after which it returns <code>retVal</code>. </p> <p> While this is far from idiomatic C# code, the point is that you can <em>compose</em> your way to the desired behaviour. You don't have to write a new class. Not that this is necessarily an improvement in C#. I'm mostly stating this to highlight a difference in philosophy. </p> <p> Of course, this is all much more elegant in Haskell, where the same functionality is as terse as this: </p> <p> <pre><span style="color:#2b91af;">handle</span>&nbsp;::&nbsp;(<span style="color:blue;">Ord</span>&nbsp;k,&nbsp;<span style="color:blue;">MonadState</span>&nbsp;(<span style="color:blue;">Map</span>&nbsp;k&nbsp;[a])&nbsp;m)&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;k&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;m&nbsp;(<span style="color:#2b91af;">Maybe</span>&nbsp;(a,&nbsp;a,&nbsp;a)) handle&nbsp;correlationId&nbsp;value&nbsp;=&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;m&nbsp;&lt;-&nbsp;get &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;(retVal,&nbsp;newState)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">case</span>&nbsp;Map.<span style="color:blue;">lookup</span>&nbsp;correlationId&nbsp;m&nbsp;<span style="color:blue;">of</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just&nbsp;[x,&nbsp;y]&nbsp;-&gt;&nbsp;(Just&nbsp;(x,&nbsp;y,&nbsp;value),&nbsp;Map.delete&nbsp;correlationId&nbsp;m) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just&nbsp;&nbsp;_&nbsp;-&gt;&nbsp;(Nothing,&nbsp;Map.adjust&nbsp;(++&nbsp;[value])&nbsp;correlationId&nbsp;m) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nothing&nbsp;-&gt;&nbsp;(Nothing,&nbsp;Map.insert&nbsp;correlationId&nbsp;[value]&nbsp;m) &nbsp;&nbsp;put&nbsp;newState &nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;retVal</pre> </p> <p> Notice that this implementation also makes use of <code>get</code> and <code>put</code>. </p> <h3 id="2263fd914af84e0082c77a5ad2b9afe6"> Modify <a href="#2263fd914af84e0082c77a5ad2b9afe6" title="permalink">#</a> </h3> <p> The <code>Get</code> and <code>Put</code> functions are basic functions based on the State monad abstraction. These two functions, again, can be used to define some secondary helper functions, whereof <code>Modify</code> is one: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;S,&nbsp;Unit&gt;&nbsp;<span style="color:#74531f;">Modify</span>&lt;<span style="color:#2b91af;">S</span>&gt;(Func&lt;S,&nbsp;S&gt;&nbsp;<span style="color:#1f377f;">modify</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Get&lt;S&gt;().SelectMany(<span style="color:#1f377f;">s</span>&nbsp;=&gt;&nbsp;Put(modify(s))); }</pre> </p> <p> It wasn't required for the above <code>Aggregate</code> function, but here's a basic unit test that demonstrates how it works: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ModifyExample</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">x</span>&nbsp;=&nbsp;State.Modify((<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">i</span>)&nbsp;=&gt;&nbsp;i&nbsp;+&nbsp;1); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;x.Run(1); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(2,&nbsp;actual.Item2); }</pre> </p> <p> It can be useful if you need to perform an 'atomic' state modification. For a realistic Haskell example, you may want to refer to my article <a href="/2019/03/11/an-example-of-state-based-testing-in-haskell">An example of state-based testing in Haskell</a>. </p> <h3 id="b1e1ba440a4f4b6aad12bc6721ba6f8b"> Gets <a href="#b1e1ba440a4f4b6aad12bc6721ba6f8b" title="permalink">#</a> </h3> <p> Another occasionally useful second-order helper function is <code>Gets</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IState&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#74531f;">Gets</span>&lt;<span style="color:#2b91af;">S</span>,&nbsp;<span style="color:#2b91af;">T</span>&gt;(Func&lt;S,&nbsp;T&gt;&nbsp;<span style="color:#1f377f;">selector</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;Get&lt;S&gt;().Select(selector); }</pre> </p> <p> This function can be useful as a combinator if you need just a projection of the current state, instead of the whole state. </p> <p> Here's another basic unit test as an example: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">GetsExample</span>() { &nbsp;&nbsp;&nbsp;&nbsp;IState&lt;<span style="color:blue;">string</span>,&nbsp;<span style="color:blue;">int</span>&gt;&nbsp;<span style="color:#1f377f;">x</span>&nbsp;=&nbsp;State.Gets((<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">s</span>)&nbsp;=&gt;&nbsp;s.Length); &nbsp;&nbsp;&nbsp;&nbsp;Tuple&lt;<span style="color:blue;">int</span>,&nbsp;<span style="color:blue;">string</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;x.Run(<span style="color:#a31515;">&quot;bar&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(Tuple.Create(3,&nbsp;<span style="color:#a31515;">&quot;bar&quot;</span>),&nbsp;actual); }</pre> </p> <p> While the above Aggregator example didn't require <code>Modify</code> or <code>Gets</code>, I wanted to include them here for completeness sake. </p> <h3 id="05a2046a71e94b85a3f5445df9669800"> F# <a href="#05a2046a71e94b85a3f5445df9669800" title="permalink">#</a> </h3> <p> Most of the code shown in this article has been C#, with the occasional Haskell code. You can also implement the State monad, as well as the helper methods, in <a href="https://fsharp.org">F#</a>, where it'd feel more natural to dispense with interfaces and instead work directly with functions. To make things a little clearer, you may want to define a type alias: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;State&lt;&#39;a,&nbsp;&#39;s&gt;&nbsp;=&nbsp;(&#39;s&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;&#39;a&nbsp;*&nbsp;&#39;s)</pre> </p> <p> You can now define a <code>State</code> module that works directly with that kind of function: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;State&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;run&nbsp;state&nbsp;(f&nbsp;:&nbsp;State&lt;_,&nbsp;_&gt;)&nbsp;=&nbsp;f&nbsp;state &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;lift&nbsp;x&nbsp;state&nbsp;=&nbsp;x,&nbsp;state &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;map&nbsp;f&nbsp;x&nbsp;state&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&#39;,&nbsp;newState&nbsp;=&nbsp;run&nbsp;state&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f&nbsp;x&#39;,&nbsp;newState &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;bind&nbsp;(f&nbsp;:&nbsp;&#39;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;State&lt;&#39;b,&nbsp;&#39;s&gt;)&nbsp;(x&nbsp;:&nbsp;State&lt;&#39;a,&nbsp;&#39;s&gt;)&nbsp;state&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&#39;,&nbsp;newState&nbsp;=&nbsp;run&nbsp;state&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;run&nbsp;newState&nbsp;(f&nbsp;x&#39;) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;get&nbsp;state&nbsp;=&nbsp;state,&nbsp;state &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;put&nbsp;newState&nbsp;_&nbsp;=&nbsp;(),&nbsp;newState &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;modify&nbsp;f&nbsp;=&nbsp;get&nbsp;|&gt;&nbsp;map&nbsp;f&nbsp;|&gt;&nbsp;bind&nbsp;put</pre> </p> <p> This is code I originally wrote for <a href="https://codereview.stackexchange.com/a/139652/3878">a Code Review answer</a>. You can go there to see all the details, as well as a motivating example. </p> <p> I see that I never got around to add a <code>gets</code> function... I'll leave that as an exercise. </p> <p> In C#, I've based the example on an interface (<code>IState&lt;S, T&gt;</code>), but it would also be possible to implement the State monad as extension methods on <code>Func&lt;S, Tuple&lt;T, S&gt;&gt;</code>. Try it! It might be another good exercise. </p> <h3 id="994529b38af241b9ba6090f5d670bcc8"> Conclusion <a href="#994529b38af241b9ba6090f5d670bcc8" title="permalink">#</a> </h3> <p> The State monad usually comes with a few helper functions: <em>get</em>, <em>put</em>, <em>modify</em>, and <em>gets</em>. They can be useful as combinators you can use to compose a stateful combination from smaller building blocks, just like you can use LINQ to compose complex queries over data. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Test Double clocks https://blog.ploeh.dk/2022/06/27/test-double-clocks 2022-06-27T05:44:00+00:00 Mark Seemann <div id="post"> <p> <em>A short exploration of replacing the system clock with Test Doubles.</em> </p> <p> In a comment to my article <a href="/2022/05/23/waiting-to-never-happen">Waiting to never happen</a>, <a href="https://github.com/ladeak">Laszlo</a> asks: </p> <blockquote> <p> "Why have you decided to make the date of the reservation relative to the SystemClock, and not the other way around? Would it be more deterministic to use a faked system clock instead?" </p> <footer><cite><a href="/2022/05/23/waiting-to-never-happen#c0b2e0bd555b5d5c5555c60bf11bff69">Laszlo</a></cite></footer> </blockquote> <p> The short answer is that I hadn't thought of the alternative. Not in this context, at least. </p> <p> It's a question worth exploring, which I will now proceed to do. </p> <h3 id="d7283861d9494c478b2bd8bb90218208"> Why IClock? <a href="#d7283861d9494c478b2bd8bb90218208" title="permalink">#</a> </h3> <p> The article in question discusses a unit test, which ultimately arrives at this: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;<span style="color:#74531f;">ChangeDateToSoldOutDate</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r1</span>&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Some.Reservation.WithDate(DateTime.Now.AddDays(8).At(20,&nbsp;15)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r2</span>&nbsp;=&nbsp;r1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithId(Guid.NewGuid()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.TheDayAfter() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithQuantity(10); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">db</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;FakeDatabase(); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r1); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r2); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">sut</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ReservationsController( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;SystemClock(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;InMemoryRestaurantDatabase(Grandfather.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">dto</span>&nbsp;=&nbsp;r1.WithDate(r2.At).ToDto(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Put(r1.Id.ToString(<span style="color:#a31515;">&quot;N&quot;</span>),&nbsp;dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">oRes</span>&nbsp;=&nbsp;Assert.IsAssignableFrom&lt;ObjectResult&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;StatusCodes.Status500InternalServerError, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;oRes.StatusCode); }</pre> </p> <p> The keen reader may notice that the test passes a <code><span style="color:blue;">new</span>&nbsp;SystemClock()</code> to the <code>sut</code>. In case you're wondering what that is, here's the definition: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>&nbsp;:&nbsp;IClock { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;DateTime&nbsp;<span style="color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;DateTime.Now; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> While it should be possible to extrapolate the <code>IClock</code> interface from this code snippet, here it is for the sake of completeness: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IClock</span> { &nbsp;&nbsp;&nbsp;&nbsp;DateTime&nbsp;<span style="color:#74531f;">GetCurrentDateTime</span>(); }</pre> </p> <p> Since such an interface exists, why not use it in unit tests? </p> <p> That's possible, but I think it's worth highlighting what motivated this interface in the first place. If you're used to a certain style of test-driven development (TDD), you may think that interfaces exist in order to support TDD. They may. That's how I did TDD 15 years ago, but <a href="/2019/02/18/from-interaction-based-to-state-based-testing">not how I do it today</a>. </p> <p> The motivation for the <code>IClock</code> interface is another. It's there because the system clock is a source of impurity, just like random number generators, database queries, and web service invocations. In order to support <a href="/2020/03/23/repeatable-execution">repeatable execution</a>, it's useful to log the inputs and outputs of impure actions. This includes the system clock. </p> <p> The <code>IClock</code> interface doesn't exist in order to support unit testing, but in order to enable logging via the <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorator</a> pattern: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LoggingClock</span>&nbsp;:&nbsp;IClock { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">LoggingClock</span>(ILogger&lt;LoggingClock&gt;&nbsp;<span style="color:#1f377f;">logger</span>,&nbsp;IClock&nbsp;<span style="color:#1f377f;">inner</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Logger&nbsp;=&nbsp;logger; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Inner&nbsp;=&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;ILogger&lt;LoggingClock&gt;&nbsp;Logger&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IClock&nbsp;Inner&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;DateTime&nbsp;<span style="color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">output</span>&nbsp;=&nbsp;Inner.GetCurrentDateTime(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Logger.LogInformation( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;{method}()&nbsp;=&gt;&nbsp;{output}&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nameof(GetCurrentDateTime), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;output); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;output; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> All code in this article originates from the code base that accompanies <a href="/code-that-fits-in-your-head">Code That Fits in Your Head</a>. </p> <p> The web application is configured to decorate the <code>SystemClock</code> with the <code>LoggingClock</code>: </p> <p> <pre>services.AddSingleton&lt;IClock&gt;(<span style="color:#1f377f;">sp</span>&nbsp;=&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">logger</span>&nbsp;=&nbsp;sp.GetService&lt;ILogger&lt;LoggingClock&gt;&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;LoggingClock(logger,&nbsp;<span style="color:blue;">new</span>&nbsp;SystemClock()); });</pre> </p> <p> While the motivation for the <code>IClock</code> interface wasn't to support testing, now that it exists, would it be useful for unit testing as well? </p> <h3 id="3bf52db0c892409dbc372a83293a8992"> A Stub clock <a href="#3bf52db0c892409dbc372a83293a8992" title="permalink">#</a> </h3> <p> As a first effort, we might try to add a <a href="http://xunitpatterns.com/Test%20Stub.html">Stub</a> clock: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ConstantClock</span>&nbsp;:&nbsp;IClock { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;DateTime&nbsp;dateTime; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ConstantClock</span>(DateTime&nbsp;<span style="color:#1f377f;">dateTime</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.dateTime&nbsp;=&nbsp;dateTime; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;This&nbsp;default&nbsp;value&nbsp;is&nbsp;more&nbsp;or&nbsp;less&nbsp;arbitrary.&nbsp;I&nbsp;chose&nbsp;it&nbsp;as&nbsp;the&nbsp;date</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;and&nbsp;time&nbsp;I&nbsp;wrote&nbsp;these&nbsp;lines&nbsp;of&nbsp;code,&nbsp;which&nbsp;also&nbsp;has&nbsp;the&nbsp;implication</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;that&nbsp;it&nbsp;was&nbsp;immediately&nbsp;a&nbsp;time&nbsp;in&nbsp;the&nbsp;past.&nbsp;The&nbsp;actual&nbsp;value&nbsp;is,</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;however,&nbsp;irrelevant.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IClock&nbsp;Default&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;ConstantClock(<span style="color:blue;">new</span>&nbsp;DateTime(2022,&nbsp;6,&nbsp;19,&nbsp;9,&nbsp;25,&nbsp;0)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;DateTime&nbsp;<span style="color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;dateTime; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This implementation always returns the same date and time. I called it <code>ConstantClock</code> for that reason. </p> <p> It's trivial to replace the <code>SystemClock</code> with a <code>ConstantClock</code> in the above test: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;<span style="color:#74531f;">ChangeDateToSoldOutDate</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">clock</span>&nbsp;=&nbsp;ConstantClock.Default; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r1</span>&nbsp;=&nbsp;Some.Reservation.WithDate( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;clock.GetCurrentDateTime().AddDays(8).At(20,&nbsp;15)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r2</span>&nbsp;=&nbsp;r1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithId(Guid.NewGuid()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.TheDayAfter() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithQuantity(10); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">db</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;FakeDatabase(); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r1); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r2); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">sut</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;ReservationsController( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;clock, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;InMemoryRestaurantDatabase(Grandfather.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">dto</span>&nbsp;=&nbsp;r1.WithDate(r2.At).ToDto(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Put(r1.Id.ToString(<span style="color:#a31515;">&quot;N&quot;</span>),&nbsp;dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">oRes</span>&nbsp;=&nbsp;Assert.IsAssignableFrom&lt;ObjectResult&gt;(actual); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;StatusCodes.Status500InternalServerError, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;oRes.StatusCode); }</pre> </p> <p> As you can see, however, it doesn't seem to be enabling any simplification of the test. It still needs to establish that <code>r1</code> and <code>r2</code> relates to each other as required by the test case, as well as establish that they are valid reservations in the future. </p> <p> You may protest that this is straw man argument, and that it would make the test both simpler and more readable if it would, instead, use explicit, hard-coded values. That's a fair criticism, so I'll get back to that later. </p> <h3 id="58fe168e8266401990d62641020db9ac"> Fragility <a href="#58fe168e8266401990d62641020db9ac" title="permalink">#</a> </h3> <p> Before examining the above criticism, there's something more fundamental that I want to get out of the way. I find a Stub clock icky. </p> <p> It works in this case, but may lead to fragile tests. What happens, for example, if another programmer comes by and adds code like this to the System Under Test (SUT)? </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">now</span>&nbsp;=&nbsp;Clock.GetCurrentDateTime(); <span style="color:green;">//&nbsp;Sabotage:</span> <span style="color:#8f08c4;">while</span>&nbsp;(Clock.GetCurrentDateTime()&nbsp;-&nbsp;now&nbsp;&lt;&nbsp;TimeSpan.FromMilliseconds(1)) {&nbsp;}</pre> </p> <p> As the comment suggests, in this case it's pure sabotage. I don't think that anyone would deliberately do something like this. This code snippet even sits in an asynchronous method, and in .NET 'everyone' knows that if you want to suspend execution in an asynchronous method, you should use <a href="https://docs.microsoft.com/dotnet/api/system.threading.tasks.task.delay">Task.Delay</a>. I rather intend this code snippet to indicate that keeping time constant, as <code>ConstantClock</code> does, can be fatal. </p> <p> If someone comes by and attempts to implement any kind of time-sensitive logic based on an injected <code>IClock</code>, the consequences could be dire. With the above sabotage, for example, the test hangs forever. </p> <p> When I originally <a href="/2021/01/11/waiting-to-happen">refactored time-sensitive tests</a>, it was because I didn't appreciate having such ticking bombs lying around. A <code>ConstantClock</code> isn't <em>ticking</em> (that's the problem), but it still seems like a booby trap. </p> <h3 id="d58309147ef6450589ed0b5c69b33b30"> Offset clock <a href="#d58309147ef6450589ed0b5c69b33b30" title="permalink">#</a> </h3> <p> It seems intuitive that a clock that doesn't go isn't very useful. Perhaps we can address that problem by setting the clock back. Not just a few hours, but days or years: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">OffsetClock</span>&nbsp;:&nbsp;IClock { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;TimeSpan&nbsp;offset; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">OffsetClock</span>(DateTime&nbsp;<span style="color:#1f377f;">origin</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;offset&nbsp;=&nbsp;DateTime.Now&nbsp;-&nbsp;origin; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IClock&nbsp;<span style="color:#74531f;">Start</span>(DateTime&nbsp;<span style="color:#1f377f;">at</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;OffsetClock(at); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;This&nbsp;default&nbsp;value&nbsp;is&nbsp;more&nbsp;or&nbsp;less&nbsp;arbitrary.&nbsp;I&nbsp;just&nbsp;picked&nbsp;the&nbsp;same</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;date&nbsp;and&nbsp;time&nbsp;as&nbsp;ConstantClock&nbsp;(which&nbsp;see).</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IClock&nbsp;Default&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Start(at:&nbsp;<span style="color:blue;">new</span>&nbsp;DateTime(2022,&nbsp;6,&nbsp;19,&nbsp;9,&nbsp;25,&nbsp;0)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;DateTime&nbsp;<span style="color:#74531f;">GetCurrentDateTime</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;DateTime.Now&nbsp;-&nbsp;offset; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> An <code>OffsetClock</code> object starts ticking as soon as it's created, but it ticks at the same pace as the system clock. Time still passes. Rather than a Stub, I think that this implementation qualifies as a <a href="http://xunitpatterns.com/Fake%20Object.html">Fake</a>. </p> <p> Using it in a test is as easy as using the <code>ConstantClock</code>: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;<span style="color:#74531f;">ChangeDateToSoldOutDate</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">clock</span>&nbsp;=&nbsp;OffsetClock.Default; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r1</span>&nbsp;=&nbsp;Some.Reservation.WithDate( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;clock.GetCurrentDateTime().AddDays(8).At(20,&nbsp;15)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r2</span>&nbsp;=&nbsp;r1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithId(Guid.NewGuid()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.TheDayAfter() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithQuantity(10); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">db</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;FakeDatabase(); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r1); &nbsp;&nbsp;&nbsp;&nbsp;db.Grandfather.Add(r2); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f37