ploeh blog https://blog.ploeh.dk danish software design en-us Mark Seemann Thu, 02 Feb 2023 19:08:12 UTC Thu, 02 Feb 2023 19:08:12 UTC Built-in alternatives to applicative assertions https://blog.ploeh.dk/2023/01/30/built-in-alternatives-to-applicative-assertions/ Mon, 30 Jan 2023 08:08:00 UTC <div id="post"> <p> <em>Why make things so complicated?</em> </p> <p> Several readers reacted to my small article series on <a href="/2022/11/07/applicative-assertions">applicative assertions</a>, pointing out that error-collecting assertions are already supported in more than one unit-testing framework. </p> <blockquote> <p> "In the Java world this seems similar to the result gained by Soft Assertions in AssertJ. <a href="https://assertj.github.io/doc/#assertj-core-soft-assertions">https://assertj.github.io/doc/#assertj-c...</a> if you’re after a target for functionality (without the adventures through monad land)" </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1597190184134590464">Josh McK</a></cite></footer> </blockquote> <p> While I'm not familiar with the details of Java unit-testing frameworks, the situation is similar in .NET, it turns out. </p> <blockquote> <p> "Did you know there is Assert.Multiple in NUnit and now also in xUnit .Net? It seems to have quite an overlap with what you're doing here. </p> <p> "For a quick overview, I found this blogpost helpful: <a href="https://www.thomasbogholm.net/2021/11/25/xunit-2-4-2-pre-multiple-asserts-in-one-test/">https://www.thomasbogholm.net/2021/11/25/xunit-2-4-2-pre-multiple-asserts-in-one-test/</a>" </p> <footer><cite><a href="https://twitter.com/DoCh_Dev/status/1597158737357459456">DoCh_Dev</a></cite></footer> </blockquote> <p> I'm not surprised to learn that something like this exists, but let's take a quick look. </p> <h3 id="d7c0c78093084c08aa22ccaa7b86cb8a"> NUnit Assert.Multiple <a href="#d7c0c78093084c08aa22ccaa7b86cb8a" title="permalink">#</a> </h3> <p> Let's begin with <a href="https://nunit.org/">NUnit</a>, as this seems to be the first .NET unit-testing framework to support error-collecting assertions. As a beginning, the <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">documentation example</a> works as it's supposed to: </p> <p> <pre>[Test] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ComplexNumberTest() { &nbsp;&nbsp;&nbsp;&nbsp;ComplexNumber&nbsp;result&nbsp;=&nbsp;SomeCalculation(); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple(()&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.AreEqual(5.2,&nbsp;result.RealPart,&nbsp;<span style="color:#a31515;">&quot;Real&nbsp;part&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.AreEqual(3.9,&nbsp;result.ImaginaryPart,&nbsp;<span style="color:#a31515;">&quot;Imaginary&nbsp;part&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> When you run the test, it fails (as expected) with this error message: </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures or warnings in test: &nbsp;&nbsp;&nbsp;&nbsp;1) Real part &nbsp;&nbsp;&nbsp;&nbsp;Expected: 5.2000000000000002d &nbsp;&nbsp;&nbsp;&nbsp;But was: 5.0999999999999996d &nbsp;&nbsp;&nbsp;&nbsp;2) Imaginary part &nbsp;&nbsp;&nbsp;&nbsp;Expected: 3.8999999999999999d &nbsp;&nbsp;&nbsp;&nbsp;But was: 4.0d</pre> </p> <p> That seems to work well enough, but how does it actually work? I'm not interested in reading the NUnit source code - after all, the concept of <a href="/encapsulation-and-solid">encapsulation</a> is that one should be able to make use of the capabilities of an object without knowing all implementation details. Instead, I'll guess: Perhaps <code>Assert.Multiple</code> executes the code block in a <code>try/catch</code> block and collects the various exceptions thrown by the nested assertions. </p> <p> Does it catch all exception types, or only a subset? </p> <p> Let's try with the kind of composed assertion that I <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previously investigated</a>: </p> <p> <pre>[Test] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple(()&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assert.That(getResp.StatusCode,&nbsp;Is.EqualTo(HttpStatusCode.NotFound)); &nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> This test fails (again, as expected). What's the error message? </p> <p> <pre>Message: &nbsp;&nbsp;System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;&nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> (I've wrapped the result over multiple lines for readability. The <code>↩</code> symbol indicates where I've wrapped the text. I'll do that again later in this article.) </p> <p> Notice that I'm using <a href="/2020/09/28/ensuresuccessstatuscode-as-an-assertion">EnsureSuccessStatusCode as an assertion</a>. This seems to spoil the behaviour of <code>Assert.Multiple</code>. It only reports the first status code error, but not the second one. </p> <p> I admit that I don't fully understand what's going on here. In fact, I <em>have</em> taken a cursory glance at the relevant NUnit source code without being enlightened. </p> <p> One hypothesis might be that NUnit assertions throw special <code>Exception</code> sub-types that <code>Assert.Multiple</code> catch. In order to test that, I wrote a few more tests in <a href="https://fsharp.org/">F#</a> with <a href="http://www.swensensoftware.com/unquote/">Unquote</a>, assuming that, since Unquote hardly throws NUnit exceptions, the behaviour might be similar to above. </p> <p> <pre>[&lt;Test&gt;] <span style="color:blue;">let</span>&nbsp;Test4&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&nbsp;=&nbsp;1 &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;y&nbsp;=&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;z&nbsp;=&nbsp;3 &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple&nbsp;(<span style="color:blue;">fun</span>&nbsp;()&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=!&nbsp;y &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;y&nbsp;=!&nbsp;z)</pre> </p> <p> The <code>=!</code> operator is an Unquote operator that I usually read as <em>must equal</em>. How does that error message look? </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures or warnings in test: &nbsp;&nbsp;&nbsp;&nbsp;1) &nbsp;&nbsp;1 = 2 &nbsp;&nbsp;false &nbsp;&nbsp;&nbsp;&nbsp;2) &nbsp;&nbsp;2 = 3 &nbsp;&nbsp;false</pre> </p> <p> Somehow, <code>Assert.Multiple</code> understands Unquote error messages, but not <code>HttpRequestException</code>. As I wrote, I don't fully understand why it behaves this way. To a degree, I'm intellectually curious enough that I'd like to know. On the other hand, from a maintainability perspective, as a user of NUnit, I shouldn't have to understand such details. </p> <h3 id="1f28e534b0f94e93bb12cbb951fa663f"> xUnit.net Assert.Multiple <a href="#1f28e534b0f94e93bb12cbb951fa663f" title="permalink">#</a> </h3> <p> How fares the <a href="https://xunit.net/">xUnit.net</a> port of <code>Assert.Multiple</code>? </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Multiple( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;()&nbsp;=&gt;&nbsp;deleteResp.EnsureSuccessStatusCode(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;()&nbsp;=&gt;&nbsp;Assert.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode)); }</pre> </p> <p> The API is, you'll notice, not quite identical. Where the NUnit <code>Assert.Multiple</code> method takes a single delegate as input, the xUnit.net method takes an array of actions. The difference is not only at the level of API; the behaviour is different, too: </p> <p> <pre>Message: &nbsp;&nbsp;Multiple failures were encountered: &nbsp;&nbsp;---- System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request). &nbsp;&nbsp;---- Assert.Equal() Failure &nbsp;&nbsp;Expected: NotFound &nbsp;&nbsp;Actual: OK</pre> </p> <p> This error message reports both problems, as we'd like it to do. </p> <p> I also tried writing equivalent tests in F#, with and without Unquote, and they behave consistently with this result. </p> <p> If I had to use something like <code>Assert.Multiple</code>, I'd trust the xUnit.net variant more than NUnit's implementation. </p> <h3 id="c51fb932618c4464a2e9be05869005d4"> Assertion scopes <a href="#c51fb932618c4464a2e9be05869005d4" title="permalink">#</a> </h3> <p> Apparently, <a href="https://fluentassertions.com/">Fluent Assertions</a> offers yet another alternative. </p> <blockquote> <p> "Hey @ploeh, been reading your applicative assertion series. I recently discovered Assertion Scopes, so I'm wondering what is your take on them since it seems to me they are solving this problem in C# already. <a href="https://fluentassertions.com/introduction#assertion-scopes">https://fluentassertions.com/introduction#assertion-scopes</a></a>" </p> <footer><cite><a href="https://twitter.com/JernejGoricki/status/1597704973839904768">Jernej Gorički</a></cite></footer> </blockquote> <p> The linked documentation contains this example: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;DocExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">new</span>&nbsp;AssertionScope()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.Should().Be(10); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Actual&quot;</span>.Should().Be(<span style="color:#a31515;">&quot;Expected&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> It fails in the expected manner: </p> <p> <pre>Message: &nbsp;&nbsp;Expected value to be 10, but found 5 (difference of -5). &nbsp;&nbsp;Expected string to be "Expected" with a length of 8, but "Actual" has a length of 6,↩ &nbsp;&nbsp;&nbsp;&nbsp;differs near "Act" (index 0).</pre> </p> <p> How does it fare when subjected to the <code>EnsureSuccessStatusCode</code> test? </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;HttpExample() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.BadRequest); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;HttpResponseMessage(HttpStatusCode.OK); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;(<span style="color:blue;">new</span>&nbsp;AssertionScope()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getResp.StatusCode.Should().Be(HttpStatusCode.NotFound); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> That test produces this error output: </p> <p> <pre>Message: &nbsp;&nbsp;System.Net.Http.HttpRequestException :↩ &nbsp;&nbsp;&nbsp;&nbsp;Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> Again, <code>EnsureSuccessStatusCode</code> prevents further assertions from being evaluated. I can't say that I'm that surprised. </p> <h3 id="2f325809027243889d703e927a115efc"> Implicit or explicit <a href="#2f325809027243889d703e927a115efc" title="permalink">#</a> </h3> <p> You might protest that using <code>EnsureSuccessStatusCode</code> and treating the resulting <code>HttpRequestException</code> as an assertion is unfair and unrealistic. Possibly. As usual, such considerations are subject to a multitude of considerations, and there's no one-size-fits-all answer. </p> <p> My intent with this article isn't to attack or belittle the APIs I've examined. Rather, I wanted to explore their boundaries by stress-testing them. That's one way to gain a better understanding. Being aware of an API's limitations and quirks can prevent subtle bugs. </p> <p> Even if you'd <em>never</em> use <code>EnsureSuccessStatusCode</code> as an assertion, perhaps you or a colleague might inadvertently do something to the same effect. </p> <p> I'm not surprised that both NUnit's <code>Assert.Multiple</code> and Fluent Assertions' <code>AssertionScope</code> behaves in a less consistent manner than xUnit.net's <code>Assert.Multiple</code>. The clue is in the API. </p> <p> The xUnit.net API looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Multiple</span>(<span style="color:blue;">params</span>&nbsp;Action[]&nbsp;<span style="color:#1f377f;">checks</span>)</pre> </p> <p> Notice that each assertion is explicitly a separate action. This enables the implementation to isolate it and treat it independently of other actions. </p> <p> Neither the NUnit nor the Fluent Assertions API is that explicit. Instead, you can write arbitrary code inside the 'scope' of multiple assertions. For <code>AssertionScope</code>, the notion of a 'scope' is plain to see. For the NUnit API it's more implicit, but the scope is effectively the extent of the method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Multiple</span>(TestDelegate&nbsp;<span style="color:#1f377f;">testDelegate</span>)</pre> </p> <p> That <code>testDelegate</code> can have as many (nested, even) assertions as you'd like, so the <code>Multiple</code> implementation needs to somehow demarcate when it begins and when it ends. </p> <p> The <code>testDelegate</code> can be implemented in a different file, or even in a different library, and it has no way to communicate or coordinate with its surrounding scope. This reminds me of an Ambient Context, an idiom that <a href="/2019/01/21/some-thoughts-on-anti-patterns">Steven van Deursen convinced me was an anti-pattern</a>. The surrounding context changes the behaviour of the code block it surrounds, and it's quite implicit. </p> <blockquote> <p> Explicit is better than implicit. </p> <footer><cite>Tim Peters, <a href="https://peps.python.org/pep-0020/">The Zen of Python</a></cite></footer> </blockquote> <p> The xUnit.net API, at least, looks a bit saner. Still, this kind of API is quirky enough that it reminds me of <a href="https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule">Greenspun's tenth rule</a>; that these APIs are ad-hoc, informally-specified, bug-ridden, slow implementations of half of <a href="/2018/10/01/applicative-functors">applicative functors</a>. </p> <h3 id="95b723f05a7f4b4c88b5e716f5f82989"> Conclusion <a href="#95b723f05a7f4b4c88b5e716f5f82989" title="permalink">#</a> </h3> <p> Not surprisingly, popular unit-testing and assertion libraries come with facilities to compose assertions. Also, not surprisingly, these APIs are crude and require you to learn their implementation details. </p> <p> Would I use them if I had to? I probably would. As <a href="https://www.infoq.com/presentations/Simple-Made-Easy/">Rich Hickey put it</a>, they're already <em>at hand</em>. That makes them easy, but not necessarily simple. APIs that compel you to learn their internal implementation details aren't simple. </p> <p> <a href="/2017/10/04/from-design-patterns-to-category-theory">Universal abstractions</a>, on the other hand, you only have to learn one time. Once you understand what an applicative functor is, you know what to expect from it, and which capabilities it has. </p> <p> In languages with good support for applicative functors, I would favour an assertion API based on that abstraction, if given a choice. At the moment, though, that's not much of an option. Even <a href="https://hackage.haskell.org/package/HUnit/docs/Test-HUnit-Base.html#t:Assertion">HUnit assertions</a> are based on side effects. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="e3269279066146f985c8405f6d3ad286"> <div class="comment-author">Joker_vD</div> <div class="comment-content"> <p> Just a reminder: in .NET, method's execution <b>cannot</b> be resumed after an exception is thrown, there is just simply no way to do this, at all. Which means that NUnit's Assert.Multiple absolutely cannot work the way you guess it probably does, by running the delegate and resuming its execution after it throws an exception until the delegate returns. </p> <p> How could it work then? Well, considering that documentation to almost every Assert's method has "Returns without throwing an exception when inside a multiple assert block" line in it, I would assume that Assert.Multiple sets a global flag which makes actual assertions to store the failures in some global hidden context instead on throwing them, then runs the delegate and after it finishes or throws, collects and clears all those failures from the context and resets the global flag. </p> <p> Cursory inspection of NUnit's source code supports this idea, except that apparently it's not just a boolean flag but a "depth" counter; and assertions report the failures <a href="https://github.com/nunit/nunit/blob/62059054137de84b711353765d474779db95f731/src/NUnitFramework/framework/Assert.cs#L371">just the way I've speculated</a>. I personally hate such side-channels but you have to admit, they allow for some nifty, seemingly impossible magical tricks (a.k.a. "spooky action at the distance"). </p> <p> Also, why do you assume that Unquote would not throw NUnit's assertions? It literally has "Unquote integrates configuration-free with all exception-based unit testing frameworks including xUnit.net, NUnit, MbUnit, Fuchu, and MSTest" in its README, and indeed, if you look at <a href="https://github.com/SwensenSoftware/unquote/blob/78b071043c42372f3693a07e5562520046873ebc/src/Unquote/Assertions.fs">its source code</a>, you'll see that at runtime it tries to locate any testing framework it's aware of and use its assertions. More funny party tricks, this time with reflection! </p> <p> I understand that after working in more pure/functional programming environments one does start to slowly forget about those terrible things, but: those horrorterrors <i>still</i> exist, and people <i>keep making</i> more of them. Now, if you can, have a good night :) </p> </div> <div class="comment-date">2023-01-31 03:00 UTC</div> </div> <div class="comment" id="e0e7c5b258d54c30b87f157e8746150d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Joker_vD, thank you for explaining those details. I admit that I hadn't thought too deeply about implementation details, for the reasons I briefly mentioned in the post. </p> <blockquote> <p> "I understand that after working in more pure/functional programming environments one does start to slowly forget about those terrible things" </p> </blockquote> <p> Yes, that summarises my current thinking well, I'm afraid. </p> </div> <div class="comment-date">2023-01-30 6:49 UTC</div> </div> <div class="comment" id="821541be129a4ea7976ab33f71d3637a"> <div class="comment-author"><a href="https://github.com/MaxKot">Max Kiselev</a></div> <div class="comment-content"> <p> NUnit has <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/classic-assertions/Assert.DoesNotThrow.html">Assert.DoesNotThrow</a> and Fluent Assertions has <a href="https://fluentassertions.com/exceptions/">.Should().NotThrow()</a>. I did not check Fluent Assertions, but NUnit does gather failures of Assert.DoesNotThrow inside Assert.Multiple into a multi-error report. One might argue that asserting that a delegate should not throw is another application of the "explicit is better than implicit" philosophy. Here's what Fluent Assertions has to say on that matter: </p> <blockquote> <p> "We know that a unit test will fail anyhow if an exception was thrown, but this syntax returns a clearer description of the exception that was thrown and fits better to the AAA syntax." </p> </blockquote> <p> As a side note, you might also want to take a look on NUnits Assert.That syntax. It allows to construct complex conditions tested against a single actual value: </p> <p> <pre style="font-family:Consolas;font-size:13px;color:black;background:white;"><span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;3; Assert.That&nbsp;(actual,&nbsp;Is.GreaterThan&nbsp;(0).And.LessThanOrEqualTo&nbsp;(2).And.Matches&nbsp;(Has.Property&nbsp;(<span style="color:#a31515;">&quot;P&quot;</span>).EqualTo&nbsp;(<span style="color:#a31515;">&quot;a&quot;</span>)));</pre> </p> <p> A failure is then reported like this: </p> <p> <pre>Expected: greater than 0 and less than or equal to 2 and property P equal to "a" But was: 3</pre> </p> </div> <div class="comment-date">2023-01-31 18:35 UTC</div> </div> <div class="comment" id="815b4f0e18284dccb3ce38dbb476eb4d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Max, thank you for writing. I have to admit that I never understood the point of <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/assertion-models/constraint.html">NUnit's constraint model</a>, but your example clearly illustrates how it may be useful. It enables you to compose assertions. </p> <p> It's interesting to try to understand the underlying reason for that. I took a cursory glance at that <code>IResolveConstraint</code> API, and as far as I can tell, it may form a <a href="/2017/10/06/monoids">monoid</a> (I'm not entirely sure about the <code>ConstraintStatus</code> enum, but even so, it may be 'close enough' to be composable). </p> <p> I can see how that may be useful when making assertions against complex objects (i.e. object composed from other objects). </p> <p> In xUnit.net you'd typically address that problem with custom <a href="https://learn.microsoft.com/dotnet/api/system.collections.generic.iequalitycomparer-1">IEqualityComparers</a>. This is more verbose, but also strikes me as more reusable. One disadvantage of that approach, however, is that when tests fail, the assertion message is typically useless. </p> <p> This is the reason I favour Unquote: Instead of inventing a Boolean algebra(?) from scratch, it uses the existing language and still gives you good error messages. Alas, that only works in F#. </p> <p> In general, though, I'm inclined to think that all of these APIs address symptoms rather than solve real problems. Granted, they're useful whenever you need to make assertions against values that you don't control, but for your own APIs, <a href="/2021/05/03/structural-equality-for-better-tests">a simpler solution is to model values as immutable data with structural equality</a>. </p> <p> Another question is whether aiming for clear assertion messages is optimising for the right concern. At least with TDD, <a href="/2022/12/12/when-do-tests-fail">I don't think that it is</a>. </p> </div> <div class="comment-date">2023-02-02 7:53 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2023/01/30/built-in-alternatives-to-applicative-assertions Agilean https://blog.ploeh.dk/2023/01/23/agilean/ Mon, 23 Jan 2023 07:55:00 UTC <div id="post"> <p> <em>There are other agile methodologies than scrum.</em> </p> <p> More than twenty years after <a href="https://agilemanifesto.org/">the Agile Manifesto</a> it looks as though there's only one kind of agile process left: <a href="https://en.wikipedia.org/wiki/Scrum_(software_development)">Scrum</a>. </p> <p> I recently held a workshop and as a side remark I mentioned that I don't consider scrum the best development process. This surprised some attendees, who politely inquired about my reasoning. </p> <h3 id="8837c4f67d694f93ad0b708cc1739705"> My experience with scrum <a href="#8837c4f67d694f93ad0b708cc1739705" title="permalink">#</a> </h3> <p> The first nine years I worked as a professional programmer, the companies I worked in used various <a href="https://en.wikipedia.org/wiki/Waterfall_model">waterfall</a> processes. When I joined the Microsoft Dynamics Mobile team in 2008 they were already using scrum. That was my first exposure to it, and I liked it. Looking back on it today, we weren't particular dogmatic about the process, being more interested in getting things done. </p> <p> One telling fact is that we took turns being Scrum Master. Every sprint we'd rotate that role. </p> <p> We did test-driven development, and had two-week sprints. This being a Microsoft development organisation, we had a dedicated build master, tech writers, specialised testers, and security reviews. </p> <p> I liked it. It's easily one of the most professional software organisations I've worked in. I think it was a good place to work for many reasons. Scrum may have been a contributing factor, but hardly the only reason. </p> <p> I have no issues with scrum as we practised it then. I recall later attending a presentation by <a href="https://en.wikipedia.org/wiki/Mike_Cohn">Mike Cohn</a> where he outlined four quadrants of team maturity. You'd start with scrum, but use retrospectives to evaluate what worked and what didn't. Then you'd adjust. A mature, self-organising team would arrive at its own process, perhaps initiated with scrum, but now having little resemblance with it. </p> <p> I like scrum when viewed like that. When it becomes rigid and empty ceremony, I don't. If all you do is daily stand-ups, sprints, and backlogs, you may be doing scrum, but probably not agile. </p> <h3 id="1bb8e521e78341768ca7f5942f3ace4a"> Continuous deployment <a href="#1bb8e521e78341768ca7f5942f3ace4a" title="permalink">#</a> </h3> <p> After Microsoft I joined a startup so small that formal process was unnecessary. Around that time I also became interested in <a href="https://en.wikipedia.org/wiki/Lean_software_development">lean software development</a>. In the beginning, I learned a lot from <a href="https://www.linkedin.com/in/martin-jul-39a12/">Martin Jul</a> who seemed to use the now-defunct <a href="https://ative.dk/">Ative</a> blog as a public notepad as he was reading works of <a href="https://en.wikipedia.org/wiki/W._Edwards_Deming">Deming</a>. I suppose, if you want a more canonical introduction to the topic, that you might start with one of <a href="http://www.poppendieck.com/">the Poppendiecks'</a> books, but since I've only read <a href="/ref/implementing-lean">Implementing Lean Software Development</a>, that's the only one I can recommend. </p> <p> Around 2014 I returned to a regular customer. The team had, in my absence, been busy implementing <a href="https://en.wikipedia.org/wiki/Continuous_deployment">continuous deployment</a>. Instead of artificial periods like 'sprints' we had a <a href="https://en.wikipedia.org/wiki/Kanban_board">kanban board</a> to keep track of our work. We used a variation of <a href="https://en.wikipedia.org/wiki/Feature_toggle">feature flags</a> and marked features as done when they were complete and in production. </p> <p> Why wait until <em>next</em> Friday if the feature is <em>done, done</em> on a Wednesday? Why wait until the <em>next</em> Monday to identify what to work on next, if you're ready to take on new work on a Thursday? Why not move towards <em>one-piece flow?</em> </p> <p> An effective self-organising team typically already knows what it's doing. Much process is introduced in order to give external stakeholders visibility into what a team is doing. </p> <p> I found, in that organisation, that continuous deployment eliminated most of that need. At one time I asked a stakeholder what he thought of the feature I'd deployed a week before - a feature that <em>he had requested</em>. He replied that he hadn't had time to look at it yet. </p> <p> The usual inquires about status (<em>Is it done yet? When is it done?</em>) were gone. The team moved faster than the stakeholders could keep up. That also gave us <a href="/2022/09/19/when-to-refactor">enough slack to keep the code base in good order</a>. We also used test-driven development throughout (TDD). </p> <p> TDD with continuous deployment and a kanban board strikes me as congenial with the ideas of lean software development, but that's not all. </p> <h3 id="9f5e725164f54d32b5d56a28d3b1619e"> Stop-the-line issues <a href="#9f5e725164f54d32b5d56a28d3b1619e" title="permalink">#</a> </h3> <p> An <a href="https://en.wikipedia.org/wiki/Andon_(manufacturing)">andon cord</a> is a central concept in <a href="https://en.wikipedia.org/wiki/Lean_manufacturing">lean manufactoring</a>. If a worker (or anyone, really) discovers a problem during production, he or she pulls the andon cord and <em>stops the production line</em>. Then everyone investigates and determines what to do about the problem. Errors are not allowed to accumulate. </p> <p> I think that I've internalised this notion to such a degree that I only recently connected it to lean software development. </p> <p> In <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>, I recommend turning compiler warnings into errors at the beginning of a code base. Don't allow warnings to pile up. Do the same with static code analysis and linters. </p> <p> When discussing software engineering with developers, I'm beginning to realise that this runs even deeper. </p> <ul> <li>Turn warnings into errors. Don't allow warnings to accumulate.</li> <li>The correct number of unhandled exceptions in production is zero. If you observe an unhandled exception in your production logs, fix it. Don't let them accumulate.</li> <li>The correct number of known bugs is zero. Don't let bugs accumulate.</li> </ul> <p> If you're used to working on a code base with hundreds of known bugs, and frequent exceptions in production, this may sound unrealistic. If you deal with issues as soon as they arise, however, this is not only possible - it's faster. </p> <p> In lean software development, bugs are stop-the-line issues. When something unexpected happens, you stop what you're doing and make fixing the problem the top priority. You build quality in. </p> <p> This has been my modus operandi for years, but I only recently connected the dots to realise that this is a typical lean practice. I may have picked it up from there. Or perhaps it's just common sense. </p> <h3 id="e2513b8e32b148cbab8dbd6b75c67348"> Conclusion <a href="#e2513b8e32b148cbab8dbd6b75c67348" title="permalink">#</a> </h3> <p> When Agile was new and exciting, there were <a href="https://en.wikipedia.org/wiki/Extreme_programming">extreme programming</a> and scrum, and possibly some lesser known techniques. Lean was around the corner, but didn't come to my attention, at least, until around 2010. Then it seems to have faded away again. </p> <p> Today, agile looks synonymous with scrum, but I find lean software development more efficient. Why divide work into artificial time periods when you can release continuously? Why <em>plan</em> bug fixing when it's more efficient to stop the line and deal with the problem as it arises? </p> <p> That may sound counter-intuitive, but it works because it prevents technical debt from accumulating. </p> <p> Lean software development is, in my experience, a better agile methodology than scrum. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2023/01/23/agilean In the long run https://blog.ploeh.dk/2023/01/16/in-the-long-run/ Mon, 16 Jan 2023 08:28:00 UTC <div id="post"> <p> <em>Software design decisions should be time-aware.</em> </p> <p> A common criticism of modern capitalism is that maximising <a href="https://en.wikipedia.org/wiki/Shareholder_value">shareholder value</a> leads to various detrimental outcomes, both societal, but possibly also for the maximising organisation itself. One major problem is when company leadership is incentivised to optimise stock market price for the next quarter, or other short terms. When considering only the short term, decision makers may (rationally) decide to sacrifice long-term benefits for short-term gains. </p> <p> We often see similar behaviour in democracies. Politicians tend to optimise within a time frame that coincides with the election period. Getting re-elected is more important than good policy in the next period. </p> <p> These observations are crude generalisations. Some democratic politicians and CEOs take longer views. Inherent in the context, however, is an incentive to short-term thinking. </p> <p> This, it strikes me, is frequently the case in software development. </p> <p> Particularly in the context of <a href="https://en.wikipedia.org/wiki/Scrum_(software_development)">scrum</a> there's a focus on delivering at the end of every sprint. I've observed developers and other stakeholders together engage in short-term thinking in order to meet those arbitrary and fictitious deadlines. </p> <p> Even when deadlines are more remote than two weeks, project members rarely think beyond some perceived end date. As I describe in <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>, a <em>project</em> is rarely is good way to organise software development work. Projects end. Successful software doesn't. </p> <p> Regardless of the specific circumstances, a too myopic focus on near-term goals gives you an incentive to cut corners. To not care about code quality. </p> <h3 id="5cd528d178504f2ba849019e5d6d425e"> ...we're all dead <a href="#5cd528d178504f2ba849019e5d6d425e" title="permalink">#</a> </h3> <p> As <a href="https://en.wikipedia.org/wiki/John_Maynard_Keynes">Keynes</a> once quipped: </p> <blockquote> <p> "In the long run we are all dead." </p> <footer><cite>John Maynard Keynes</cite></footer> </blockquote> <p> Clearly, while you can be too short-sighted, you can also take too long a view. Sometimes deadlines matter, and software not used makes no-one happy. </p> <p> Working software remains the ultimate test of value, but as I've tried to express many times before, this does <em>not</em> imply that anything else is worthless. </p> <p> You can't measure code quality. <a href="/2019/03/04/code-quality-is-not-software-quality">Code quality isn't software quality</a>. Low code quality <a href="https://martinfowler.com/articles/is-quality-worth-cost.html">slows you down</a>, and that, eventually, costs you money, blood, sweat, and tears. </p> <p> This is, however, not difficult to predict. All it takes is a slightly wider time horizon. Consider the impact of your decisions past the next deadline. </p> <h3 id="b0f0f05bd00142379c2d713adde6eb77"> Conclusion <a href="#b0f0f05bd00142379c2d713adde6eb77" title="permalink">#</a> </h3> <p> Don't be too short-sighted, but don't forget the immediate value of what you do. Your <a href="/2019/03/18/the-programmer-as-decision-maker">decisions</a> matter. The impact is not always immediate. Consider what consequences short-term optimisations may have in a longer perspective. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2023/01/16/in-the-long-run The IO monad https://blog.ploeh.dk/2023/01/09/the-io-monad/ Mon, 09 Jan 2023 07:39:00 UTC <div id="post"> <p> <em>The IO container forms a monad. An article for object-oriented programmers.</em> </p> <p> This article is an instalment in <a href="/2022/03/28/monads">an article series about monads</a>. A previous article described <a href="/2020/06/22/the-io-functor">the IO functor</a>. As is the case with many (but not all) <a href="/2018/03/22/functors">functors</a>, this one also forms a monad. </p> <h3 id="bee075f5bf474b099146cbc7c96f4888"> SelectMany <a href="#bee075f5bf474b099146cbc7c96f4888" title="permalink">#</a> </h3> <p> A monad must define either a <em>bind</em> or <em>join</em> function. In C#, monadic bind is called <code>SelectMany</code>. In a recent article, I gave an example of <a href="/2020/06/15/io-container-in-a-parallel-c-universe">what <em>IO</em> might look like in C#</a>. Notice that it already comes with a <code>SelectMany</code> function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;IO&lt;TResult&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(Func&lt;T,&nbsp;IO&lt;TResult&gt;&gt;&nbsp;<span style="color:#1f377f;">selector</span>)</pre> </p> <p> Unlike other monads, the IO implementation is considered a black box, but if you're interested in a prototypical implementation, I already posted <a href="/2020/07/13/implementation-of-the-c-io-container">a sketch</a> in 2020. </p> <h3 id="1ba6a4e6da014fc3a511e71bdba4f0a9"> Query syntax <a href="#1ba6a4e6da014fc3a511e71bdba4f0a9" title="permalink">#</a> </h3> <p> I have also, already, demonstrated <a href="/2020/06/29/syntactic-sugar-for-io">syntactic sugar for IO</a>. In that article, however, I used an implementation of the required <code>SelectMany</code> overload that is more explicit than it has to be. The <a href="/2022/03/28/monads">monad introduction</a> makes the prediction that you can always implement that overload in the same way, and yet here I didn't. </p> <p> That's an oversight on my part. You can implement it like this instead: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;TResult&gt;&nbsp;<span style="color:#74531f;">SelectMany</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">U</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#1f377f;">source</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;IO&lt;U&gt;&gt;&nbsp;<span style="color:#1f377f;">k</span>, &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;T,&nbsp;U,&nbsp;TResult&gt;&nbsp;<span style="color:#1f377f;">s</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;k(x).Select(<span style="color:#1f377f;">y</span>&nbsp;=&gt;&nbsp;s(x,&nbsp;y))); }</pre> </p> <p> Indeed, the conjecture from the introduction still holds. </p> <h3 id="f234f9e9d3724a30b8465d176a0d98a6"> Join <a href="#f234f9e9d3724a30b8465d176a0d98a6" title="permalink">#</a> </h3> <p> In <a href="/2022/03/28/monads">the introduction</a> you learned that if you have a <code>Flatten</code> or <code>Join</code> function, you can implement <code>SelectMany</code>, and the other way around. Since we've already defined <code>SelectMany</code> for <code>IO&lt;T&gt;</code>, we can use that to implement <code>Join</code>. In this article I use the name <code>Join</code> rather than <code>Flatten</code>. This is an arbitrary choice that doesn't impact behaviour. Perhaps you find it confusing that I'm inconsistent, but I do it in order to demonstrate that the behaviour is the same even if the name is different. </p> <p> The concept of a monad is universal, but the names used to describe its components differ from language to language. What C# calls <code>SelectMany</code>, Scala calls <code>flatMap</code>, and what <a href="https://www.haskell.org/">Haskell</a> calls <code>join</code>, other languages may call <code>Flatten</code>. </p> <p> You can always implement <code>Join</code> by using <code>SelectMany</code> with the identity function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#74531f;">Join</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;IO&lt;IO&lt;T&gt;&gt;&nbsp;<span style="color:#1f377f;">source</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;source.SelectMany(<span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x); }</pre> </p> <p> In C# the identity function is <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatically</a> given as the lambda expression <code><span style="color:#1f377f;">x</span>&nbsp;=&gt;&nbsp;x</code> since C# doesn't come with a built-in identity function. </p> <h3 id="c57bcf1e74144046bc7191280d78519b"> Return <a href="#c57bcf1e74144046bc7191280d78519b" title="permalink">#</a> </h3> <p> Apart from monadic bind, a monad must also define a way to put a normal value into the monad. Conceptually, I call this function <em>return</em> (because that's the name that Haskell uses). In <a href="/2020/06/22/the-io-functor">the IO functor</a> article, I wrote that the <code>IO&lt;T&gt;</code> constructor corresponds to <em>return</em>. That's not strictly true, though, since the constructor takes a <code>Func&lt;T&gt;</code> and not a <code>T</code>. </p> <p> This issue is, however, trivially addressed: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;IO&lt;T&gt;&nbsp;<span style="color:#74531f;">Return</span>&lt;<span style="color:#2b91af;">T</span>&gt;(T&nbsp;<span style="color:#1f377f;">x</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;IO&lt;T&gt;(()&nbsp;=&gt;&nbsp;x); }</pre> </p> <p> Take the value <code>x</code> and wrap it in a lazily-evaluated function. </p> <h3 id="e0548aa855224081a5c1c3ec80f23951"> Laws <a href="#e0548aa855224081a5c1c3ec80f23951" title="permalink">#</a> </h3> <p> While <a href="/2020/07/06/referential-transparency-of-io">IO values are referentially transparent</a> you can't compare them. You also can't 'run' them by other means than running a program. This makes it hard to talk meaningfully about the <a href="/2022/04/11/monad-laws">monad laws</a>. </p> <p> For example, the left identity law is: </p> <p> <pre>return &gt;=&gt; h ≡ h</pre> </p> <p> Note the implied equality. The composition of <code>return</code> and <code>h</code> should be equal to <code>h</code>, for some reasonable definition of equality. How do we define that? </p> <p> Somehow we must imagine that two alternative compositions would produce the same observable effects <a href="https://en.wikipedia.org/wiki/Ceteris_paribus">ceteris paribus</a>. If you somehow imagine that you have two parallel universes, one with one composition (say <code>return &gt;=&gt; h</code>) and one with another (<code>h</code>), if all else in those two universes were equal, then you would observe no difference in behaviour. </p> <p> That may be useful as a thought experiment, but isn't particularly practical. Unfortunately, due to side effects, things <em>do</em> change when non-deterministic behaviour and side effects are involved. As a simple example, consider an IO action that gets the current time and prints it to the console. That involves both non-determinism and a side effect. </p> <p> In Haskell, that's a straightforward composition of two <code>IO</code> actions: </p> <p> <pre>&gt; h () = getCurrentTime &gt;>= print</pre> </p> <p> How do we compare two compositions? By running them? </p> <p> <pre>&gt; return () &gt;&gt;= h 2022-06-25 16:47:30.6540847 UTC &gt; h () 2022-06-25 16:47:37.5281265 UTC</pre> </p> <p> The outputs are not the same, because time goes by. Can we thereby conclude that the monad laws don't hold for IO? Not quite. </p> <p> The IO Container is referentially transparent, but evaluation isn't. Thus, we have to pretend that two alternatives will lead to the same evaluation behaviour, all things being equal. </p> <p> This property seems to hold for both the identity and associativity laws. Whether or not you compose with <em>return</em>, or in which evaluation order you compose actions, it doesn't affect the outcome. </p> <p> For completeness sake, the <a href="/2020/07/13/implementation-of-the-c-io-container">C# implementation sketch</a> is just a wrapper over a <code>Func&lt;T&gt;</code>. We can also think of such a function as a function from <a href="/2018/01/15/unit-isomorphisms">unit</a> to <code>T</code> - in pseudo-C# <code>() =&gt; T</code>. That's a function; in other words: <a href="/2022/11/14/the-reader-monad">The Reader monad</a>. We already know that the Reader monad obeys the monad laws, so the C# implementation, at least, should be okay. </p> <h3 id="74154d1a95a44a91a8d534fbfda59d8d"> Conclusion <a href="#74154d1a95a44a91a8d534fbfda59d8d" title="permalink">#</a> </h3> <p> IO forms a monad, among other abstractions. This is what enables Haskell programmers to compose an arbitrary number of impure actions with monadic bind without ever having to force evaluation. In C# <a href="/2020/06/15/io-container-in-a-parallel-c-universe">it might have looked the same</a>, except that it doesn't. </p> <p> <strong>Next:</strong> <a href="/2018/01/08/software-design-isomorphisms">Software design isomorphisms</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2023/01/09/the-io-monad Adding NuGet packages when offline https://blog.ploeh.dk/2023/01/02/adding-nuget-packages-when-offline/ Mon, 02 Jan 2023 05:41:00 UTC <div id="post"> <p> <em>A fairly trivial technical detective story.</em> </p> <p> I was recently in an air plane, writing code, when I realised that I needed to add a couple of NuGet packages to my code base. I was on one of those less-travelled flights in Europe, on board an <a href="https://en.wikipedia.org/wiki/Embraer_E-Jet_family">Embraer E190</a>, and as is usually the case on those 1½-hour flights, there was no WiFi. </p> <p> Adding a NuGet package typically requires that you're online so that the tools can query the relevant NuGet repository. You'll need to download the package, so if you're offline, you're just out of luck, right? </p> <p> Fortunately, I'd previously used the packages I needed in other projects, on the same laptop. While <a href="/2014/01/29/nuget-package-restore-considered-harmful">I'm no fan of package restore</a>, I know that the local NuGet tools cache packages somewhere on the local machine. </p> <p> So, perhaps I could entice the tools to reuse a cached package... </p> <p> First, I simply tried adding a package that I needed: </p> <p> <pre>$ dotnet add package unquote Determining projects to restore... Writing C:\Users\mark\AppData\Local\Temp\tmpF3C.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET. info : Adding PackageReference for package 'unquote' into project '[redacted]'. error: Unable to load the service index for source https://api.nuget.org/v3/index.json. error: No such host is known. (api.nuget.org:443) error: No such host is known.</pre> </p> <p> Fine plan, but no success. </p> <p> Clearly the <code>dotnet</code> tool was trying to access <code>api.nuget.org</code>, which, obviously, couldn't be reached because my laptop was in flight mode. It occurred to me, though, that the reason that the tool was querying <code>api.nuget.org</code> was that it wanted to see which version of the package was the most recent. After all, I hadn't specified a version. </p> <p> What if I were to specify a version? Would the tool use the cached version of the package? </p> <p> That seemed worth a try, but which versions did I already have on my laptop? </p> <p> I don't go around remembering which version numbers I've used of various NuGet packages, but I expected the NuGet tooling to have that information available, somewhere. </p> <p> But where? Keep in mind that I was offline, so couldn't easily look this up. </p> <p> On the other hand, I knew that these days, most Windows applications keep data of that kind somewhere in <code>AppData</code>, so I started spelunking around there, looking for something that might be promising. </p> <p> After looking around a bit, I found a subdirectory named <code>AppData\Local\NuGet\v3-cache</code>. This directory contained a handful of subdirectories obviously named with GUIDs. Each of these contained a multitude of <code>.dat</code> files. The names of those files, however, looked promising: </p> <p> <pre>list_antlr_index.dat list_autofac.dat list_autofac.extensions.dependencyinjection.dat list_autofixture.automoq.dat list_autofixture.automoq_index.dat list_autofixture.automoq_range_2.0.0-3.6.7.dat list_autofixture.automoq_range_3.30.3-3.50.5.dat list_autofixture.automoq_range_3.50.6-4.17.0.dat list_autofixture.automoq_range_3.6.8-3.30.2.dat list_autofixture.dat ...</pre> </p> <p> and so on. </p> <p> These names were clearly(?) named <code>list_[package-name].dat</code> or <code>list_[package-name]_index.dat</code>, so I started looking around for one named after the package I was looking for (<a href="https://www.nuget.org/packages/Unquote/">Unquote</a>). </p> <p> Often, both files are present, which was also the case for Unquote. </p> <p> <pre>$ ls list_unquote* -l -rw-r--r-- 1 mark 197609 348 Oct 1 18:38 list_unquote.dat -rw-r--r-- 1 mark 197609 42167 Sep 23 21:29 list_unquote_index.dat</pre> </p> <p> As you can tell, <code>list_unquote_index.dat</code> is much larger than <code>list_unquote.dat</code>. Since I didn't know what the format of these files were, I decided to look at the smallest one first. It had this content: </p> <p> <pre>{ "versions": [ "1.3.0", "2.0.0", "2.0.1", "2.0.2", "2.0.3", "2.1.0", "2.1.1", "2.2.0", "2.2.1", "2.2.2", "3.0.0", "3.1.0", "3.1.1", "3.1.2", "3.2.0", "4.0.0", "5.0.0", "6.0.0-rc.1", "6.0.0-rc.2", "6.0.0-rc.3", "6.0.0", "6.1.0" ] }</pre> </p> <p> A list of versions. Sterling. It looked as though version 6.1.0 was the most recent one on my machine, so I tried to add that one to my code base: </p> <p> <pre>$ dotnet add package unquote --version 6.1.0 Determining projects to restore... Writing C:\Users\mark\AppData\Local\Temp\tmp815D.tmp info : X.509 certificate chain validation will use the default trust store selected by .NET. info : Adding PackageReference for package 'unquote' into project '[redacted]'. info : Restoring packages for [redacted]... info : Package 'unquote' is compatible with all the specified frameworks in project '[redacted]'. info : PackageReference for package 'unquote' version '6.1.0' added to file '[redacted]'. info : Generating MSBuild file [redacted]. info : Writing assets file to disk. Path: [redacted] log : Restored [redacted] (in 397 ms).</pre> </p> <p> Jolly good! That worked. </p> <p> This way I managed to install all the NuGet packages I needed. This was fortunate, because I had so little time to transfer to my connecting flight that I never got to open the laptop before I was airborne again - in another E190 without WiFi, and another session of offline programming. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="7387859cda784fdf8612ddb34666d49b"> <div class="comment-author"><a href="https://github.com/asherber">Aaron Sherber</a></div> <div class="comment-content"> <p> A postscript to your detective story might note that the primary NuGet cache lives at <code>%userprofile%\.nuget\packages</code> on Windows and <code>~/.nuget/packages</code> on Mac and Linux. The folder names there are much easier to decipher than the folders and files in the http cache. </p> </div> <div class="comment-date">2023-01-02 13:46 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2023/01/02/adding-nuget-packages-when-offline Functors as invariant functors https://blog.ploeh.dk/2022/12/26/functors-as-invariant-functors/ Mon, 26 Dec 2022 13:05:00 UTC <div id="post"> <p> <em>A most likely useless set of invariant functors that nonetheless exist.</em> </p> <p> This article is part of <a href="/2022/08/01/invariant-functors">a series of articles about invariant functors</a>. An invariant functor is a <a href="/2018/03/22/functors">functor</a> that is neither covariant nor contravariant. See the series introduction for more details. </p> <p> It turns out that all <a href="/2018/03/22/functors">functors</a> are also invariant functors. </p> <p> Is this useful? Let me be honest and say that if it is, I'm not aware of it. Thus, if you're interested in practical applications, you can stop reading here. This article contains nothing of practical use - as far as I can tell. </p> <h3 id="d748d07afe5342e9847e858849053911"> Because it's there <a href="#d748d07afe5342e9847e858849053911" title="permalink">#</a> </h3> <p> Why describe something of no practical use? </p> <p> Why do some people climb <a href="https://en.wikipedia.org/wiki/Mount_Everest">Mount Everest</a>? <em>Because it's there</em>, or for other irrational reasons. Which is fine. I've no personal goals that involve climbing mountains, but <a href="/2020/10/12/subjectivity">I happily engage in other irrational and subjective activities</a>. </p> <p> One of them, apparently, is to write articles of software constructs of no practical use, <em>because it's there</em>. </p> <p> All functors are also invariant functors, even if that's of no practical use. That's just the way it is. This article explains how, and shows a few (useless) examples. </p> <p> I'll start with a few <a href="https://www.haskell.org/">Haskell</a> examples and then move on to showing the equivalent examples in C#. If you're unfamiliar with Haskell, you can skip that section. </p> <h3 id="268965f94deb48d79335afea6e9b85e4"> Haskell package <a href="#268965f94deb48d79335afea6e9b85e4" title="permalink">#</a> </h3> <p> For Haskell you can find an existing definition and implementations in the <a href="https://hackage.haskell.org/package/invariant">invariant</a> package. It already makes most common functors <code>Invariant</code> instances, including <code>[]</code> (list), <code>Maybe</code>, and <code>Either</code>. Here's an example of using <code>invmap</code> with a small list: </p> <p> <pre>ghci&gt; invmap secondsToNominalDiffTime nominalDiffTimeToSeconds [0.1, 60] [0.1s,60s]</pre> </p> <p> Here I'm using the <a href="https://hackage.haskell.org/package/time">time</a> package to convert fixed-point decimals into <code>NominalDiffTime</code> values. </p> <p> How is this different from normal functor mapping with <code>fmap</code>? In observable behaviour, it's not: </p> <p> <pre>ghci&gt; fmap secondsToNominalDiffTime [0.1, 60] [0.1s,60s]</pre> </p> <p> When invariantly mapping a functor, only the covariant mapping function <code>a -&gt; b</code> is used. Here, that's <code>secondsToNominalDiffTime</code>. The contravariant mapping function <code>b -&gt; a</code> (<code>nominalDiffTimeToSeconds</code>) is simply ignored. </p> <p> While the <em>invariant</em> package already defines certain common functors as <code>Invariant</code> instances, every <code>Functor</code> instance can be converted to an <code>Invariant</code> instance. There are two ways to do that: <code>invmapFunctor</code> and <code>WrappedFunctor</code>. </p> <p> In order to demonstrate, we need a custom <code>Functor</code> instance. This one should do: </p> <p> <pre>data Pair a = Pair (a, a) deriving (Eq, Show, Functor)</pre> </p> <p> If you just want to perform an ad-hoc invariant mapping, you can use <code>invmapFunctor</code>: </p> <p> <pre>ghci&gt; invmapFunctor secondsToNominalDiffTime nominalDiffTimeToSeconds $ Pair (0.1, 60) Pair (0.1s,60s)</pre> </p> <p> I can't think of any reason to do this, but it's possible. </p> <p> <code>WrappedFunctor</code> is perhaps marginally more relevant. If you run into a function that takes an <code>Invariant</code> argument, you can convert any <code>Functor</code> to an <code>Invariant</code> instance by wrapping it in <code>WrappedFunctor</code>: </p> <p> <pre>ghci&gt; invmap secondsToNominalDiffTime nominalDiffTimeToSeconds $ WrapFunctor $ Pair (0.1, 60) WrapFunctor {unwrapFunctor = Pair (0.1s,60s)}</pre> </p> <p> A realistic, useful example still escapes me, but there it is. </p> <h3 id="2578032e937f4a68997f10b0e2412ffb"> Pair as an invariant functor in C# <a href="#2578032e937f4a68997f10b0e2412ffb" title="permalink">#</a> </h3> <p> What would the above Haskell example look like in C#? First, we're going to need a <code>Pair</code> data structure: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Pair</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Pair</span>(T&nbsp;x,&nbsp;T&nbsp;y) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;X&nbsp;=&nbsp;x; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Y&nbsp;=&nbsp;y; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;X&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;T&nbsp;Y&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;follow...</span></pre> </p> <p> Making <code>Pair&lt;T&gt;</code> a functor is so easy that Haskell can do it automatically with the <code>DeriveFunctor</code> extension. In C# you must explicitly write the function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Pair&lt;T1&gt;&nbsp;Select&lt;<span style="color:#2b91af;">T1</span>&gt;(Func&lt;T,&nbsp;T1&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;T1&gt;(selector(X),&nbsp;selector(Y)); }</pre> </p> <p> An example equivalent to the above <code>fmap</code> example might be this, here expressed as a unit test: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;FunctorExample() { &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;/&nbsp;10, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;*&nbsp;60); &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;TimeSpan&gt;&nbsp;actual&nbsp;=&nbsp;sut.Select(ticks&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(ticks)); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;TimeSpan&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(.1), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(60)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;actual); }</pre> </p> <p> You can trivially make <code>Pair&lt;T&gt;</code> an invariant functor by giving it a function equivalent to <code>invmap</code>. As I outlined in <a href="/2022/08/01/invariant-functors">the introduction</a> it's possible to add an <code>InvMap</code> method to the class, but it might be more <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> to instead add a <code>Select</code> overload: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;Pair&lt;T1&gt;&nbsp;Select&lt;<span style="color:#2b91af;">T1</span>&gt;(Func&lt;T,&nbsp;T1&gt;&nbsp;tToT1,&nbsp;Func&lt;T1,&nbsp;T&gt;&nbsp;t1ToT) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Select(tToT1); }</pre> </p> <p> Notice that this overload simply ignores the <code>t1ToT</code> argument and delegates to the normal <code>Select</code> overload. That's consistent with the Haskell package. This unit test shows an examples: </p> <p> <pre>[Fact] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;InvariantFunctorExample() { &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;<span style="color:blue;">long</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;/&nbsp;10, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.TicksPerSecond&nbsp;*&nbsp;60); &nbsp;&nbsp;&nbsp;&nbsp;Pair&lt;TimeSpan&gt;&nbsp;actual&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sut.Select(ticks&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;TimeSpan(ticks),&nbsp;ts&nbsp;=&gt;&nbsp;ts.Ticks); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;Pair&lt;TimeSpan&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(.1), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromSeconds(60)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;actual); }</pre> </p> <p> I can't think of a reason to do this in C#. In Haskell, at least, you have enough power of abstraction to describe something as simply an <code>Invariant</code> functor, and then let client code decide whether to use <code>Maybe</code>, <code>[]</code>, <code>Endo</code>, or a custom type like <code>Pair</code>. You can't do that in C#, so the abstraction is even less useful here. </p> <h3 id="8dbd7bd0f9c64fa5a0ac7c397617521b"> Conclusion <a href="#8dbd7bd0f9c64fa5a0ac7c397617521b" title="permalink">#</a> </h3> <p> All functors are invariant functors. You simply use the normal functor mapping function (<code>fmap</code> in Haskell, <code>map</code> in many other languages, <code>Select</code> in C#). This enables you to add an invariant mapping (<code>invmap</code>) that only uses the covariant argument (<code>a -&gt; b</code>) and ignores the contravariant argument (<code>b -&gt; a</code>). </p> <p> Invariant functors are, however, not particularly useful, so neither is this result. Still, it's there, so deserves a mention. The situation is similar for the next article. </p> <p> <strong>Next:</strong> Contravariant functors as invariant functors. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2022/12/26/functors-as-invariant-functors Error-accumulating composable assertions in C# https://blog.ploeh.dk/2022/12/19/error-accumulating-composable-assertions-in-c/ Mon, 19 Dec 2022 08:39:00 UTC <div id="post"> <p> <em>Perhaps the list monoid is all you need for non-short-circuiting assertions.</em> </p> <p> This article is the second instalment in a small articles series about <a href="/2022/11/07/applicative-assertions">applicative assertions</a>. It explores a way to compose assertions in such a way that failure messages accumulate rather than short-circuit. It assumes that you've read the <a href="/2022/11/07/applicative-assertions">article series introduction</a> and the <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previous article</a>. </p> <p> Unsurprisingly, the previous article showed that you can use an <a href="/2018/10/01/applicative-functors">applicative functor</a> to create composable assertions that don't short-circuit. It also concluded that, in C# at least, the API is awkward. </p> <p> This article explores a simpler API. </p> <h3 id="dc146cd5b792473db7f7e64cac3c4607"> A clue left by the proof of concept <a href="#dc146cd5b792473db7f7e64cac3c4607" title="permalink">#</a> </h3> <p> The previous article's proof of concept left a clue suggesting a simpler API. Consider, again, how the rather horrible <code>RunAssertions</code> method decides whether or not to throw an exception: </p> <p> <pre><span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty); <span style="color:blue;">if</span>&nbsp;(!<span style="color:blue;">string</span>.IsNullOrEmpty(errors)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(errors);</pre> </p> <p> Even though <code><span style="color:#2b91af;">Validated</span>&lt;<span style="color:#2b91af;">F</span>,&nbsp;<span style="color:#2b91af;">S</span>&gt;</code> is a <a href="https://en.wikipedia.org/wiki/Tagged_union">sum type</a>, the <code>RunAssertions</code> method declines to take advantage of that. Instead, it reduces <code>composition</code> to a simple type: A <code>string</code>. It then decides to throw an exception if the <code>errors</code> value is not null or empty. </p> <p> This suggests that using a sum type may not be necessary to distinguish between the success and the failure case. Rather, an empty error string is all it takes to indicate success. </p> <h3 id="801897c3ffc0455e8e7a7ac7531f2be3"> Non-empty errors <a href="#801897c3ffc0455e8e7a7ac7531f2be3" title="permalink">#</a> </h3> <p> The proof-of-concept assertion type is currently defined as <code>Validated</code> with a particular combination of type arguments: <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. Consider, again, this <code>Match</code> expression: </p> <p> <pre><span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty);</pre> </p> <p> Does an empty string unambiguously indicate success? Or is it possible to arrive at an empty string even if <code>composition</code> actually represents a failure case? </p> <p> You can arrive at an empty string from a failure case if the collection of error messages is empty. Consider the type argument that takes the place of the <code>F</code> generic type: <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. A collection of this type can be empty, which would also cause the above <code>Match</code> to produce an empty string. </p> <p> Even so, the proof-of-concept works in practice. The reason it works is that failure cases will never have empty assertion messages. We know this because (in the proof-of-concept code) only two functions produce assertions, and they each populate the error message collection with a string. You may want to revisit the <code>AssertTrue</code> and <code>AssertEqual</code> functions in the <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">previous article</a> to convince yourself that this is true. </p> <p> This is a good example of knowledge that 'we' as developers know, but the code currently doesn't capture. Having to deal with such knowledge taxes your working memory, so why not <a href="/encapsulation-and-solid">encapsulate</a> such information in the type itself? </p> <p> How do you encapsulate the knowledge that a collection is never empty? Introduce a <code>NotEmptyCollection</code> collection. I'll reuse the class from the article <a href="/2017/12/11/semigroups-accumulate">Semigroups accumulate</a> and add a <code>Concat</code> instance method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;NotEmptyCollection&lt;T&gt;&nbsp;Concat(NotEmptyCollection&lt;T&gt;&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;T&gt;(Head,&nbsp;Tail.Concat(other).ToArray()); }</pre> </p> <p> Since the two assertion-producing functions both supply an error message in the failure case, it's trivial to change them to return <code>Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code> - just change the types used: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;condition, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;(message)); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertEqual&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;expected, &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Equals(expected,&nbsp;actual) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>)); }</pre> </p> <p> This change guarantees that the <code>RunAssertions</code> method only produces an empty <code>errors</code> string in success cases. </p> <h3 id="e5097dc5d94f4a1480eeaf483ac08336"> Error collection isomorphism <a href="#e5097dc5d94f4a1480eeaf483ac08336" title="permalink">#</a> </h3> <p> Assertions are still defined by the <code>Validated</code> sum type, but the <em>success</em> case carries no information: <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code>, and the <em>failure</em> case is always guaranteed to contain at least one error message. </p> <p> This suggests that a simpler representation is possible: One that uses a normal collection of errors, and where an empty collection indicates an absence of errors: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>()&nbsp;:&nbsp;<span style="color:blue;">this</span>(Array.Empty&lt;T&gt;()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>(T&nbsp;error)&nbsp;:&nbsp;<span style="color:blue;">this</span>(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;error&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Asserted</span>(IReadOnlyCollection&lt;T&gt;&nbsp;errors) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Errors&nbsp;=&nbsp;errors; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;Asserted&lt;T&gt;&nbsp;And(Asserted&lt;T&gt;&nbsp;other) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentNullException(nameof(other)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;(Errors.Concat(other.Errors).ToList()); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;IReadOnlyCollection&lt;T&gt;&nbsp;Errors&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} }</pre> </p> <p> The <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> class is scarcely more than a glorified wrapper around a normal collection, but it's isomorphic to <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code>, which the following two functions prove: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;T&gt;&nbsp;FromValidated&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;&nbsp;v) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;v.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;failures&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;(failures), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;T&gt;()); } <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;&nbsp;ToValidated&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;Asserted&lt;T&gt;&nbsp;a) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(a.Errors.Any()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;errors&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;NotEmptyCollection&lt;T&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;a.Errors.First(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;a.Errors.Skip(1).ToArray()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Validated.Fail&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;(errors); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Validated.Succeed&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;(Unit.Value); }</pre> </p> <p> You can translate back and forth between <code>Validated&lt;NotEmptyCollection&lt;T&gt;,&nbsp;Unit&gt;</code> and <code>Asserted&lt;T&gt;</code> without loss of information. </p> <p> A collection, however, gives rise to a <a href="/2017/10/06/monoids">monoid</a>, which suggests a much simpler way to compose assertions than using an applicative functor. </p> <h3 id="0ccfc89ad23f4b95932637723ff3c29a"> Asserted truth <a href="#0ccfc89ad23f4b95932637723ff3c29a" title="permalink">#</a> </h3> <p> You can now rewrite the assertion-producing functions to return <code>Asserted&lt;<span style="color:blue;">string</span>&gt;</code> instead of <code>Validated&lt;NotEmptyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;True(<span style="color:blue;">bool</span>&nbsp;condition,&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition&nbsp;?&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;()&nbsp;:&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(message); }</pre> </p> <p> This <code>Asserted.True</code> function returns no error messages when <code>condition</code> is <code>true</code>, but a collection with the single element <code>message</code> when it's <code>false</code>. </p> <p> You can use it in a unit test like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Asserted.True( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> You'll see how <code>assertResponse</code> composes with another assertion later in this article. The example continues from <a href="/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c">the previous article</a>. It's the same test from the same code base. </p> <h3 id="519d7e0d757f4dceb6a2833c09d019d8"> Asserted equality <a href="#519d7e0d757f4dceb6a2833c09d019d8" title="permalink">#</a> </h3> <p> You can also rewrite the other assertion-producing function in the same way: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;Equal(<span style="color:blue;">object</span>&nbsp;expected,&nbsp;<span style="color:blue;">object</span>&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(Equals(expected,&nbsp;actual)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>); }</pre> </p> <p> Again, when the assertion passes, it returns no errors; otherwise, it returns a collection with a single error message. </p> <p> Using it may look like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode);</pre> </p> <p> At this point, each of the assertions are objects that represent a verification step. By themselves, they neither pass nor fail the test. You have to execute them to reach a verdict. </p> <h3 id="29825aed26124471803324863ea7d897"> Evaluating assertions <a href="#29825aed26124471803324863ea7d897" title="permalink">#</a> </h3> <p> The above code listing of the <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> class already shows how to combine two <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> objects into one. The <code>And</code> instance method is a binary operation that, together with the parameterless constructor, makes <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> a <a href="/2017/10/06/monoids">monoid</a>. </p> <p> Once you've combined all assertions into a single <code><span style="color:#2b91af;">Asserted</span>&lt;<span style="color:#2b91af;">T</span>&gt;</code> object, you need to <code>Run</code> it to produce a test outcome: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;Run(<span style="color:blue;">this</span>&nbsp;Asserted&lt;<span style="color:blue;">string</span>&gt;&nbsp;assertions) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(assertions?.Errors.Any()&nbsp;??&nbsp;<span style="color:blue;">false</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;messages&nbsp;=&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;assertions.Errors); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(messages); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> If there are no errors, <code>Run</code> does nothing; otherwise it combines all the error messages together and throws an exception. As was also the case in the previous article, I've allowed myself a few proof-of-concept shortcuts. <a href="https://learn.microsoft.com/dotnet/standard/design-guidelines/using-standard-exception-types">The framework design guidelines admonishes against throwing System.Exception</a>. It might be more appropriate to introduce a new <code>Exception</code> type that also allows enumerating the error messages. </p> <p> The entire <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">assertion phase</a> of the test looks like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Asserted.True( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); <span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); assertResponse.And(assertState).Run();</pre> </p> <p> You can see the entire test in the previous article. Notice how the two assertion objects are first combined into one with the <code>And</code> binary operation. The result is a single <code>Asserted&lt;<span style="color:blue;">string</span>&gt;</code> object on which you can call <code>Run</code>. </p> <p> Like the previous proof of concept, this assertion passes and fails in the same way. It's possible to compose assertions and collect error messages, instead of short-circuiting on the first failure, even without an applicative functor. </p> <h3 id="f3d9baf1ad564c05be99b32f0b4a1017"> Method chaining <a href="#f3d9baf1ad564c05be99b32f0b4a1017" title="permalink">#</a> </h3> <p> If you don't like to come up with variable names just to make assertions, it's also possible to use the <code>Asserted</code> API's <a href="https://martinfowler.com/bliki/FluentInterface.html">fluent interface</a>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); Asserted &nbsp;&nbsp;&nbsp;&nbsp;.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;.And(Asserted.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode)) &nbsp;&nbsp;&nbsp;&nbsp;.Run();</pre> </p> <p> This isn't necessarily better, but it's an option. </p> <h3 id="ecabe91c2b9949edbcb4cfed5213ec72"> Conclusion <a href="#ecabe91c2b9949edbcb4cfed5213ec72" title="permalink">#</a> </h3> <p> While it's possible to design non-short-circuiting composable assertions using an applicative functor, it looks as though a simpler solution might solve the same problem. Collect error messages. If none were collected, interpret that as a success. </p> <p> As I wrote in the <a href="/2022/11/07/applicative-assertions">introduction article</a>, however, this may not be the last word. Some assertions return values that can be used for other assertions. That's a scenario that I have not yet investigated in this light, and it may change the conclusion. If so, I'll add more articles to this small article series. As I'm writing this, though, I have no such plans. </p> <p> Did I just, in a roundabout way, write that <em>more research is needed?</em> </p> <p> <strong>Next:</strong> <a href="/2023/01/30/built-in-alternatives-to-applicative-assertions">Built-in alternatives to applicative assertions</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="2aae9a7f4c1145aa92b487925f4b46ba"> <div class="comment-author"><a href="https://ptupitsyn.github.io/">Pavel Tupitsyn</a></div> <div class="comment-content"> <p> I think NUnit's <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">Assert.Multiple</a> is worth mentioning in this series. It does not require any complicated APIs, just wrap your existing test with multiple asserts into a delegate. </p> </div> <div class="comment-date">2022-12-20 08:02 UTC</div> </div> <div class="comment" id="5b34eb2d997f417bac27738ee7cbb16d"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Pavel, thank you for writing. I'm aware of both that API and similar ones for other testing frameworks. As is usually the case, there are trade-offs to consider. I'm currently working on some material that may turn into another article about that. </p> </div> <div class="comment-date">2022-12-21 20:17 UTC</div> </div> <div class="comment" id="f244b81042414b98be67a937cf652648"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> A new article is now available: <a href="/2023/01/30/built-in-alternatives-to-applicative-assertions">Built-in alternatives to applicative assertions</a>. </p> </div> <div class="comment-date">2023-01-30 12:31 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2022/12/19/error-accumulating-composable-assertions-in-c When do tests fail? https://blog.ploeh.dk/2022/12/12/when-do-tests-fail/ Mon, 12 Dec 2022 08:33:00 UTC <div id="post"> <p> <em>Optimise for the common scenario.</em> </p> <p> Unit tests occasionally fail. When does that happen? How often? What triggers it? What information is important when tests fail? </p> <p> Regularly I encounter the viewpoint that it should be easy to understand the purpose of a test <em>when it fails</em>. Some people consider test names important, a topic that <a href="/2022/06/13/some-thoughts-on-naming-tests">I've previously discussed</a>. Recently I discussed the <a href="http://xunitpatterns.com/Assertion%20Roulette.html">Assertion Roulette</a> test smell on Twitter, and again I learned some surprising things about what people value in unit tests. </p> <h3 id="6673d3ef3f374f849fe7fe38a7b96985"> The importance of clear assertion messages <a href="#6673d3ef3f374f849fe7fe38a7b96985" title="permalink">#</a> </h3> <p> The Assertion Roulette test smell is often simplified to degeneracy, but it really describes situations where it may be a problem if you can't tell which of several assertions actually caused a test to fail. </p> <p> <a href="https://www.joshka.net">Josh McKinney</a> gave a more detailed example than Gerard Meszaros does in <a href="/ref/xunit-patterns">the book</a>: </p> <blockquote> <p> "Background. In a legacy product, we saw some tests start failing intermittently. They weren’t just flakey, but also failed without providing enough info to fix. One of things which caused time to fix to increase was multiple ways of a single test to fail." </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572528796125003777">Josh McK</a></cite></footer> </blockquote> <p> He goes on: </p> <blockquote> <p> "I.e. if you fix the first assertion and you know there still could be flakiness, or long cycle times to see the failure. Multiple assertions makes any test problem worse. In an ideal state, they are fine, but every assertion doubles the amount of failures a test catches." </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572529894361534464">Josh McK</a></cite></footer> </blockquote> <p> and concludes: </p> <blockquote> <p> "the other main way (unrelated) was things like: </p> <p> assertTrue(someListResult.isRmpty()) </p> <p> Which tells you what failed, but nothing about how. </p> <p> But the following is worse. You must run the test twice to fix: </p> <p> assertFalse(someList.isEmpty());<br/> assertEqual(expected, list.get(0));" </p> <footer><cite><a href="https://twitter.com/joshuamck/status/1572534297403469824">Josh McK</a></cite></footer> </blockquote> <p> The final point is due to the short-circuiting nature of most assertion libraries. That, however, is <a href="/2022/11/07/applicative-assertions">a solvable problem</a>. </p> <p> I find the above a compelling example of why Assertion Roulette may be problematic. </p> <p> It did give me pause, though. How common is this scenario? </p> <h3 id="5ae99f4515444bd6ba483beffb819557"> Out of the blue <a href="#5ae99f4515444bd6ba483beffb819557" title="permalink">#</a> </h3> <p> The situation described by Josh McKinney comes with more than a single warning flag. I hope that it's okay to point some of them out. I didn't get the impression from my interaction with Josh McKinney that he considered the situation ideal in any way. </p> <p> First, of course, there's the lack of information about the problem. Here, that's a real problem. As I understand it, it makes it harder to reproduce the problem in a development environment. </p> <p> Next, there's long cycle times, which I interpret as significant time may pass from when you attempt a fix until you can actually observe whether or not it worked. Josh McKinney doesn't say how long, but I wouldn't surprised if it was measured in days. At least, if the cycle time is measured in days, I can see how this is a problem. </p> <p> Finally, there's the observation that "some tests start failing intermittently". This was the remark that caught my attention. How often does that happen? </p> <p> Tests shouldn't do that. Tests should be deterministic. If they're not, you should work to <a href="https://martinfowler.com/articles/nonDeterminism.html">eradicate non-determinism in tests</a>. </p> <p> I'll be the first to admit that that I also write non-deterministic tests. Not by design, but because I make mistakes. I've written many <a href="http://xunitpatterns.com/Erratic%20Test.html">Erratic Tests</a> in my career, and I've documented a few of them here: </p> <ul> <li><a href="/2021/01/11/waiting-to-happen">Waiting to happen</a></li> <li><a href="/2022/05/23/waiting-to-never-happen">Waiting to never happen</a></li> <li><a href="/2020/10/05/fortunately-i-dont-squash-my-commits">Fortunately, I don't squash my commits</a></li> <li><a href="/2016/01/18/make-pre-conditions-explicit-in-property-based-tests">Make pre-conditions explicit in Property-Based Tests</a></li> </ul> <p> While it <em>can</em> happen, it shouldn't be the norm. When it nonetheless happens, eradicating that source of non-determinism should be top priority. Pull the <a href="https://en.wikipedia.org/wiki/Andon_(manufacturing)">andon cord</a>. </p> <h3 id="5fd251665441473ca3048a9137f9bc2b"> When tests fail <a href="#5fd251665441473ca3048a9137f9bc2b" title="permalink">#</a> </h3> <p> Ideally, tests should rarely fail. As examined above, you may have Erratic Tests in your test suite, and if you do, these tests will occasionally (or often) fail. As Martin Fowler writes, this is a problem and you should do something about it. He also outlines strategies for it. </p> <p> Once you've eradicated non-determinism in unit tests, then when do tests fail? </p> <p> I can think of a couple of situations. </p> <p> Tests routinely fail as part of the <a href="/2019/10/21/a-red-green-refactor-checklist">red-green-refactor cycle</a>. This is by design. If no test is failing in the <em>red</em> phase, you probably made a mistake (which also regularly <a href="/2019/10/14/tautological-assertion">happens to me</a>), or you may not really be doing test-driven development (TDD). </p> <p> Another situation that may cause a test to fail is if you changed some code and triggered a regression test. </p> <p> In both cases, tests don't just fail <a href="https://amzn.to/3SPdHAO">out of the blue</a>. They fail as an immediate consequence of something you did. </p> <h3 id="aaeebfa1a96b47398219817bc3327a9c"> Optimise for the common scenario <a href="#aaeebfa1a96b47398219817bc3327a9c" title="permalink">#</a> </h3> <p> In both cases you're (hopefully) in a tight feedback loop. If you're in a tight feedback loop, then how important is the assertion message really? How important is the test name? </p> <p> You work on the code base, make some changes, run the tests. If one or more tests fail, it's correlated to the change you just made. You should have a good idea of what went wrong. Are code forensics and elaborate documentation really necessary to understand a test that failed because you just did something a few minutes before? </p> <p> The reason I don't care much about test names or whether there's one or more assertion in a unit test is exactly that: When tests fail, it's usually because of something I just did. I don't need diagnostics tools to find the root cause. The root cause is the change that I just made. </p> <p> That's my common scenario, and I try to optimise my processes for the common scenarios. </p> <h3 id="21b16c9b2fb649ec859e53b0a1ab431a"> Fast feedback <a href="#21b16c9b2fb649ec859e53b0a1ab431a" title="permalink">#</a> </h3> <p> There's an implied way of working that affects such attitudes. Since I learned about TDD in 2003 I've always relished the fast feedback I get from a test suite. Since I tried continuous deployment around 2014, I consider it central to <a href="/ref/modern-software-engineering">modern software engineering</a> (and <a href="/ref/accelerate">Accelerate</a> strongly suggests so, too). </p> <p> The modus operandi I outline above is one of fast feedback. If you're sitting on a feature branch for weeks before integrating into master, or if you can only deploy two times a year, this influences what works and what doesn't. </p> <p> Both <em>Modern Software Engineering</em> and <em>Accelerate</em> make a strong case that short feedback cycles are pivotal for successful software development organisations. </p> <p> I also understand that that's not the reality for everyone. When faced with long cycle times, a multitude of Erratic Tests, a legacy code base, and so on, other things become important. In those circumstances, tests may fail for different reasons. </p> <p> When you work with TDD, continuous integration (CI), and continuous deployment (CD), then when do tests fail? They fail because you made them fail, only minutes earlier. Fix your code and move forward. </p> <h3 id="53cc6ce19bad4bdea3f40fd752f6338d"> Conclusion <a href="#53cc6ce19bad4bdea3f40fd752f6338d" title="permalink">#</a> </h3> <p> When discussing test names and assertion messages, I've been surprised by the emphasis some people put on what I consider to be of secondary importance. I think the explanation is that circumstances differ. </p> <p> With TDD and CI/CD you mostly look at a unit test when you write it, or if some regression test fails because you changed some code (perhaps in response to a test you just wrote). Your test suite may have hundreds or thousands of tests. Most of these pass every time you run the test suite. That's the normal state of affairs. </p> <p> In other circumstances, you may have Erratic Tests that fail unpredictably. You should make it a priority to stop that, but as part of that process, you may need good assertion messages and good test names. </p> <p> Different circumstances call for different reactions, so what works well in one situation may be a liability in other situations. I hope that this article has shed a little light on the forces you may want to consider. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2022/12/12/when-do-tests-fail GitHub Copilot preliminary experience report https://blog.ploeh.dk/2022/12/05/github-copilot-preliminary-experience-report/ Mon, 05 Dec 2022 08:37:00 UTC <div id="post"> <p> <em>Based on a few months of use.</em> </p> <p> I've been evaluating <a href="https://github.com/features/copilot">GitHub Copilot</a> since August 2022. Perhaps it's time to collect my thoughts so far. </p> <p> In short, it's surprisingly good, but also gets a lot of things wrong. It does seem helpful to the experienced programmer, but I don't see it replacing all programmers yet. </p> <h3 id="19d8478cc2d5429f9c62138574a30a45"> Not only for boilerplate code <a href="#19d8478cc2d5429f9c62138574a30a45" title="permalink">#</a> </h3> <p> I was initially doubtful. I'd seen some demos where Copilot created fifteen to twenty lines of code to, say, make a REST API request. These examples mostly struck me as auto-generation of something that ought to be a proper abstraction: A method in a reusable library. </p> <p> In general <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">I don't think that typing is a productivity bottleneck</a>, and I'm <a href="/2013/02/04/BewareofProductivityTools">sceptical of productivity tools</a>, and particularly code generators. The more code a code base contains, the more code there is to read. Accelerating code production doesn't strike me as a goal in itself. </p> <p> On the other hand, I'm past fifty and aware that in my age, one often tends to resist new ideas. Perhaps the above justifications are nothing but <a href="https://en.wikipedia.org/wiki/Motivated_reasoning">motivated reasoning</a>. I decided to install the plug-ins and go about my business to see what would happen. </p> <p> It has surprised and delighted me more than once. It's not only for boilerplate code. Here's a recent example. </p> <p> I was doing an exercise in <a href="https://fsharp.org">F#</a> and wrote this type definition: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Instruction&nbsp;=&nbsp;Enqueue&nbsp;<span style="color:blue;">of</span>&nbsp;int&nbsp;|&nbsp;Dequeue</pre> </p> <p> I wanted to write a property-based test with this type, and since I was using <a href="https://hedgehog.qa/">Hedgehog</a> I needed a generator for the type. While <a href="https://fscheck.github.io/FsCheck/">FsCheck</a> can automatically generate values of custom <a href="https://learn.microsoft.com/dotnet/fsharp/language-reference/discriminated-unions">discriminated unions</a>, Hedgehog has a more <a href="https://peps.python.org/pep-0020/">explicit-is-better-than-implicit</a> design philosophy. In other words, if there's a way to make Hedgehog automatically generate values of a type like the above <code>Instruction</code>, it has so far escaped me. </p> <p> So I started typing <code><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;genInstruction&nbsp;=</code> after which Copilot suggested the following: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;<span style="color:blue;">private</span>&nbsp;genInstruction&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Gen.frequency &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[&nbsp;1,&nbsp;Gen.constant&nbsp;Dequeue &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1,&nbsp;Gen.int32&nbsp;(Range.constant&nbsp;0&nbsp;10_000)&nbsp;|&gt;&nbsp;Gen.map&nbsp;Enqueue&nbsp;]</pre> </p> <p> I was, frankly, stunned. While I do understand that Copilot doesn't actually understand the code it suggests, Hedgehog is sufficiently esoteric that I didn't expect Copilot to have enough training data to enable it to make a useful contribution in this niche. I was wrong. I've repeatedly seen Copilot make suggestions to my F# and <a href="https://www.haskell.org/">Haskell</a> code. It's not just for C#, <a href="https://www.javascript.com">JavaScript</a>, or <a href="https://www.python.org/">python</a> code. </p> <p> The above suggestion was, to be clear, absolutely appropriate and compiled right away. The only detail I decided to change was the <code>Range</code>, which I decided to change to <code>Range.linear</code>. That's not, however, a significant change. </p> <p> Perhaps you're not impressed by three lines of auto-generated code. How much of a productivity improvement is that? Quite a bit, in my case. </p> <p> It wouldn't have taken me long to type those three lines of code, but as I already mentioned, <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">typing isn't a bottleneck</a>. On the other hand, looking up an unfamiliar API can take some time. <a href="/ref/programmers-brain">The Programmer's Brain</a> discusses this kind of problem and suggests exercises to address it. Does Copilot offer a shortcut? </p> <p> While I couldn't remember the details of Hedgehog's API, once I saw the suggestion, I recognised <code>Gen.frequency</code>, so I understood it as an appropriate code suggestion. The productivity gain, if there is one, may come from saving you the effort of looking up unfamiliar APIs, rather than saving you some keystrokes. </p> <p> In this example, I already knew of the <code>Gen.frequency</code> function - I just couldn't recall the exact name and type. This enabled me to evaluate Copilot's suggestion and deem it correct. If I hadn't known that API already, how could I have known whether to trust Copilot? </p> <h3 id="424f6a4928d945d7bcb13f2de8f3a98e"> Detectably wrong suggestions <a href="#424f6a4928d945d7bcb13f2de8f3a98e" title="permalink">#</a> </h3> <p> As amazing as Copilot can be, it's hardly faultless. It makes many erroneous suggestions. Sometimes the suggestion is obviously wrong. If you accept it, it doesn't compile. Sometimes, the compilation error is only a little edit from being correct, but at least in such situations you'll be explicitly aware that the suggestion couldn't be used verbatim. </p> <p> Other suggestions are wrong, but less conspicuously so. Here's an example. </p> <p> I was recently subjecting the code base that accompanies <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a> to the <a href="https://en.wikipedia.org/wiki/Mutation_testing">mutation testing</a> tool <a href="https://stryker-mutator.io/">Stryker</a>. Since it did point out a few possible mutations, I decided to add a few tests. One was of a wrapper class called <code>TimeOfDay</code>. Because of static code analysis rules, it came with conversions to and from <a href="https://learn.microsoft.com/dotnet/api/system.timespan">TimeSpan</a>, but these methods weren't covered by any tests. </p> <p> In order to remedy that situation, I started writing an FsCheck property and came as far as: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>)</pre> </p> <p> At that point Copilot suggested the following, which I accepted: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">timeOfDay</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;TimeOfDay(timeSpan); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;(TimeSpan)timeOfDay; &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(timeSpan,&nbsp;actual); }</pre> </p> <p> Looks good, doesn't it? Again, I was impressed. It compiled, and it even looks as though Copilot had picked up one of my naming conventions: <a href="/2020/11/30/name-by-role">naming variables by role</a>, in this case <code>actual</code>. </p> <p> While I tend to be on guard, I immediately ran the test suite instead of thinking it through. It failed. Keep in mind that this is a <a href="https://en.wikipedia.org/wiki/Characterization_test">characterisation test</a>, so it was supposed to pass. </p> <p> The <code>TimeOfDay</code> constructor reveals why: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TimeOfDay</span>(TimeSpan&nbsp;<span style="color:#1f377f;">durationSinceMidnight</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(durationSinceMidnight&nbsp;&lt;&nbsp;TimeSpan.Zero&nbsp;|| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TimeSpan.FromHours(24)&nbsp;&lt;&nbsp;durationSinceMidnight) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;ArgumentOutOfRangeException( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nameof(durationSinceMidnight), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Please&nbsp;supply&nbsp;a&nbsp;TimeSpan&nbsp;between&nbsp;0&nbsp;and&nbsp;24&nbsp;hours.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.durationSinceMidnight&nbsp;=&nbsp;durationSinceMidnight; }</pre> </p> <p> While FsCheck knows how to generate <code>TimeSpan</code> values, it'll generate arbitrary durations, including negative values and spans much longer than 24 hours. That explains why the test fails. </p> <p> Granted, this is hardly a searing indictment against Copilot. After all, I could have made this mistake myself. </p> <p> Still, that prompted me to look for more issues with the code that Copilot had suggested. Another problem with the code is that it tests the wrong API. The suggested test tries to round-trip via the <code>TimeOfDay</code> class' explicit cast operators, which were already covered by tests. Well, I might eventually have discovered that, too. Keep in mind that I was adding this test to improve the code base's Stryker score. After running the tool again, I would probably eventually have discovered that the score didn't improve. It takes Stryker around 25 minutes to test this code base, though, so it wouldn't have been rapid feedback. </p> <p> Since, however, I examined the code with a critical eye, I noticed this by myself. This would clearly require changing the test code as well. </p> <p> In the end, I wrote this test: </p> <p> <pre>[Property] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConversionsRoundTrip</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">expected</span>&nbsp;=&nbsp;ScaleToTimeOfDay(timeSpan); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">sut</span>&nbsp;=&nbsp;TimeOfDay.ToTimeOfDay(expected); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;TimeOfDay.ToTimeSpan(sut); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;TimeSpan&nbsp;<span style="color:#74531f;">ScaleToTimeOfDay</span>(TimeSpan&nbsp;<span style="color:#1f377f;">timeSpan</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Convert&nbsp;an&nbsp;arbitrary&nbsp;TimeSpan&nbsp;to&nbsp;a&nbsp;24-hour&nbsp;TimeSpan.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;The&nbsp;data&nbsp;structure&nbsp;that&nbsp;underlies&nbsp;TimeSpan&nbsp;is&nbsp;a&nbsp;64-bit&nbsp;integer,</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;so&nbsp;first&nbsp;we&nbsp;need&nbsp;to&nbsp;identify&nbsp;the&nbsp;range&nbsp;of&nbsp;possible&nbsp;TimeSpan</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;values.&nbsp;It&nbsp;might&nbsp;be&nbsp;easier&nbsp;to&nbsp;understand&nbsp;to&nbsp;calculate</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;TimeSpan.MaxValue&nbsp;-&nbsp;TimeSpan.MinValue,&nbsp;but&nbsp;that&nbsp;underflows.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Instead,&nbsp;the&nbsp;number&nbsp;of&nbsp;possible&nbsp;64-bit&nbsp;integer&nbsp;values&nbsp;is&nbsp;the&nbsp;same</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;as&nbsp;the&nbsp;number&nbsp;of&nbsp;possible&nbsp;unsigned&nbsp;64-bit&nbsp;integer&nbsp;values.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">range</span>&nbsp;=&nbsp;<span style="color:blue;">ulong</span>.MaxValue; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">domain</span>&nbsp;=&nbsp;TimeSpan.FromHours(24).Ticks; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">scale</span>&nbsp;=&nbsp;(<span style="color:blue;">ulong</span>)domain&nbsp;/&nbsp;range; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">expected</span>&nbsp;=&nbsp;timeSpan&nbsp;*&nbsp;scale; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;expected; }</pre> </p> <p> In this case, Copilot didn't improve my productivity. It may actually have slowed me down a bit. </p> <p> This time, it wasn't too difficult to spot issues with the suggested code. What if the problems are more subtle? </p> <h3 id="02cb23fc57394a1c963a2f4ebe75ec48"> Errors that are difficult to detect <a href="#02cb23fc57394a1c963a2f4ebe75ec48" title="permalink">#</a> </h3> <p> How do bugs appear? We write them, thinking that our code is going to do one thing, while the compiler decides otherwise. Even when we actively engage with the code, we may be looking at a bug and still fail to see it. </p> <p> Why should we trust Copilot to produce bug-free code? </p> <p> Here's another example. I was going through the <a href="https://codingdojo.org/kata/Range/">Range kata</a> in F# and made my <code>Endpoint</code> discriminated union a <a href="/2018/03/22/functors">functor</a>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Endpoint&lt;&#39;a&gt;&nbsp;=&nbsp;Open&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;a&nbsp;|&nbsp;Closed&nbsp;<span style="color:blue;">of</span>&nbsp;&#39;a <span style="color:blue;">module</span>&nbsp;Endpoint&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;map&nbsp;f&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Open&nbsp;&nbsp;&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Open&nbsp;&nbsp;&nbsp;(f&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Closed&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Closed&nbsp;(f&nbsp;x)</pre> </p> <p> Making a type a functor is such a well-understood process that <a href="https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/derive-functor">Haskell can automate it</a>. I knew how to write the above <code>map</code> function as well, so I wrote it without testing it. </p> <p> After I'd written the <code>map</code> function, I decided to add tests of the two functor laws. Not because I doubted the implementation, but because I might need that as example code later. </p> <p> Copilot suggested this test after I named it: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Xunit <span style="color:blue;">open</span>&nbsp;Hedgehog [&lt;Fact&gt;] <span style="color:blue;">let</span>&nbsp;``First&nbsp;functor&nbsp;law``&nbsp;()&nbsp;=&nbsp;Property.check&nbsp;&lt;|&nbsp;property&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&nbsp;=&nbsp;Gen.int32&nbsp;(Range.linearBounded&nbsp;()) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;f&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;+&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;g&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;*&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Endpoint.map&nbsp;f&nbsp;(Endpoint.map&nbsp;g&nbsp;(Open&nbsp;x)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;Endpoint.map&nbsp;(f&nbsp;&gt;&gt;&nbsp;g)&nbsp;(Open&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual)&nbsp;}</pre> </p> <p> It compiles. It also fails. </p> <p> What's wrong with it? </p> <p> If you know the functor laws by heart, you'll be able to spot that this is clearly not the first functor law. On the other hand, it looks convincingly like the second functor law. Should I just change the name and move on? </p> <p> I can't, though, since the test fails. Could there be a bug in my <code>map</code> function, after all? </p> <p> No, there's an error in the test. I invite you to spot it. </p> <p> In terms of keystrokes, it's easy to fix the problem: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Xunit <span style="color:blue;">open</span>&nbsp;Hedgehog [&lt;Fact&gt;] <span style="color:blue;">let</span>&nbsp;``First&nbsp;functor&nbsp;law``&nbsp;()&nbsp;=&nbsp;Property.check&nbsp;&lt;|&nbsp;property&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;x&nbsp;=&nbsp;Gen.int32&nbsp;(Range.linearBounded&nbsp;()) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;f&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;+&nbsp;1) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;g&nbsp;=&nbsp;(<span style="color:blue;">fun</span>&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;x&nbsp;*&nbsp;2) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;Endpoint.map&nbsp;f&nbsp;(Endpoint.map&nbsp;g&nbsp;(Open&nbsp;x)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;expected&nbsp;=&nbsp;Endpoint.map&nbsp;(f&nbsp;&lt;&lt;&nbsp;g)&nbsp;(Open&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(expected,&nbsp;actual)&nbsp;}</pre> </p> <p> Spot the edit. I bet it'll take you longer to find it than it took me to type it. </p> <p> The test now passes, but for one who has spent less time worrying over functor laws than I have, troubleshooting this could have taken a long time. </p> <p> These almost-right suggestions from Copilot both worry me and give me hope. </p> <h3 id="7c90d2bd03054906a5f2505baaf78a31"> Copilot for experienced programmers <a href="#7c90d2bd03054906a5f2505baaf78a31" title="permalink">#</a> </h3> <p> When a new technology like Copilot appears, it's natural to speculate on the consequences. <em>Does this mean that programmers will lose their jobs?</em> </p> <p> This is just a preliminary evaluation after a few months, so I could be wrong, but I think we programmers are safe. If you're experienced, you'll be able to tell most of Copilot's hits from its misses. Perhaps you'll get a productivity improvement out of, but it could also slow you down. </p> <p> The tool is likely to improve over time, so I'm hopeful that this could become a net productivity gain. Still, with this high an error rate, I'm not too worried yet. </p> <p> <a href="/ref/pragmatic-programmer">The Pragmatic Programmer</a> describes a programming style named <em>Programming by Coincidence</em>. People who develop software this way have only a partial understanding of the code they write. </p> <blockquote> <p> "Fred doesn't know why the code is failing because <em>he didn't know why it worked in the first place.</em>" </p> <footer><cite>Andy Hunt and Dave Thomas, <a href="/ref/pragmatic-programmer">The Pragmatic Programmer</a></cite></footer> </blockquote> <p> I've encountered my fair share of these people. When editing code, they make small adjustments and do cursory manual testing until 'it looks like it works'. If they have to start a new feature or are otherwise faced with a metaphorical blank page, they'll copy some code from somewhere else and use that as a starting point. </p> <p> You'd think that Copilot could enhance the productivity of such people, but I'm not sure. It might actually slow them down. These people don't fully understand the code they themselves 'write', so why should we expect them to understand the code that Copilot suggests? </p> <p> If faced with a Copilot suggestion that 'almost works', will they be able to spot if it's a genuinely good suggestion, or whether it's off, like I've described above? If the Copilot code doesn't work, how much time will they waste thrashing? </p> <h3 id="f3e17cbf9dbc41f19ae46974d2f28a90"> Conclusion <a href="#f3e17cbf9dbc41f19ae46974d2f28a90" title="permalink">#</a> </h3> <p> GitHub Copilot has the potential to be a revolutionary technology, but it's not, yet. So far, I'm not too worried. It's an assistant, like a pairing partner, but it's up to you to evaluate whether the code that Copilot suggests is useful, correct, and safe. How can you do that unless you already know what you're doing? </p> <p> If you don't have the qualifications to evaluate the suggested code, I fail to see how it's going to help you. Granted, it does have potential to help you move on in less time that you would otherwise have spent. In this article, I showed one example where I would have had to spend significant time looking up API documentation. Instead, Copilot suggested the correct code to use. </p> <p> Pulling in the other direction are the many <a href="https://en.wikipedia.org/wiki/False_positives_and_false_negatives">false positives</a>. Copilot makes many suggestions, and many of them are poor. The ones that are recognisably bad are unlikely to slow you down. I'm more concerned with those that are subtly wrong. They have the potential to waste much time. </p> <p> Which of these forces are strongest? The potential for wasting time is infinite, while the maximum productivity gain you can achieve is 100 percent. That's an asymmetric distribution. There's a long tail of time wasters, but there's no equivalent long tail of improvement. </p> <p> I'm not, however, trying to be pessimistic. I expect to keep Copilot around for the time being. It could very well be here to stay. Used correctly, it seems useful. </p> <p> Is it going to replace programmers? Hardly. Rather, it may enable poor developers to make such a mess of things that you need even more good programmers to subsequently fix things. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2022/12/05/github-copilot-preliminary-experience-report An initial proof of concept of applicative assertions in C# https://blog.ploeh.dk/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c/ Mon, 28 Nov 2022 06:47:00 UTC <div id="post"> <p> <em>Worthwhile? Not obviously.</em> </p> <p> This article is the first instalment in a small articles series about <a href="/2022/11/07/applicative-assertions">applicative assertions</a>. It explores a way to compose assertions in such a way that failure messages accumulate rather than short-circuit. It assumes that you've read the <a href="/2022/11/07/applicative-assertions">article series introduction</a>. </p> <p> Assertions are typically based on throwing exceptions. As soon as one assertion fails, an exception is thrown and no further assertions are evaluated. This is normal short-circuiting behaviour of exceptions. In some cases, however, it'd be useful to keep evaluating other assertions and collect error messages. </p> <p> This article series explores <a href="https://twitter.com/lucasdicioccio/status/1572264819109003265">an intriguing idea</a> to address such issues: Use an <a href="/2018/10/01/applicative-functors">applicative functor</a> to collect multiple assertion messages. I started experimenting with the idea to see where it would lead. The article series serves as a report of what I found. It is neither a recommendation nor a caution. I still find the idea interesting, but I'm not sure whether the complexity is warranted. </p> <h3 id="877c6f5ee5b34ecf990d1126b8139720"> Example scenario <a href="#877c6f5ee5b34ecf990d1126b8139720" title="permalink">#</a> </h3> <p> A realistic example is often illustrative, although there's a risk that the realism carries with it some noise that detracts from the core of the matter. I'll reuse an example that I've <a href="https://stackoverflow.blog/2022/11/03/multiple-assertions-per-test-are-fine/">already discussed and explained in greater detail</a>. The code is from the code base that accompanies my book <a href="/2021/06/14/new-book-code-that-fits-in-your-head">Code That Fits in Your Head</a>. </p> <p> This test has two independent assertions: </p> <p> <pre>[Theory] [InlineData(884,&nbsp;18,&nbsp;47,&nbsp;<span style="color:#a31515;">&quot;c@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Nick&nbsp;Klimenko&quot;</span>,&nbsp;2)] [InlineData(902,&nbsp;18,&nbsp;50,&nbsp;<span style="color:#a31515;">&quot;emot@example.gov&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Emma&nbsp;Otting&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;Task&nbsp;DeleteReservation( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days,&nbsp;<span style="color:blue;">int</span>&nbsp;hours,&nbsp;<span style="color:blue;">int</span>&nbsp;minutes,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;LegacyApi(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;DateTime.Today.AddDays(days).At(hours,&nbsp;minutes) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToIso8601DateTimeString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;Create.ReservationDto(at,&nbsp;email,&nbsp;name,&nbsp;quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;postResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.PostReservation(dto); &nbsp;&nbsp;&nbsp;&nbsp;Uri&nbsp;address&nbsp;=&nbsp;FindReservationAddress(postResp); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;deleteResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().DeleteAsync(address); &nbsp;&nbsp;&nbsp;&nbsp;Assert.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); &nbsp;&nbsp;&nbsp;&nbsp;Assert.Equal(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); }</pre> </p> <p> The test exercises the REST API to first create a reservation, then delete it, and finally check that the reservation no longer exists. Two independent postconditions must be true for the test to pass: </p> <ul> <li>The <code>DELETE</code> request must result in a status code that indicates success.</li> <li>The resource must no longer exist.</li> </ul> <p> It's conceivable that a bug might fail one of these without invalidating the other. </p> <p> As the test is currently written, it uses <a href="https://xunit.net/">xUnit.net</a>'s standard assertion library. If the <code>Assert.True</code> verification fails, the <code>Assert.Equal</code> statement isn't evaluated. </p> <h3 id="cd132aaee5484068a890cc4b7995bed3"> Assertions as validations <a href="#cd132aaee5484068a890cc4b7995bed3" title="permalink">#</a> </h3> <p> Is it possible to evaluate the <code>Assert.Equal</code> postcondition even if the first assertion fails? You could use a <code>try/catch</code> block, but is there a more composable and elegant option? How about an applicative functor? </p> <p> Since I was interested in exploring this question as a proof of concept, I decided to reuse the machinery that I'd already put in place for the article <a href="/2022/07/25/an-applicative-reservation-validation-example-in-c">An applicative reservation validation example in C#</a>: The <code>Validated</code> class and its associated functions. In a sense, you can think of an assertion as a validation of a postcondition. </p> <p> This is not a resemblance I intend to carry too far. What I learn by experimenting with <code>Validated</code> I can apply to a more appropriately-named class like <code>Asserted</code>. </p> <p> Neither of the two above assertions return a value; they are one-stop assertions. If they succeed, they return nothing; if they fail, they produce an error. </p> <p> It's possible to model this kind of behaviour with <code>Validated</code>. You can model a collection of errors with, well, a collection. To keep the proof of concept simple, I decided to use a collection of strings: <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. To model 'nothing' I had to add a <a href="/2018/01/15/unit-isomorphisms">unit</a> type: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Unit</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">Unit</span>()&nbsp;{&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Unit&nbsp;Value&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;Unit(); }</pre> </p> <p> This enabled me to define assertions as <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code> values: Either a collection of error messages, or nothing. </p> <h3 id="5ac18fb532c04876b00f7de080905a16"> Asserting truth <a href="#5ac18fb532c04876b00f7de080905a16" title="permalink">#</a> </h3> <p> Instead of xUnit.net's <code>Assert.True</code>, you can now define an equivalent function: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;condition, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;message) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;condition &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;message&nbsp;}); }</pre> </p> <p> It simply returns a <code>Success</code> value containing nothing when <code>condition</code> is <code>true</code>, and otherwise a <code>Failure</code> value containing the error <code>message</code>. </p> <p> You can use it like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Validated.AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> Later in the article you'll see how this assertion combines with another assertion. </p> <h3 id="adc98673e4734918ac983a5bbb520e15"> Asserting equality <a href="#adc98673e4734918ac983a5bbb520e15" title="permalink">#</a> </h3> <p> Instead of xUnit.net's <code>Assert.Equal</code>, you can also define a function that works the same way but returns a <code>Validated</code> value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;AssertEqual&lt;<span style="color:#2b91af;">T</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;expected, &nbsp;&nbsp;&nbsp;&nbsp;T&nbsp;actual) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Equals(expected,&nbsp;actual) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;?&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(Unit.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;Fail&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;(<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;<span style="color:#a31515;">$&quot;Expected&nbsp;</span>{expected}<span style="color:#a31515;">,&nbsp;but&nbsp;got&nbsp;</span>{actual}<span style="color:#a31515;">.&quot;</span>&nbsp;}); }</pre> </p> <p> The <code>AssertEqual</code> function first uses <a href="https://learn.microsoft.com/dotnet/api/system.object.equals">Equals</a> to compare <code>expected</code> with <code>actual</code>. If the result is <code>true</code>, the function returns a <code>Success</code> value containing nothing; otherwise, it returns a <code>Failure</code> value containing a failure message. Since this is only a proof of concept, the failure message is useful, but minimal. </p> <p> Notice that this function returns a value of the same type (<code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>) as <code>AssertTrue</code>. </p> <p> You can use the function like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertState&nbsp;=&nbsp;Validated.AssertEqual(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode);</pre> </p> <p> Again, you'll see how to combine this assertion with the above <code>assertResponse</code> value later in this article. </p> <h3 id="1d29ecff5ea64911918168cb985ed6b5"> Evaluating assertions <a href="#1d29ecff5ea64911918168cb985ed6b5" title="permalink">#</a> </h3> <p> The <code>DeleteReservation</code> test only has two independent assertions, so in my proof of concept, all I needed to do was to figure out a way to combine two applicative assertions into one, and then evaluate it. This rather horrible method does that: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;RunAssertions( &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;assertion1, &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;assertion2) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;f&nbsp;=&nbsp;Succeed&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Func&lt;Unit,&nbsp;Unit,&nbsp;Unit&gt;&gt;((_,&nbsp;__)&nbsp;=&gt;&nbsp;Unit.Value); &nbsp;&nbsp;&nbsp;&nbsp;Func&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;combine&nbsp;=&nbsp;(x,&nbsp;y)&nbsp;=&gt;&nbsp;x.Concat(y).ToArray(); &nbsp;&nbsp;&nbsp;&nbsp;Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;&nbsp;composition&nbsp;=&nbsp;f &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(assertion1,&nbsp;combine) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Apply(assertion2,&nbsp;combine); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;errors&nbsp;=&nbsp;composition.Match( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;onFailure:&nbsp;f&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Join(Environment.NewLine,&nbsp;f), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;onSuccess:&nbsp;_&nbsp;=&gt;&nbsp;<span style="color:blue;">string</span>.Empty); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!<span style="color:blue;">string</span>.IsNullOrEmpty(errors)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;Exception(errors);</pre> </p> <p> C# doesn't have good language features for applicative functors the same way that <a href="https://fsharp.org/">F#</a> and <a href="https://www.haskell.org/">Haskell</a> do, and although you can use various tricks to make the programming experience better that what is on display here, I was still doing a proof of concept. If it turns out that this approach is useful and warranted, we can introduce some of the facilities to make the API more palatable. For now, though, we're dealing with all the rough edges. </p> <p> The way that applicative functors work, you typically use a 'lifted' function to combine two (or more) 'lifted' values. Here, 'lifted' means 'being inside the <code>Validated</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers/">container</a>'. </p> <p> Each of the assertions that I want to combine has the same type: <code>Validated&lt;IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;,&nbsp;Unit&gt;</code>. Notice that the <code>S</code> (<em>success</em>) generic type argument is <code>Unit</code> in both cases. While it seems redundant, formally I needed a 'lifted' function to combine two <code>Unit</code> values into a single value. This single value can (in principle) have any type I'd like it to have, but since you can't extract any information out of a <code>Unit</code> value, it makes sense to use the <a href="/2018/01/15/unit-isomorphisms#4657dbd4b5fc4abda6e8dee2cac67ea9">monoidal nature of unit</a> to combine two into one. </p> <p> Basically, you just ignore the <code>Unit</code> input values because they carry no information. Also, they're all the same value anyway, since the type is a <a href="https://en.wikipedia.org/wiki/Singleton_pattern">Singleton</a>. In its 'naked' form, the function might be implemented like this: <code>(_,&nbsp;__)&nbsp;=&gt;&nbsp;Unit.Value</code>. Due to the <a href="/2019/12/16/zone-of-ceremony">ceremony</a> required by the combination of C# and applicative functors, however, this <a href="/2017/10/06/monoids">monoidal</a> binary operation has to be 'lifted' to a <code>Validated</code> value. That's the <code>f</code> value in the <code>RunAssertions</code> function body. </p> <p> The <code>Validated.Apply</code> function requires as an argument a function that combines the generic <code>F</code> (<em>failure</em>) values into one, in order to deal with the case where there's multiple failures. In this case <code>F</code> is <code>IReadOnlyCollection&lt;<span style="color:blue;">string</span>&gt;</code>. Since declarations of <code>Func</code> values in C# requires explicit type declaration, that's a bit of a mouthful, but the <code>combine</code> function just concatenates two collections into one. </p> <p> The <code>RunAssertions</code> method can now <code>Apply</code> both <code>assertion1</code> and <code>assertion2</code> to <code>f</code>, which produces a combined <code>Validated</code> value, <code>composition</code>. It then matches on the combined value to produce a <code>string</code> value. If there are no assertion messages, the result is the empty string; otherwise, the function combines the assertion messages with a <code>NewLine</code> between each. Again, this is proof-of-concept code. A more robust and flexible API (if warranted) might keep the errors around as a collection of strongly typed <a href="https://martinfowler.com/bliki/ValueObject.html">Value Objects</a>. </p> <p> Finally, if the resulting <code>errors</code> string is not null or empty, the <code>RunAssertions</code> method throws an exception with the combined error message(s). Here I once more invoked my proof-of-concept privilege to throw an <a href="https://learn.microsoft.com/dotnet/api/system.exception">Exception</a>, even though <a href="https://learn.microsoft.com/dotnet/standard/design-guidelines/using-standard-exception-types">the framework design guidelines admonishes against doing so</a>. </p> <p> Ultimately, then, the <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">assert phase</a> of the test looks like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;assertResponse&nbsp;=&nbsp;Validated.AssertTrue( &nbsp;&nbsp;&nbsp;&nbsp;deleteResp.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{deleteResp.StatusCode}<span style="color:#a31515;">.&quot;</span>); <span style="color:blue;">var</span>&nbsp;getResp&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(address); <span style="color:blue;">var</span>&nbsp;assertState&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Validated.AssertEqual(HttpStatusCode.NotFound,&nbsp;getResp.StatusCode); Validated.RunAssertions(assertResponse,&nbsp;assertState);</pre> </p> <p> The rest of the test hasn't changed. </p> <h3 id="42e68063b7a3472e960031fde5e3f40d"> Outcomes <a href="#42e68063b7a3472e960031fde5e3f40d" title="permalink">#</a> </h3> <p> Running the test with the applicative assertions passes, as expected. In order to verify that it works as it's supposed to, I tried to sabotage the System Under Test (SUT) in various ways. First, I made the <code>Delete</code> method that handles <code>DELETE</code> requests a <a href="https://en.wikipedia.org/wiki/NOP_(code)">no-op</a>, while still returning <code>200 OK</code>. As you'd expect, the result is a test failure with this message: </p> <p> <pre>Message: System.Exception : Expected NotFound, but got OK.</pre> </p> <p> This is the assertion that verifies that <code>getResp.StatusCode</code> is <code>404 Not Found</code>. It fails because the sabotaged <code>Delete</code> method doesn't delete the reservation. </p> <p> Then I further sabotaged the SUT to also return an incorrect status code (<code>400 Bad Request</code>), which produced this failure message: </p> <p> <pre>Message: System.Exception : Actual status code: BadRequest. Expected NotFound, but got OK.</pre> </p> <p> Notice that the message contains information about both failure conditions. </p> <p> Finally, I re-enabled the correct behaviour (deleting the reservation from the data store) while still returning <code>400 Bad Request</code>: </p> <p> <pre>Message: System.Exception : Actual status code: BadRequest.</pre> </p> <p> As desired, the assertions collect all relevant failure messages. </p> <h3 id="4a2c496da999408ea6c6cddd6a4f33d4"> Conclusion <a href="#4a2c496da999408ea6c6cddd6a4f33d4" title="permalink">#</a> </h3> <p> Not surprisingly, it's possible to design a composable assertion API that collects multiple failure messages using an applicative functor. Anyone who knows how <a href="/2018/11/05/applicative-validation">applicative validation</a> works would have been able to predict that outcome. That's not what the above proof of concept was about. What I wanted to see was rather how it would play out in a realistic scenario, and whether using an applicative functor is warranted. </p> <p> Applicative functors don't gel well with C#, so unsurprisingly the API is awkward. It's likely possible to smooth much of the friction, but without good language support and syntactic sugar, it's unlikely to become <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C#. </p> <p> Rather than taking the edge off the unwieldy API, the implementation of <code>RunAssertions</code> suggests another alternative. </p> <p> <strong>Next:</strong> <a href="/2022/12/19/error-accumulating-composable-assertions-in-c">Error-accumulating composable assertions in C#</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mark Seemann https://blog.ploeh.dk/2022/11/28/an-initial-proof-of-concept-of-applicative-assertions-in-c