ploeh blog 2021-04-16T06:30:08+00:00 Mark Seemann danish software design https://blog.ploeh.dk Threading context through a catamorphism https://blog.ploeh.dk/2021/04/12/threading-context-through-a-catamorphism 2021-04-12T11:09:00+00:00 Mark Seemann <div id="post"> <p> <em>A problem solved after 1½ years.</em> </p> <p> You've probably noticed that it's easier to learn something new if it looks or sounds like something you already know. As a native Dane, I've found it easier to learn English and German than Russian and Japanese. If you originally were a Java or C# developer, you probably find <a href="https://en.wikipedia.org/wiki/JavaScript">JavaScript</a> more approachable than <a href="https://clojure.org">Clojure</a> or <a href="https://en.wikipedia.org/wiki/APL_(programming_language)">APL</a>. </p> <p> I believe that this extends to <a href="/2017/10/04/from-design-patterns-to-category-theory">design patterns and universal abstractions</a> as well. If code new to you follows well-known abstractions, it may be easier to learn than if it's structured in an entirely ad-hoc manner. This is my motivation for learning such universal abstractions as <a href="/2017/10/06/monoids">monoids</a>, <a href="/2018/03/22/functors">functors</a>, and <a href="/2019/04/29/catamorphisms">catamorphisms</a>. </p> <p> I particularly enjoy when it's possible to apply such abstractions to a proper problem. This occasionally happens. One example is my small article series on a <a href="/2019/08/26/functional-file-system">functional file system</a>. </p> <h3 id="2aa7c4b2f26840e7bfed2a42064853a7"> A fly in the ointment <a href="#2aa7c4b2f26840e7bfed2a42064853a7" title="permalink">#</a> </h3> <p> In those articles, I described how you could base most of the code on the <a href="/2019/08/05/rose-tree-catamorphism">rose tree catamorphism</a>. There was just one snag. There was one function, <code>calculateMoves</code>, that I was unable to implement with the catamorphism. In the article, I acknowledged my failure: <blockquote> "Earlier, I wrote that you can implement desired <code>Tree</code> functionality with the <code>foldTree</code> function, but that was a simplification. If you can implement the functionality of <code>calculateMoves</code> with <code>foldTree</code>, I don't know how." </blockquote> This was true for both <a href="/2019/09/09/picture-archivist-in-haskell">the Haskell proof of concept</a> as well as <a href="/2019/09/16/picture-archivist-in-f">the F# port</a>. </p> <p> <a href="https://about.me/tysonwilliams">Tyson Williams</a> and I <a href="/2019/09/16/picture-archivist-in-f#68b26807cc424856b8f762f214389826">discussed this wart</a> without getting closer to a solution. </p> <p> As the idiom goes, perfect is the enemy of good, so I decided to move on, although it nagged me. </p> <h3 id="641a9df4d9a24cb7bf687b041f8c305e"> Problem, condensed <a href="#641a9df4d9a24cb7bf687b041f8c305e" title="permalink">#</a> </h3> <p> The problem with the <code>calculateMoves</code> function was that it needed to thread a 'context' recursively through the entire data structure. In this case, the context was a file path. </p> <p> When <code>calculateMoves</code> runs over the input tree, it needs to thread a relative <code>path</code> through the function, building it up as it traverses the data structure. </p> <p> For example, if a leaf node named <code>1</code> is in a directory named <code>b</code>, which itself is a subdirectory of <code>a</code>, the relative path should be <code>a/b/1</code>. This example is straight from the test cases shown in both articles. You can also find the tests in the <a href="https://github.com/ploeh/picture-archivist">GitHub repository</a>. </p> <p> Each time <code>calculateMoves</code> visits a <code>Node</code> or <code>Leaf</code> it needs to know the parent <code>path</code> to calculate the destination path. As the articles show, this isn't too hard to do with regular pattern matching and recursion. </p> <p> I couldn't figure out, however, how to thread the <code>path</code> through the function when I tried to implement it with the catamorphism. </p> <h3 id="2461821e89084c5f8da43b2199d77d33"> Breakthrough <a href="#2461821e89084c5f8da43b2199d77d33" title="permalink">#</a> </h3> <p> While I'm ready to walk away from problems when I'm stuck, I tend to remember them. Sometimes, I run into a solution much later. </p> <p> This happened to me yesterday. I was trying to answer <a href="https://stackoverflow.com/q/67037663/126014">a Stack Overflow question</a> which was explicitly about the application of universal abstractions. Once more, I was stuck by being unable to thread a 'context' through a catamorphism. This time, instead of a <code>path</code>, the context was an indentation depth. Basically, the question was how to render a tree with proper indentation. </p> <p> Again, this isn't hard if you resort to explicit pattern matching and recursion, but I couldn't figure out how to do it via the data structure's catamorphism. </p> <p> Fortunately, <a href="https://stackoverflow.com/users/1364288/danidiaz">the user danidiaz</a> posted <a href="https://stackoverflow.com/a/67042881/126014">an awesome answer</a> while I was struggling. The answer uses a trick that I hadn't noticed before: It threads the indentation depth through the data structure by using the catamorphism to produce a structure map with <em>a function</em> as the carrier type. Specifically, danidiaz defines the algebra <code>Todo' (Int -&gt; String) -&gt; Int -&gt; String</code> to reduce a <code>Todo' (Int -&gt; String)</code> to an <code>Int -&gt; String</code> function. This function then gets initialised with the depth <code>0</code>. </p> <p> While I've been doing functional programming for years, I sometimes still tend to forget that functions are first-class values... </p> <p> This trick, though, seems to be universally applicable. If you need to thread a context through a catamorphism, define the algebra to work on <em>functions</em> that take the context as an argument. </p> <p> If this is a universally applicable trick, it also ought to work with the <code>calculateMoves</code> function. </p> <h3 id="f3e4908437ea4ab7bf0dd01d82ec470e"> Haskell re-implementation <a href="#f3e4908437ea4ab7bf0dd01d82ec470e" title="permalink">#</a> </h3> <p> In my <a href="https://www.haskell.org">Haskell</a> proof of concept, the <code>calculateMoves</code> function originally looked like this: </p> <p> <pre><span style="color:#2b91af;">calculateMoves</span>&nbsp;::&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span> calculateMoves&nbsp;=&nbsp;imp&nbsp;<span style="color:#a31515;">&quot;&quot;</span> &nbsp;&nbsp;<span style="color:blue;">where</span>&nbsp;imp&nbsp;path&nbsp;&nbsp;&nbsp;&nbsp;(Leaf&nbsp;x)&nbsp;=&nbsp;Leaf&nbsp;$&nbsp;Move&nbsp;x&nbsp;$&nbsp;replaceDirectory&nbsp;x&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;imp&nbsp;path&nbsp;(Node&nbsp;x&nbsp;xs)&nbsp;=&nbsp;Node&nbsp;(path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;$&nbsp;imp&nbsp;(path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;&lt;$&gt;&nbsp;xs</pre> </p> <p> It uses an <code>imp</code> (for <em>implementation</em>) function to explicitly <a href="/2015/12/01/recurse">recurse</a> over a <code>Tree FilePath FilePath</code>. Until yesterday, I couldn't come up with a better solution to thread the <code>path</code> through the data structure. </p> <p> The new trick suggests that it'd be possible to implement the function on <code>foldTree</code> (the catamorphism) by using <em>a function</em> as the carrier type. Since the context to be threaded through the catamorphism is a <code>String</code> (the <code>path</code>), the catamorphism should produce a function that takes a <code>String</code> as argument. In other words, the carrier type of the <code>Tree</code> should be <code>String -&gt; Tree FilePath Move</code>. </p> <p> Let's expand on this: The type of <code>foldTree</code> is <code>foldTree :: (a -&gt; [c] -&gt; c) -&gt; (b -&gt; c) -&gt; Tree a b -&gt; c</code>. Usually, I tend to think of the type parameter <code>c</code> as the type of some value, but since it's unconstrained, it can also be <em>a function</em>. That's what we need here: <code>c</code> should be <code>String -&gt; Tree FilePath Move</code>. </p> <p> That's not too hard to do, because of currying. Just write functions that take an extra <code>String</code> argument and pass them to <code>foldTree</code>: </p> <p> <pre><span style="color:#2b91af;">calculateMoves</span>&nbsp;::&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span> calculateMoves&nbsp;t&nbsp;=&nbsp;foldTree&nbsp;fNode&nbsp;fLeaf&nbsp;t&nbsp;<span style="color:#a31515;">&quot;&quot;</span> &nbsp;&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">fLeaf</span>&nbsp;::&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#2b91af;">String</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span> &nbsp;&nbsp;&nbsp;&nbsp;fLeaf&nbsp;x&nbsp;&nbsp;&nbsp;&nbsp;path&nbsp;=&nbsp;Leaf&nbsp;$&nbsp;Move&nbsp;x&nbsp;$&nbsp;replaceDirectory&nbsp;x&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">fNode</span>&nbsp;::&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;[<span style="color:#2b91af;">String</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span>]&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#2b91af;">String</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span> &nbsp;&nbsp;&nbsp;&nbsp;fNode&nbsp;x&nbsp;fs&nbsp;path&nbsp;=&nbsp;Node&nbsp;(path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;$&nbsp;($&nbsp;path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;&lt;$&gt;&nbsp;fs</pre> </p> <p> Here I've used type annotations for the local functions, but that's entirely optional: </p> <p> <pre><span style="color:#2b91af;">calculateMoves</span>&nbsp;::&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Tree</span>&nbsp;<span style="color:#2b91af;">FilePath</span>&nbsp;<span style="color:blue;">Move</span> calculateMoves&nbsp;t&nbsp;=&nbsp;foldTree&nbsp;fNode&nbsp;fLeaf&nbsp;t&nbsp;<span style="color:#a31515;">&quot;&quot;</span> &nbsp;&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;&nbsp;&nbsp;fLeaf&nbsp;x&nbsp;&nbsp;&nbsp;&nbsp;path&nbsp;=&nbsp;Leaf&nbsp;$&nbsp;Move&nbsp;x&nbsp;$&nbsp;replaceDirectory&nbsp;x&nbsp;path &nbsp;&nbsp;&nbsp;&nbsp;fNode&nbsp;x&nbsp;fs&nbsp;path&nbsp;=&nbsp;Node&nbsp;(path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;$&nbsp;($&nbsp;path&nbsp;&lt;/&gt;&nbsp;x)&nbsp;&lt;$&gt;&nbsp;fs</pre> </p> <p> I included the type annotations to make it a little clearer what's going on. Recall that the type of <code>foldTree</code> is <code>foldTree :: (a -&gt; [c] -&gt; c) -&gt; (b -&gt; c) -&gt; Tree a b -&gt; c</code>. First consider the second of the two function arguments, the one I call <code>fLeaf</code> in the above code. It's the simplest of the two, so it makes sense to start with that one. </p> <p> The generic type of <code>fLeaf</code> is <code>b -&gt; c</code>. How does that map to the type of <code>fLeaf</code>, which is <code>FilePath -&gt; String -&gt; Tree FilePath Move</code>? </p> <p> Well, the <code>Tree</code> that the catamorphism runs on is a <code>Tree FilePath FilePath</code>. Mapped to the parametrically polymorphic type of <code>foldTree</code> that's <code>Tree a b</code>. In other words, <code>b</code> maps to <code>FilePath</code>. Thus, in order to fit the type of <code>b -&gt; c</code>, the type corresponding to <code>b</code> in <code>fLeaf</code> must be <code>FilePath</code>. What's left? <code>String -&gt; Tree FilePath Move</code> is what's left. The function takes a <code>FilePath</code> as input and returns a <code>String -&gt; Tree FilePath Move</code>. In other words, <code>c ~ String -&gt; Tree FilePath Move</code>. </p> <p> How does that fit with <code>fNode</code>? </p> <p> Generically, this function must have the type <code>a -&gt; [c] -&gt; c</code>. We've already established that <code>c</code> must be <code>String -&gt; Tree FilePath Move</code>. Since the catamorphism runs on a <code>Tree FilePath FilePath</code>, we also know that <code>a</code> must be <code>FilePath</code>. Thus, plugging in all the types, <code>fNode</code> must have the type <code>FilePath -&gt; [String -&gt; Tree FilePath Move] -&gt; String -&gt; Tree FilePath Move</code>. Note, particularly, that the second argument is a list of functions. That's why I decided to name the parameter <code>fs</code>, for <em>functions</em>. </p> <p> The entire expression <code>foldTree fNode fLeaf t</code>, then, has the type <code>String -&gt; Tree FilePath Move</code>, since <code>c</code> is <code>String -&gt; Tree FilePath Move</code> and the return type of <code>foldTree</code> is <code>c</code>. </p> <p> The final trick is to apply this function to the initial relative path <code>""</code>, which returns a <code>Tree FilePath Move</code>. </p> <p> This compiles and passes all tests. <code>calculateMoves</code> is now implemented using the <code>Tree</code> catamorphism, a goal that eluded me for more than one and a half years. </p> <h3 id="48fe66e3bffa413c8fe15e465581cb4e"> F# re-implementation <a href="#48fe66e3bffa413c8fe15e465581cb4e" title="permalink">#</a> </h3> <p> With the Haskell proof of concept in place, it's fairly trivial to port the new implementation to the <a href="https://fsharp.org">F#</a> code base. </p> <p> The <code>calculateMoves</code> function originally looked like this: </p> <p> <pre><span style="color:green;">//&nbsp;Tree&lt;string,FileInfo&gt;&nbsp;-&gt;&nbsp;Tree&lt;string,Move&gt;</span> <span style="color:blue;">let</span>&nbsp;calculateMoves&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;replaceDirectory&nbsp;(f&nbsp;:&nbsp;FileInfo)&nbsp;d&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FileInfo&nbsp;(Path.Combine&nbsp;(d,&nbsp;f.Name)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">rec</span>&nbsp;imp&nbsp;path&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Leaf&nbsp;x&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Leaf&nbsp;{&nbsp;Source&nbsp;=&nbsp;x;&nbsp;Destination&nbsp;=&nbsp;replaceDirectory&nbsp;x&nbsp;path&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Node&nbsp;(x,&nbsp;xs)&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;newNPath&nbsp;=&nbsp;Path.Combine&nbsp;(path,&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tree.node&nbsp;newNPath&nbsp;(List.map&nbsp;(imp&nbsp;newNPath)&nbsp;xs) &nbsp;&nbsp;&nbsp;&nbsp;imp&nbsp;<span style="color:#a31515;">&quot;&quot;</span></pre> </p> <p> In the F# code base, the catamorphism is called <code>Tree.cata</code>, but otherwise looks like the Haskell <code>foldTree</code> function. The refactoring is also similar: </p> <p> <pre><span style="color:green;">//&nbsp;Tree&lt;string,&nbsp;FileInfo&gt;&nbsp;-&gt;&nbsp;Tree&lt;string,&nbsp;Move&gt;</span> <span style="color:blue;">let</span>&nbsp;calculateMoves&nbsp;t&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;FileInfo&nbsp;-&gt;&nbsp;string&nbsp;-&gt;&nbsp;FileInfo</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;replaceDirectory&nbsp;(f&nbsp;:&nbsp;FileInfo)&nbsp;d&nbsp;=&nbsp;FileInfo&nbsp;(Path.Combine&nbsp;(d,&nbsp;f.Name)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;FileInfo&nbsp;-&gt;&nbsp;string&nbsp;-&gt;&nbsp;Tree&lt;&#39;a,&nbsp;Move&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fLeaf&nbsp;x&nbsp;path&nbsp;=&nbsp;Leaf&nbsp;{&nbsp;Source&nbsp;=&nbsp;x;&nbsp;Destination&nbsp;=&nbsp;replaceDirectory&nbsp;x&nbsp;path&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;(string&nbsp;-&gt;&nbsp;Tree&lt;string,&nbsp;&#39;a&gt;)&nbsp;list&nbsp;-&gt;&nbsp;string&nbsp;-&gt;&nbsp;Tree&lt;string,&nbsp;&#39;a&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;fNode&nbsp;x&nbsp;fs&nbsp;path&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;newNPath&nbsp;=&nbsp;Path.Combine&nbsp;(path,&nbsp;x) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tree.node&nbsp;newNPath&nbsp;(List.map&nbsp;(<span style="color:blue;">fun</span>&nbsp;f&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;f&nbsp;newNPath)&nbsp;fs) &nbsp;&nbsp;&nbsp;&nbsp;Tree.cata&nbsp;fNode&nbsp;fLeaf&nbsp;t&nbsp;<span style="color:#a31515;">&quot;&quot;</span></pre> </p> <p> Again, the expression <code>Tree.cata fNode fLeaf t</code> has the type <code>string -&gt; Tree&lt;string, Move&gt;</code>, so applying it to <code>""</code> produces a <code>Tree&lt;string, Move&gt;</code> return value. </p> <h3 id="21524d98dcbb4593abc18c55efbfd105"> Conclusion <a href="#21524d98dcbb4593abc18c55efbfd105" title="permalink">#</a> </h3> <p> I don't recall where I read the following, but I was under the impression that a data structure's catamorphism was its 'universal API', upon which you could implement any other functionality. I'd love it if it was true, but after my 2019 failure to implement <code>calculateMoves</code> via the <code>Tree</code> catamorphism, I wasn't sure if such a conjecture would hold. </p> <p> I still don't know if that assertion holds universally, but at least one reason to doubt it has now been removed. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="1301010b460845db8730e4aa617504a4"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Excellent work Mark! I too had not forgotten about this, and it nagged me as well. </p> <p> To some extent, I feel like your explanation of how to implement <code>calculateMoves</code> via <code>Tree.cata</code> is top-down. By top-down, I mean it might depend on discovering the key idea of having <code>Tree.cata</code> return a function and then figuring out the correct type for that function. A good thing about such top-down approaches is being immediately aware that a better solution likely exists even if it takes some time and effort to find it. </p> <p> I was curious if a bottom-up approach would work. By bottom-up, I mean applying small refacorings to the code that are motivated by the principles, conventions, or style of functional programming. I do think I found such an approach. Of course it is a bit contradictory of me to only be able to find this approach after I read your presentation of the top-down approach. However, I am thinking of it like a kata. I now know such a bottom-up approach should be possible, and I want to find it. </p> <p> My bottom-up approach is in <a href="https://github.com/TysonMN/picture-archivist/tree/calculateMoves_via_cata_slow_refactor">this branch</a>. Here is a brief summary of how I want myself to think of making those commits in that order. </p> <p> Each case of the discriminated union could be extracted to its own function. This is easy to do in the <code>Leaf</code> case (so do it now), but it is not as easy to do in the <code>Node</code> case because of recursion, so delay that change for a bit. If we did extract both functions though, both functions would include the argument that I called <code>pathToParent</code>. Since it is passed in everywhere, it should be passed in nowhere (by eta reducing). To do that, we need it to be the last parameter to <code>imp</code>. After switching this order, we now deal with the recursion by doing it as soon as possible. Then the remaining code in that case can be extracted, and <code>imp</code> is essentially <code>Tree.cata</code>. </p> <p> In this approach, I never thought about the possibility of <code>Tree.cata</code> returning a function. It just sort of fell out as a consequence of my other changes. </p> </div> <div class="comment-date">2021-04-12 17:49 UTC</div> </div> <div class="comment" id="c90c8190c0f44e60a44fc605b9a84113"> <div class="comment-author"><a href="https://github.com/gonzaw">Gonzalo Waszczuk</a></div> <div class="comment-content"> <p> Very nice! </p> <p> In Haskell there is a library called <a href="https://hackage.haskell.org/package/recursion-schemes-5.2.2.1">recursion-schemes</a> that showcases these types of recursion with catamorphisms, but also with many others recursion schemes. You can check it out and see if it gives you any new ideas. </p> <p> Regarding this use of catamorphism, the library itself I believe shows a very similar example <a href="https://hackage.haskell.org/package/recursion-schemes-5.2.2.1/docs/Data-Functor-Foldable.html">here</a>, using the Reader type (which is isomorphic to the function you used in your example): </p> <pre> >>> :{ let pprint2 :: Tree Int -> String pprint2 = flip runReader 0 . cataA go where go :: TreeF Int (Reader Int String) -> Reader Int String go (NodeF i rss) = do -- rss :: [Reader Int String] -- ss :: [String] ss <- local (+ 2) $ sequence rss indent <- ask let s = replicate indent ' ' ++ "* " ++ show i pure $ intercalate "\n" (s : ss) :} </pre> <pre> >>> putStrLn $ pprint2 myTree * 0 * 1 * 2 * 3 * 31 * 311 * 3111 * 3112 </pre> </div> <div class="comment-date">2021-04-14 02:27 UTC</div> </div> <div class="comment" id="d26feeefed69421499e766dc1737ca31"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Gonzalo, thank you for reminding me of the <em>recursion-schemes</em> library. It's one of those tomes of knowledge of which I'm aware, but never really have gotten around to look at... </p> </div> <div class="comment-date">2021-04-16 6:29 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Mazes on Voronoi tessellations https://blog.ploeh.dk/2021/04/05/mazes-on-voronoi-tesselations 2021-04-05T09:03:00+00:00 Mark Seemann <div id="post"> <p> <em>Recursive backtracker maze generation on a Voronoi diagram.</em> </p> <p> Today's blog post <a href="https://observablehq.com/@ploeh/mazes-on-voronoi-tessellations">appears on Observable</a>. It's an interactive environment where you can play with and fork the code. <a href="https://observablehq.com/@ploeh/mazes-on-voronoi-tessellations">Go there to read it</a>. </p> <p> <img src="/content/binary/recursive-backtracker-on-voronoi-tiles.png" alt="Recursive backtracker algorithm running on a Voronoi tessellation."> </p> <p> <a href="https://observablehq.com">Observable</a> is a really neat platform which has managed to do what I thought was nigh-impossible: make me return to <a href="https://en.wikipedia.org/wiki/JavaScript">JavaScript</a>. The site's <a href="https://observablehq.com/about">been around for some years</a>, and I hope it'll be around for even more years. </p> <p> <em>ploeh blog</em>, on the other hand, has been around <a href="/2009/01/28/LivingInInterestingTimes">since 2009</a>, and I intend to keep it around for much longer. Who knows if <em>Observable</em> will outlive the blog. Enjoy the article while it's there. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Table-driven tennis scoring https://blog.ploeh.dk/2021/03/29/table-driven-tennis-scoring 2021-03-29T06:15:00+00:00 Mark Seemann <div id="post"> <p> <em>Probably the most boring implementation of the tennis kata I've ever written.</em> </p> <p> Regular readers of this blog will know that I keep coming back to the <a href="https://codingdojo.org/kata/Tennis">tennis kata</a>. It's an interesting little problem to attack from various angles. </p> <p> The tennis scoring rules essentially describe a finite state machine, and while I was thinking about the state transitions involved, I came across an article by <a href="http://blog.mikemccandless.com">Michael McCandless</a> about <a href="http://blog.mikemccandless.com/2014/08/scoring-tennis-using-finite-state.html">scoring tennis using finite-state automata</a>. </p> <p> This isn't the first time I've thought about simply enumerating all possible states in the state machine, but I decided to spend half an hour on actually doing it. While Michael McCandless shows that an optimisation is possible, his minimised version doesn't enable us to report all intermediary states with the correct labels. For example, he optimises away <em>thirty-all</em> by replacing it with <em>deuce</em>. The end result is still the same, in the sense that the minimised state machine will arrive at the same winner for the same sequence of balls, but it can't correctly report the score while the game is in progress. </p> <p> For that reason, I decided to use his non-optimised state machine as a launch pad. </p> <h3 id="d32da6b59b7448959d6bed2f10f84569"> States <a href="#d32da6b59b7448959d6bed2f10f84569" title="permalink">#</a> </h3> <p> I used <a href="https://fsharp.org">F#</a> to enumerate all twenty states: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Score&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveAll &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveFifteen &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenAll &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveThirty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyFifteen &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenThirty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveForty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyFifteen &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyAll &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenForty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyThirty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyForty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;GamePlayerTwo &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;AdvantagePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Deuce &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;AdvantagePlayerTwo</pre> </p> <p> Utterly boring, yes, but perhaps boring code might be good code. </p> <h3 id="1eb656d32be0461aa11c321ec478a1cf"> Table-driven methods <a href="#1eb656d32be0461aa11c321ec478a1cf" title="permalink">#</a> </h3> <p> <a href="http://amzn.to/1dLYr0r">Code Complete</a> describes a programming technique called <em>table-driven methods</em>. The idea is to replace branching instructions such as <code>if</code>, <code>else</code>, and <code>switch</code> with a lookup table. The book assumes that the table exists in memory, but in this case, we can implement the table lookup with pattern matching: </p> <p> <pre><span style="color:green;">//&nbsp;Score&nbsp;-&gt;&nbsp;Score</span> <span style="color:blue;">let</span>&nbsp;ballOne&nbsp;=&nbsp;<span style="color:blue;">function</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveAll&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FifteenLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenLove&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;ThirtyLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveFifteen&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FifteenAll &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyLove&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FortyLove &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenAll&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;ThirtyFifteen &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveThirty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FifteenThirty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyLove&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyFifteen&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FortyFifteen &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenThirty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;ThirtyAll &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;LoveForty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FifteenForty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyFifteen&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyAll&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;FortyThirty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FifteenForty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;ThirtyForty &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;GamePlayerOne&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;FortyThirty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;ThirtyForty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Deuce &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;GamePlayerTwo&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerTwo &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;AdvantagePlayerOne&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;GamePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Deuce&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;AdvantagePlayerOne &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;AdvantagePlayerTwo&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Deuce</pre> </p> <p> The <code>ballOne</code> function returns the new score when player one wins a ball. It takes the old score as input. </p> <p> I'm going to leave <code>ballTwo</code> as an exercise to the reader. </p> <h3 id="47bf74d3fafd478fafa354563459f0ad"> Smoke test <a href="#47bf74d3fafd478fafa354563459f0ad" title="permalink">#</a> </h3> <p> Does it work, then? Here's a few interactions with the API in <em>F# Interactive:</em> </p> <p> <pre>&gt; ballOne LoveAll;; val it : Score = FifteenLove &gt; LoveAll |&gt; ballOne |&gt; ballTwo;; val it : Score = FifteenAll &gt; LoveAll |&gt; ballOne |&gt; ballTwo |&gt; ballTwo;; val it : Score = FifteenThirty &gt; LoveAll |&gt; ballOne |&gt; ballTwo |&gt; ballTwo |&gt; ballTwo;; val it : Score = FifteenForty &gt; LoveAll |&gt; ballOne |&gt; ballTwo |&gt; ballTwo |&gt; ballTwo |&gt; ballOne;; val it : Score = ThirtyForty &gt; LoveAll |&gt; ballOne |&gt; ballTwo |&gt; ballTwo |&gt; ballTwo |&gt; ballOne |&gt; ballTwo;; val it : Score = GamePlayerTwo</pre> </p> <p> It looks like it's working. </p> <h3 id="17290385a4a042189ef0b5b04cf200c6"> Automated tests <a href="#17290385a4a042189ef0b5b04cf200c6" title="permalink">#</a> </h3> <p> Should I be writing unit tests for this implementation? </p> <p> I don't see how a test would be anything but a duplication of the two 'transition tables'. <em>Given that the score is thirty-love, when player one wins the ball, then the new score should be forty-love</em>. Indeed, the <code>ballOne</code> function already states that. </p> <p> We <a href="/2013/04/02/why-trust-tests">trust tests because they are simple</a>. When the implementation is as simple as the test that would exercise it, then what's the benefit of the test? </p> <p> To be clear, there are still compelling reasons to write tests for some simple implementations, but that's another discussion. I don't think those reasons apply here. I'll write no tests. </p> <h3 id="fb03ecc39926472790ba2c5eaa025a91"> Code size <a href="#fb03ecc39926472790ba2c5eaa025a91" title="permalink">#</a> </h3> <p> While this code is utterly dull, it takes up some space. In all, it runs to 67 lines of code. </p> <p> For comparison, the code base that evolves throughout my <a href="/2016/02/10/types-properties-software">Types + Properties = Software</a> article series is 65 lines of code, not counting the tests. When I also count the tests, that entire code base contains around 300 lines of code. That's more than four times as much code. </p> <p> Preliminary <a href="https://amzn.to/2OBNBoY">research implies that bug count correlates linearly with line count</a>. The more lines of code, the more bugs. </p> <p> While I believe that this is probably a simplistic rule of thumb, there's much to like about smaller code bases. In total, this utterly dull implementation is actually smaller than a comparable implementation built from small functions. </p> <h3 id="5ac7b217f375464aa21320c8f0bfe0a7"> Conclusion <a href="#5ac7b217f375464aa21320c8f0bfe0a7" title="permalink">#</a> </h3> <p> Many software problems can be modelled as finite state machines. I find that this is often the case in my own field of line-of-business software and web services. </p> <p> It's not always possible to exhaustively enumerate all states, because each 'type' of state carries data that can't practically be enumerated. For example, <a href="/2017/06/27/pure-times">polling consumers</a> need to carry timing statistics. These statistics influence how the state machine transitions, but the range of possible values is so vast that it can't be enumerated as types. </p> <p> It may not happen often that you can fully enumerate all states and transitions of a finite state machine, but I think it's worthwhile to be aware of such refactoring opportunities. It might make your code dully simple. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="57354557ce724eefb675d8788c5bea94"> <div class="comment-author"><a href="https://taeguk.co.uk">Dave Shaw</a></div> <div class="comment-content"> <p>Hi Mark, I have had a similar experience whilst coding a <a href="https://en.wikipedia.org/wiki/Shut_the_box"> Shut the box</a> game, when trying to detect if it was game over or not. <br/> Originally it was a complex set of loops to calculate all the discrete summands for each roll of the dice, then checking if the remaining flaps were in that set. This was done along with a suite of tests for every possible combination set of summands up to 12 (for 2 dice). <br /> Then whilst explaining the pain in writing this to a friend, they simply said, <i>there's only a finite list, why not hard code them?</i>, and that's what I went with, a dictionary with each possible roll from 2 dice, and the possible values from the flaps that could be used to meet that roll. All the tests were removed; as you pointed out, they would just be a reimplmentation of the table. </p> </div> <div class="comment-date">2021-04-07 13:30 UTC</div> </div> <div class="comment" id="858ccadaae1b41268d3bf4454ab9fcb0"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Dave, thank you for writing. It's good to hear that you have a similar experience. I wonder if it's constrained to game simulation, or if 'real-world' examples exist. </p> </div> <div class="comment-date">2021-04-09 6:30 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. The dispassionate developer https://blog.ploeh.dk/2021/03/22/the-dispassionate-developer 2021-03-22T06:50:00+00:00 Mark Seemann <div id="post"> <p> <em>Caring for your craft is fine, but should you work for free?</em> </p> <p> I've met many passionate developers in my career. Programmers who are deeply interested in technology, programming languages, methodology, and self-improvement. I've also seen many online profiles where people present themselves as 'passionate developers'. </p> <p> These are the people who organise and speak at user groups. They write blog posts and host podcasts. They contribute to open source development in their free time. </p> <p> I suppose that I can check many of those boxes myself. In the last few years, though, I've become increasingly sceptic that this is a good idea. </p> <h3 id="8894ee52bd514210a5f8193f38634f43"> Working for free <a href="#8894ee52bd514210a5f8193f38634f43" title="permalink">#</a> </h3> <p> In the last five years or so, I've noticed what looks like a new trend. Programmers contact me to ask about paid mentorship. They offer to pay me <em>out of their own pocket</em> to mentor them. </p> <p> I find that flattering, but it also makes me increasingly disenchanted with the software development industry. To be clear, this isn't an attack on the good people who care so much about their craft that they are willing to spend their hard-earned cash on improving their skill. This is more a reflection on employers. </p> <p> For reasons that are complicated and that I don't fully understand, the software development community in the eighties and nineties developed a culture of anti-capitalism and liberal values that put technology on a pedestal for its own sake. Open source good; commercial software bad. Free software good; commercial software bad. </p> <p> I'm not entirely unsympathetic to such ideas, but it's seems clear, now, that these ideas have had unintended consequences. The idea of free software, for example, has led to a software economy where you, the user, are no longer the customer, but the product. </p> <p> The idea of open source, too, seems largely defunct as a means of 'sticking it to the man'. The big tech companies now embrace open source. Despite initial enmity towards open source, <a href="https://en.wikipedia.org/wiki/Microsoft_and_open_source">Microsoft now owns GitHub and is one of the most active contributors</a>. Google and Facebook control popular front-end platforms such as <a href="https://en.wikipedia.org/wiki/Angular_(web_framework)">Angular</a> and <a href="https://en.wikipedia.org/wiki/React_(JavaScript_library)">React</a>, as well as many other technologies such as <a href="https://en.wikipedia.org/wiki/Android_(operating_system)">Android</a> or <a href="https://en.wikipedia.org/wiki/GraphQL">GraphQL</a>. Continue the list at your own leisure. </p> <p> Developing open source is seen as a way to establish credibility, not only for companies, but for individuals as well. Would you like a cool job in tech? Show me your open-source portfolio. </p> <p> Granted, the focus on open-source contributions as a replacement for a CV seems to have peaked, and good riddance. </p> <p> I deliberately chose to use the word <em>portfolio</em>, above. Like a struggling artist, you're expected to show up with such a stunning sample of your work that you amaze your potential new employer and blow away your competition. Unlike struggling artists, though, you've already given away everything in your portfolio, and so have other job applicants. Employers benefit from this. You work for free. </p> <h3 id="2d37b0297422447c9c92e2f18828f5a7"> The passion ethos <a href="#2d37b0297422447c9c92e2f18828f5a7" title="permalink">#</a> </h3> <p> You're expected to 'contribute' to open source software. Why? Because employers want employees who are <em>passionate</em> about their craft. </p> <p> As you start to ponder the implied ethos, the stranger it gets. Would you like engineers to be <em>passionate</em> as they design new bridges? Would you like a surgeon to be <em>passionate</em> as she operates on you? Would you like judges to be <em>passionate</em> as they pass sentence on your friend? </p> <p> I'd like such people to <em>care</em> about their vocation, but I'd prefer that they keep a cool head and make as rational decisions as possible. </p> <p> Why should programmers be <em>passionate?</em> </p> <p> I don't think that it's in our interest to be passionate, but it <em>is</em> in employers' interest. Not only are passionate people expected to work for free, they're also easier to manipulate. Tell a passionate person something he wants to hear, and he may turn off further critical thinking because the praise <em>feels</em> good. </p> <p> Some open-source maintainers have created crucial software that runs everywhere. Companies make millions off that free software, while maintainers are often left with an increasing support burden and no money. </p> <p> They do, however, often get a pat on the back. They get invited to speak at conferences, and can add <em>creator of Xyz</em> to their social media bios. </p> <p> Until they burn out, that is. <ins datetime="2021-03-22T11:49Z"><em>Passion</em>, after all, comes from the Latin for <em>suffering</em></ins>. </p> <h3 id="192b9f90f1e94fcf97649edc5c234f39"> Self-improvement <a href="#192b9f90f1e94fcf97649edc5c234f39" title="permalink">#</a> </h3> <p> I remember consulting with a development organisation, helping them adopt some new technology. As my engagement was winding down, I had a meeting with the manager to discuss how they should be able to carry on without me. This was back in my Microsoft days, so I suggested that they institute a training programme for the employees. To give it structure, they could, for example, study for some Microsoft certifications. </p> <p> The development manager immediately shot down that idea: <em>"If we do that, they'll leave us once they have the certification."</em> </p> <p> I was flabbergasted. </p> <p> You've probably seen quotes like this: <blockquote> <p> "What happens if we train our people and they leave?" </p> <p> "What happens if we don't and they stay?" </p> </blockquote> This is one of those bon mots that seem impossible to attribute to a particular source, but the idea is clear enough. The sentiment doesn't seem to represent mainstream behaviour, though. </p> <p> Granted, I've met more than one visionary leader willing to invest in employees' careers, but most managers don't. </p> <p> While I teach and coach internationally, I naturally have more experience with my home region of Copenhagen, and more broadly Scandinavia. Here, it's a common position that anything that relates to work should only happen during work hours. If the employer doesn't allow training on the job, then most employees don't train. </p> <p> What happens if you don't keep up to date with new methodologies, new frameworks, new programming languages? Your skill set becomes obsolete. Not overnight, but over the years. Finding a new job becomes harder and harder. </p> <p> As your marketability atrophies, your employer can treat you worse and worse. After all, where are you going to go? </p> <p> If you're tired of working with legacy code without tests, most of your suggestions for improvements will be met by a shrug. <em>We don't have time for that now. It's more important to deliver value to the customer.</em> </p> <p> You'll have to work long hours and weekends fire-fighting 'unexpected' issues in production while still meeting deadlines. </p> <p> A sufficiently cynical employer may have no qualms keeping employees busy this way. </p> <p> To be clear, I'm not saying that it's good business sense to treat skilled employees like this, and I'm not saying that this is how <em>all</em> employers conduct business, but I've seen enough development organisations that fit the description. </p> <p> As disappointing as it may be, keeping up to date with technology is <em>your</em> responsibility, and if you can't sneak in some time for self-improvement at work, you'll have to do it on your own time. </p> <p> This has little to do with passion, but much to do with self-preservation. </p> <h3 id="a40b07d82eef4ee0a657df53662b4418"> Can I help you? <a href="#a40b07d82eef4ee0a657df53662b4418" title="permalink">#</a> </h3> <p> The programmers who contact me (and others) for mentorship are the enlightened ones who've already figured this out. </p> <p> That doesn't mean that I'm comfortable taking people's hard-earned money. If I teach you something that improves your productivity, your employer benefits, too. I think that your employer should pay for that. </p> <p> I'm aware that most companies don't want to do that. It's also my experience that while most employers couldn't care less whether you pay me for mentorship, they don't want you to show me their code. This basically means that I can't really mentor you, unless you can reproduce the problems you're having as anonymised code examples. </p> <p> But if you can do that, you can ask the whole internet. You can try asking on <a href="https://stackoverflow.com">Stack Overflow</a> and then ping me. You're also welcome to ask me. If your <a href="https://en.wikipedia.org/wiki/Minimal_working_example">minimal working example</a> is interesting, I may turn it into a blog post, and you pay nothing. </p> <p> People also ask me how they can convince their managers or colleagues to do things differently. I often wonder why they don't <a href="/2019/03/18/the-programmer-as-decision-maker">make technical decisions</a> already, but this may be my cultural bias talking. In Denmark you can often get away with the ask-for-forgiveness-rather-than-permission attitude, but it may not be a good idea in your culture. </p> <p> Can I magically convince your manager to do things differently? Not magically, but I do know an effective trick: get him or her to hire me (or another expensive consultant). Most people don't heed advice given for free, but if they pay dearly for it, they tend to pay attention. </p> <p> Other than that, I can only help you as I've passionately tried to help the world-wide community for decades: by blogging, answering questions on Stack Overflow, writing books, <a href="/schedule">speaking at user groups and conferences</a>, <a href="/about#video">publishing videos</a>, and so on. </p> <h3 id="2eb915d9d2aa411884599d592a310be0"> Ticking most of the boxes <a href="#2eb915d9d2aa411884599d592a310be0" title="permalink">#</a> </h3> <p> Yes, I know that I fit the mould of the <em>passionate developer</em>. I've blogged regularly since 2006, I've <a href="https://stackoverflow.com/users/126014/mark-seemann?tab=answers">answered thousands of questions on Stack Overflow</a>, I've given more than a hundred presentations, been a podcast guest, and co-written <a href="/dippp">a book</a>, none of which has made me rich. If I don't do it for the passion, then why do I do it? </p> <p> Sometimes it's hard and tedious work, but even so, I do much of it because I can't really help it. I <em>like</em> to write and teach. I suppose that makes me passionate. </p> <p> My point with this article isn't that there's anything wrong with being passionate about software development. The point is that you might want to regard it as a <em>weakness</em> rather than an asset. If you <em>are</em> passionate, beware that someone doesn't take advantage of you. </p> <p> I realise that I didn't view the world like this when I started blogging in January 2006. I was driven by my passion. In retrospect, though, I think that I have been both privileged and fortunate. I'm not sure my career path is reproducible today. </p> <p> When I started blogging, it was a new-fangled thing. Just the fact that you blogged was enough to you get a little attention. I was in the right place at the right time. </p> <p> The same is true for Stack Overflow. The site was still fairly new when I started, and a lot of frequently asked questions were only asked on my watch. I still get upvotes on answers from 2009, because these are questions that people still ask. I was just lucky enough to be around <em>the first time</em> it was asked on the site. </p> <p> I'm also privileged by being an able-bodied man born into the middle class in one the world's richest countries. I received a free education. Denmark has free health care and generous social security. Taking some chances with your career in such an environment isn't even reckless. I've worked for more than one startup. That's not risky here. Twice, I've worked for a company that went out of business; in none of those cases did I lose money. </p> <p> Yes, I've been fortunate, but my point is that you should probably not use my career as a model for yours, just as you shouldn't use those of <a href="https://en.wikipedia.org/wiki/Robert_C._Martin">Robert C. Martin</a>, <a href="https://en.wikipedia.org/wiki/Kent_Beck">Kent Beck</a>, or <a href="https://en.wikipedia.org/wiki/Martin_Fowler_(software_engineer)">Martin Fowler</a>. It's hardly a reproducible career path. </p> <h3 id="47dbcc3e81fc408881d0a173f5226f53"> Conclusion <a href="#47dbcc3e81fc408881d0a173f5226f53" title="permalink">#</a> </h3> <p> What <em>can</em> you do, then, if you want to stand out from the crowd? How <em>do</em> you advance your software development career? </p> <p> I don't know. I never claimed that this was easy. </p> <p> Being good at something helps, but you must also make sure that the right people know what you're good at. You're probably still going to have to invest some of your 'free' time to make that happen. </p> <p> Just beware that you aren't being taken advantage of. Be dispassionate. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="c86dd3a1f4814201a7d15ba3b95c67c0"> <div class="comment-author"><a href="https://github.com/Jankowski-J">Jakub Jankowski</a></div> <div class="comment-content"> <p> Thanks Mark for your post. </p> <p> I really relate to your comment about portfolio. I am still a young developer, not even 30 years old. A few years ago, I had an unhealthy obsession, that I should have a portfolio, otherwise I would be having a hard time finding job. </p> <p> I am not entirely sure where this thought was coming from, but it is not important in what I want to convey. I was worrying that I do not have a portfolio and that anxiety itself, prevented me from doing any real work to have anything to showcase. Kinda vicious cycle. </p> <p> Anyways, even without a portfolio, I didn't have any troubles switching jobs. I focused on presenting what I have learned in every project I worked on. What was good about it, what were the struggles. I presented myself not as a just a "mercenary" if you will. I always gave my best at jobs and at the interviews and somehow managed to prove to myself that a portfolio is not a <em>must</em>. </p> <p> Granted, everybody's experience is different and we all work in different market conditions. But my takeaway is - don't fixate on a thing, if it's not an issue. That's kinda what I was doing a few years back. </p> </div> <div class="comment-date">2021-03-28 16:25 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Pendulum swing: pure by default https://blog.ploeh.dk/2021/03/15/pendulum-swing-pure-by-default 2021-03-15T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Favour pure functions over polymorphic dependencies.</em> </p> <p> This is an article in <a href="/2021/02/22/pendulum-swings">a small series of articles about personal pendulum swings</a>. Here, I'll discuss another contemporary one-eighty. This one is older than the other two I've discussed in this article series, but I believe that it deserves to be included. </p> <p> Once upon I time, I used to consider Dependency Injection (DI) and injected interfaces an unequivocal good: the more, the merrier. These days, I tend to only model true application dependencies as injected dependencies. For the rest, I use <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>. </p> <h3 id="52438454eb1747cb81e897e1a44dfa1b"> Background <a href="#52438454eb1747cb81e897e1a44dfa1b" title="permalink">#</a> </h3> <p> When I started my programming career, I'd barely taught myself to program. I worked in both Visual Basic, VBScript, and C++ before I encountered the concept of an interface. What C++ I wrote was entirely procedural, and I don't recall being aware of inheritance. Visual Basic 6 didn't have inheritance, and I'm fairly sure that VBScript didn't, either. </p> <p> I vaguely recall first being introduced to the concept of an interface in Visual Basic. It took me some time to wrap my head around it, and while I thought it seemed clever, I couldn't find any practical use for it. </p> <p> I think that I wrote my first professional C# code base in 2002. We didn't use Dependency Injection or interfaces. I don't even recall that we used much inheritance. </p> <h3 id="cae8dc76412149dbbe4eb4854888f652"> Inject all the things <a href="#cae8dc76412149dbbe4eb4854888f652" title="permalink">#</a> </h3> <p> When I discovered test-driven development (TDD) the year after, it didn't take me too long to figure out that I'd need to isolate units from their dependencies. Based on initial successes, I even wrote <a href="https://docs.microsoft.com/archive/msdn-magazine/2004/october/unit-testing-mock-objects-to-the-rescue-test-your-net-code-with-nmock">an article about mock objects</a> for MSDN Magazine October 2004. </p> <p> At that time I'd made interfaces a part of my active technique. I still struggled with how to replace a unit's 'real' dependencies with the mock objects. Initially, I used what I in <a href="https://amzn.to/36xLycs">Dependency Injection in .NET</a> later called <em>Bastard Injection</em>. As I also described in the book, things took a dark turn for while as I discovered the <a href="/2010/02/03/ServiceLocatorisanAnti-Pattern">Service Locator anti-pattern</a> - only, at that time, I didn't realise that it was an anti-pattern. Soon after, fortunately, I discovered <a href="/2014/06/10/pure-di">Pure DI</a>. </p> <p> That problem solved, I began an era of my programming career where everything became an interface. It does enable unit testing, so it's better than not being able to test, but after some years I began to sense the limits. </p> <p> Perhaps the worst problem is that you get a deluge of interfaces. Many of these interfaces have similar-sounding names like <code>IReservationsManager</code> and <code>IRestaurantManager</code>. This makes discoverability harder: Which of these interfaces should you use? One defines <a href="/2020/11/23/good-names-are-skin-deep">a <code>TrySave</code> method, the other a <code>Check</code> method</a>, and they aren't that different. </p> <p> This wasn't clear to me when I worked in teams with one or two programmers. Once I saw how this played out in larger teams, however, I began to understand that one developer's interface remained undiscovered by other team members. When existing 'abstractions' are unclear, it leads to frequent reinvention of interfaces to implement the same functionality. Duplication abounds. </p> <p> Designing with many fine-grained dependencies also has a tendency drag into existence many <em>factory interfaces</em>, a <a href="https://blogs.cuttingedge.it/steven/posts/2016/abstract-factories-are-a-code-smell">well-known design smell</a>. </p> <h3 id="b29309513d1946238cb08718f6be8903"> Have a sandwich <a href="#b29309513d1946238cb08718f6be8903" title="permalink">#</a> </h3> <p> It's remarkable how effectively you can lie to yourself. As late as 2017 <a href="https://softwareengineering.stackexchange.com/a/356822/19115">I still concluded that fine-grained dependencies were best</a>, despite most of my arguments pointing in another direction. </p> <p> I first encountered functional programming in 2010, but was off to a slow start. It took me years before I realised that <a href="/2017/01/27/from-dependency-injection-to-dependency-rejection">Dependency Injection isn't functional</a>. There are other ways to address the problem of separating pure functions from impure actions, the simplest of which is the <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>. </p> <p> Which parts of the application architecture are inherently impure? The usual suspects: the system clock, random number generators, the file system, databases, network resources. Notice how these are the dependencies that you usually need to replace with <a href="http://xunitpatterns.com/Test%20Double.html">Test Doubles</a> in order to make unit tests deterministic. </p> <p> It makes sense to model these as dependencies. I still define interfaces for those and use Dependency Injection to control them. I do, however, use the impureim sandwich architecture to deal with the impure actions first, so that I can then delegate all the complex decision logic to pure functions. </p> <p> <a href="/2015/05/07/functional-design-is-intrinsically-testable">Pure functions are intrinsically testable</a>, so that solves many of the problems with testability. There's still a need to test how the impure actions interact with the pure functions. Here I take a step up in the <a href="https://martinfowler.com/bliki/TestPyramid.html">Test Pyramid</a> and write just enough <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based integration tests</a> to render it probable that the integration works as intended. You can see an example of such a test <a href="/2021/01/18/parametrised-test-primitive-obsession-code-smell">here</a>. </p> <h3 id="bb9264be0f1648b48f6572e1054fab95"> Conclusion <a href="#bb9264be0f1648b48f6572e1054fab95" title="permalink">#</a> </h3> <p> From having favoured fine-grained Dependency Injection, I now write all decision logic as pure functions by default. These only need to implement interfaces if you need the <em>logic</em> of the system to be interchangeable, which isn't that often. I do still use Dependency Injection for the impure dependencies of the system. There's usually only a handful of those. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Pendulum swing: sealed by default https://blog.ploeh.dk/2021/03/08/pendulum-swing-sealed-by-default 2021-03-08T07:28:00+00:00 Mark Seemann <div id="post"> <p> <em>Inheritance is evil. Seal your classes.</em> </p> <p> This is an article in <a href="/2021/02/22/pendulum-swings">a small series of articles about personal pendulum swings</a>. Here, I document another recent change of heart that's been a long way coming. In short, I now <code>seal</code> C# classes whenever I remember to do it. </p> <h3 id="1f7f814e8f314fd291ad5fd73b12c7bf"> Background <a href="#1f7f814e8f314fd291ad5fd73b12c7bf" title="permalink">#</a> </h3> <p> After I discovered test-driven development (TDD) (circa 2003) I embarked on a quest for proper ways to enable testability. <a href="https://martinfowler.com/articles/nonDeterminism.html">Automated tests should be deterministic</a>, but real software systems rarely are. Software depends on the system clock, random number generators, the file system, the states of databases, web services, and so on. All of these may change independently of the software, making it difficult to express an automated systems test in a deterministic manner. </p> <p> This is a known problem in TDD. In order to get the system under test (SUT) under control, you have to <a href="http://bit.ly/working-effectively-with-legacy-code">introduce what Michael Feathers calls <em>seams</em></a>. In C#, there's traditionally been two ways you could do that: <a href="/2016/06/15/sut-double">extract and override</a>, and interfaces. </p> <p> The original <a href="http://amzn.to/16E4q3z">Framework Design Guidelines</a> explicitly recommended base classes over interfaces, and I wasn't wise to how unfortunate that recommendation was. For a long time, I'd define abstractions with (abstract) base classes. I was even envious of Java, where instance members are virtual (overridable) by default. In C# you must explicitly declare a method <code>virtual</code> to make it overridable. </p> <p> Abstract base classes aren't too bad if you leave them completely empty, but I never had much success with non-abstract base classes and virtual members and the whole extract-and-override manoeuvre. I soon concluded that <a href="/2016/06/15/sut-double">Dependency Injection with interfaces</a> was a better alternative. </p> <p> Even after I changed to exclusively relying on interfaces (instead of abstract base classes), remnants of the rule stuck with me for years: <em>unsealed good; sealed bad</em>. Even today, the framework design guidelines favour unsealed classes: <blockquote> <p> "CONSIDER using unsealed classes with no added virtual or protected members as a great way to provide inexpensive yet much appreciated extensibility to a framework." </p> <footer><cite><a href="https://docs.microsoft.com/dotnet/standard/design-guidelines/unsealed-classes">Framework Design Guidelines</a></cite></footer> </blockquote> I can no longer agree with this guidance; I think it's poor advice. </p> <h3 id="ae5513333afc437fb4ac795c2af8a4a1"> You don't need inheritance <a href="#ae5513333afc437fb4ac795c2af8a4a1" title="permalink">#</a> </h3> <p> Base classes imply class inheritance as a reuse and extensibility mechanism. We've known since 1994, though, that inheritance probably isn't the best design principle. <blockquote> <p> "Favor object composition over class inheritance." </p> <footer><cite><a href="http://amzn.to/XBYukB">Design Patterns</a></cite></footer> </blockquote> In single-inheritance languages like C# and Java, inheritance is just evil. Once you decide to inherit from a base class, you exclude all other base classes. <em>Inheritance signifies a single 'yes' and an infinity of 'noes'.</em> This is particularly problematic if you rely on inheritance for reuse. You can only 'reuse' a single base class, which again leads to duplication or bloated base classes. </p> <p> It's been years (probably more than a decade) since I stopped relying on base classes for anything. You don't need inheritance. <a href="https://www.haskell.org">Haskell</a> doesn't have it at all, and I only use it in C# when a framework forces me to derive from some base class. </p> <p> There's little you can do with an abstract class that you can't do in some other way. <a href="/2018/02/19/abstract-class-isomorphism">Abstract classes are isomorphic with Dependency Injection up to accessibility.</a> </p> <h3 id="516f72a613aa4884a86448da1c256ebb"> Seal <a href="#516f72a613aa4884a86448da1c256ebb" title="permalink">#</a> </h3> <p> If I already follow a design principle of not relying on inheritance, then why keep classes unsealed? <a href="https://www.python.org/dev/peps/pep-0020">Explicit is better than implicit</a>, so why not make that principle visible? Seal classes. </p> <p> It doesn't have any immediate impact on the code, but it might make it clearer to other programmers that an explicit decision was made. </p> <p> You already saw examples in the <a href="/2021/03/01/pendulum-swing-internal-by-default">previous article</a>: Both <code>Month</code> and <code>Seating</code> are sealed classes. They're also immutable records. I seal more than record types, too: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">HomeController</span></pre> </p> <p> I seal Controllers, as well as services: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SmtpPostOffice</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IPostOffice</span></pre> </p> <p> Another example is <a href="/2020/11/09/checking-signed-urls-with-aspnet">an ASP.NET filter</a> named <code>UrlIntegrityFilter</code>. </p> <p> A common counter-argument is that 'you may need extensibility in the future': <blockquote> <p> "by using "sealed" and not virtual in libs dev says "I thought of all extension point" which seems arrogant" </p> <footer><cite><a href="https://twitter.com/dun3/status/586790512528596992">Tobias Hertkorn</a></cite></footer> </blockquote> I agree that it'd be arrogant to claim that you've thought about all extension points. Trying to predict future need is futile. </p> <p> I don't agree, however, that making everything virtual is a good idea, but it's because I disagree with the underlying premise. The presupposition is that extensibility should be enabled through inheritance. If it's not already clear, I believe that this has many undesirable consequences. There are better ways to enable extensibility than through inheritance. </p> <h3 id="3ab4e94848d742aa8e3b12377415e8d1"> Conclusion <a href="#3ab4e94848d742aa8e3b12377415e8d1" title="permalink">#</a> </h3> <p> I've begun to routinely seal new classes. I don't always remember to do it, but I think that I ought to. As I also explained in the previous article, this is only my default. If something has to be a base class, that's still an option. Likewise, just because a class starts out <code>sealed</code> doesn't mean that it has to stay <code>sealed</code> forever. While sealing an unsealed class is a breaking change, unsealing a sealed class isn't. </p> <p> I can't think of any reason why I'd do that, though. </p> <p> <strong>Next: </strong> <a href="/2021/03/15/pendulum-swing-pure-by-default">Pendulum swing: pure by default</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Pendulum swing: internal by default https://blog.ploeh.dk/2021/03/01/pendulum-swing-internal-by-default 2021-03-01T08:26:00+00:00 Mark Seemann <div id="post"> <p> <em>Declare new C# classes as internal by default, and public by choice.</em> </p> <p> This is an article in <a href="/2021/02/22/pendulum-swings">a small series of articles about personal pendulum swings</a>. Here, I document a recent change of heart that's been a long way coming. In short, I now declare C# classes as <code>internal</code> unless they're driven by tests. </p> <h3 id="199c0f3d9cdc4ee5a6e93ce54a3f39a8"> Background <a href="#199c0f3d9cdc4ee5a6e93ce54a3f39a8" title="permalink">#</a> </h3> <p> When you create a new class in Visual Studio, the default accessibility is <code>internal</code>. In fact, Visual Studio's default templates don't add an access modifier at all, but if no access modifier is present, it implies <code>internal</code>. </p> <p> When I started out programming C#, I don't recall thinking much about accessibility modifiers. By default, then, I'd be using mostly <code>internal</code> classes. What little I knew about encapsulation (<em>information hiding</em>, anyone?) led me to believe that the more <code>internal</code> my code was, the better encapsulation it had. </p> <p> It's possible that I make my past self more ignorant than I actually was. It's almost twenty years ago: I don't recall all the details. </p> <h3 id="c98c637b240f4f0c97d6fc4d0f7d10f2"> Public all the things <a href="#c98c637b240f4f0c97d6fc4d0f7d10f2" title="permalink">#</a> </h3> <p> When I discovered test-driven development (TDD) (circa 2003) all my classes became public. They had to. When tests are interacting with code in another library, they can only exercise the system under test (SUT) if they can reach it. The tests make the SUT classes public. </p> <p> Yes, it's technically possible to test <code>internal</code> classes in .NET, but <a href="/2015/09/22/unit-testing-internals">I don't believe that you should</a>. I've yet to change my mind about that; no imminent pendulum swing there. You're testing something you <em>care</em> about. If the <code>internal</code> code serves any, <em>any</em>, purpose, it must be somehow observable. If so, verify that such observable behaviour takes place; if not, delete the code. (I'm sure you can dream up some corner cases where this doesn't hold; fine: I'm painting with a broad brush, here.) </p> <p> For years, I applied TDD, but I wasn't aware of the red-green-refactor cycle. I rarely changed the public API that the tests interacted with, and when I did, I made sure to adjust the tests accordingly. If a refactoring gave rise to new classes, I'd often write tests for those new classes as well. </p> <p> Imagine, for example, invoking the <em>Extract Class</em> refactoring. The new class would be as covered by tests as before the extraction, but what happens next is typically that you need to tweak it. When that happened to me, I'd typically write completely new tests to cover it. To do that, I'd need the extracted class to be <code>public</code>. </p> <p> In this phase of my professional life, my classes were almost exclusively <code>public</code>, with <code>internal</code> classes only making a rare appearance. </p> <p> One problem this tends to cause is that it makes code bases more brittle. Every type change is a potential breaking change. When every public class is covered by tests, this makes tests brittle. </p> <p> I think that it's relevant to consider the context of the code base. At this phase of my professional life, I maintained <a href="https://github.com/AutoFixture/AutoFixture">AutoFixture</a>, a fairly popular open-source library. I wanted that library to be stable so that users could trust it. I considered the test suite a guard of the contract. As long as a change didn't break any test, I considered it likely that it wasn't a breaking change. Thus, I was already conservative when it came to editing tests. I <a href="/2013/04/02/why-trust-tests">considered test to be append-only</a> in principle. </p> <p> I still consider it prudent to be conservative when it comes to a library with a public API. This doesn't mean, however, that this line of thinking carries over to code bases without a public (language-level) API. This may include web sites and services, but could also include installed apps. As long as there's no public API, there's no contract to break. </p> <h3 id="af12c8cba91944a29b9c377f5c3e045f"> Internal by default <a href="#af12c8cba91944a29b9c377f5c3e045f" title="permalink">#</a> </h3> <p> In 2020 I wrote a REST API of middling complexity. I used <a href="/outside-in-tdd">outside-in TDD</a> as a major driver. In the spirit of behaviour-driven development I favour describing the observable behaviour of the system. I use <a href="/2021/01/25/self-hosted-integration-tests-in-aspnet">self-hosted</a> <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based integration tests</a> for this purpose. Only when I find that these tests get too complex do I grudgingly drop down to the unit-test level. </p> <p> The things that I test with unit tests have to be <code>public</code>. This still leaves plenty of room for behaviour described by the integration tests to have <code>internal</code> implementation details. The code base I mentioned has several examples of that. Some of them I've already described here on the blog. </p> <p> For example, notice that the <code>LinksFilter</code> <a href="/2020/08/24/adding-rest-links-as-a-cross-cutting-concern">shown here</a> is an <code>internal</code> class. Its behaviour is covered by abundant integration tests, so I'm not afraid to refactor it if need be. Those <code>LinkToYear</code>, <code>LinkToMonth</code>, and <code>LinkToDay</code> extension methods that it uses <a href="/2020/08/10/an-aspnet-core-url-builder">are internal too</a>. </p> <p> Another example is the <code>UrlIntegrityFilter</code> <a href="/2020/11/09/checking-signed-urls-with-aspnet">seen here</a>. The class itself is <code>internal</code> and its behaviour is composed from <code>private</code> helper functions. Its <a href="/2020/11/02/signing-urls-with-aspnet">counterpart</a> <code>SigningUrlHelper</code> is also <code>internal</code>. (Its companion <code>SigningUrlHelperFactory</code>, shown in the same article, is <code>public</code>, but that's an oversight on my part. It can easily be <code>internal</code> as well.) All that URL-signing behaviour is, again, covered by tests that verify the behaviour of the REST API. </p> <p> Another example from the same code base can be found in its so-called <em>calendar</em> feature. The system is an online restaurant reservation system. It allows clients to browse a day, a month, or even a year to see if there are any free spots for a given time slot. You can <a href="/2020/08/24/adding-rest-links-as-a-cross-cutting-concern">see an example here</a>. While I test-drove the calendar feature with integration tests, it quickly dawned on me that I had three disparate cases (day, month, year) that essentially represented the same concept: a <em>period</em>. </p> <p> A period is a closed set of heterogeneous data. A year contains only a single datum: the year itself (e.g. 2021). A month contains both a month and a year, and so on. A closed set of heterogeneous data describes a <a href="https://en.wikipedia.org/wiki/Tagged_union">sum type</a>, and since I know that in object-oriented programming, <a href="/2018/06/25/visitor-as-a-sum-type">sum types can be encoded as Visitors</a>, I introduced a Visitor API: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IPeriod</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">T</span>&nbsp;Accept&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:#2b91af;">IPeriodVisitor</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;visitor); } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">interface</span>&nbsp;<span style="color:#2b91af;">IPeriodVisitor</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">T</span>&nbsp;VisitYear(<span style="color:blue;">int</span>&nbsp;year); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">T</span>&nbsp;VisitMonth(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">T</span>&nbsp;VisitDay(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month,&nbsp;<span style="color:blue;">int</span>&nbsp;day); }</pre> </p> <p> I decided, however, to keep this API <code>internal</code>, since this isn't the only possible way to model this feature. As is the case with the other examples I've shown here, the behaviour is covered by integration tests. I feel free to refactor. In fact, this Visitor-based API is actually the result of a refactoring from something more ad hoc that I didn't like. </p> <p> Here's one of the three <code>IPeriod</code> implementation, in case you're curious: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Month</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IPeriod</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;year; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">int</span>&nbsp;month; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Month</span>(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.year&nbsp;=&nbsp;year; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.month&nbsp;=&nbsp;month; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;Accept&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:#2b91af;">IPeriodVisitor</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;visitor) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;visitor.VisitMonth(year,&nbsp;month); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>?&nbsp;obj) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">Month</span>&nbsp;month&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;year&nbsp;==&nbsp;month.year&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.month&nbsp;==&nbsp;month.month; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;GetHashCode() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">HashCode</span>.Combine(year,&nbsp;month); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This class, too, is <code>internal</code>, as are its two companions <code>Day</code> and <code>Year</code>. I'll leave it as an exercise for the interested reader to implement these two classes, as well as <code>IPeriodVisitor&lt;T&gt;</code> implementations that return the next or previous period, or the first or last tick of the period, etcetera. </p> <h3 id="af9c22ec88af4e3fa5f4b5c2356b5692"> Public by choice <a href="#af9c22ec88af4e3fa5f4b5c2356b5692" title="permalink">#</a> </h3> <p> This shifted emphasis of mine isn't a return to a simpler time. It's not <em>internal all the things!</em> It's about shifting the default for classes that are <em>not</em> driven by tests. Those classes that are artefacts of TDD are still <code>public</code> since <a href="/2015/09/22/unit-testing-internals">I don't directly unit test internal classes</a>. </p> <p> Other classes may start out as <code>internal</code> and then get promoted to <code>public</code> by choice. For example, I'd introduced a <code>Seating</code> class in the code base to model how long a seating was supposed to take: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Seating</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">Seating</span>(<span style="color:#2b91af;">TimeSpan</span>&nbsp;seatingDuration,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SeatingDuration&nbsp;=&nbsp;seatingDuration; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;=&nbsp;reservation; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Members&nbsp;follow...</span></pre> </p> <p> Some restaurants have second seatings (or more). They give you a predefined duration after which you're supposed to be done so that they can reuse your table for another party. I'd used the <code>Seating</code> class to encapsulate some logic related to that, such as the <code>Overlaps</code> method: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;Start { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation.At;&nbsp;} } <span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;End { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span>&nbsp;{&nbsp;<span style="color:blue;">return</span>&nbsp;Start&nbsp;+&nbsp;SeatingDuration;&nbsp;} } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Reservation</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;otherSeating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;other); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Start&nbsp;&lt;&nbsp;otherSeating.End&nbsp;&amp;&amp;&nbsp;otherSeating.Start&nbsp;&lt;&nbsp;End; }</pre> </p> <p> While I considered this a well-designed little class with good encapsulation, I kept it <code>internal</code> simply because there was no need to make it <code>public</code>. It was indirectly covered by test cases, but it was a result of a refactoring and not directly test-driven. </p> <p> As I started to add a new feature, I realised that I'd be able to write new unit tests in a better way if I could reuse <code>Seating</code> and a variation of its <code>Overlaps</code> method. I considered it carefully and decided to make the class and its members public: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Seating</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Seating</span>(<span style="color:#2b91af;">TimeSpan</span>&nbsp;seatingDuration,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SeatingDuration&nbsp;=&nbsp;seatingDuration; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reservation&nbsp;=&nbsp;reservation; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Members&nbsp;follow...</span></pre> </p> <p> I made this decision after explicit deliberation. It didn't take long, though, but I did shortly stop to consider whether this seemed like a good idea. This code base isn't a <a href="/2012/12/18/RangersandZookeepers">reusable library in the wild</a>, so I wasn't concerned about misuse of the API. I did consider, on the other hand, how this would increase coupling between the tests and the production code base. It didn't take me long to decide that in this case, I was okay with that. </p> <p> <code>Seating</code> had already existed as an <code>internal</code> class for some time and had proven useful and stable. Putting on my <a href="http://amzn.to/WBCwx7">DDD</a> hat, I also thought that <code>Seating</code> represented a proper domain concept. </p> <h3 id="f6ef0cc965d845fe889170ee956baa4d"> Conclusion <a href="#f6ef0cc965d845fe889170ee956baa4d" title="permalink">#</a> </h3> <p> You can go back and forth on how you write code; which rules of thumb you apply. For many years, I favoured <code>public</code> classes. I think that I even, at one time, tweaked the Visual Studio templates to explicitly create new classes as <code>public</code>. </p> <p> Now, I've changed my heuristic. Classes driven into existence by tests are <code>public</code>; they have to be. Other classes I now make <em>internal by default, and public by choice</em>. </p> <p> This is going to be my rule until I change it. </p> <p> <strong>Next:</strong> <a href="/2021/03/08/pendulum-swing-sealed-by-default">Pendulum swing: sealed by default</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Pendulum swings https://blog.ploeh.dk/2021/02/22/pendulum-swings 2021-02-22T08:04:00+00:00 Mark Seemann <div id="post"> <p> <em>The software development industry goes back and forth on how to do things, and so do I.</em> </p> <p> I've been working with something IT-related since 1994, and I've been a professional programmer since 1999. When you observe the software development industry over decades, you may start to notice some trends. One decade, service-oriented architecture (SOA) is cool; the next, consolidation sets in; then it's micro-services; and, as far as I can tell, monoliths are on the way in again, although I'm sure that we'll find something else to call them. </p> <p> It's as if a pendulum swings from one extreme to the other. Sooner or later, it comes back, only to then continue its swing in the other direction. If you view it over time and assume no loss to friction, a pendulum describes a sine wave. </p> <p> <img src="/content/binary/sine-wave.png" alt="A sine wave."> </p> <p> There's probably several reasons for this motion. The benign interpretation is that it's still a young industry and we're still learning. It's not uncommon to see oscillations in dynamic systems, particularly when feedback isn't immediate. </p> <p> Software architecture tends to produce slow feedback. Architecture solves more than one problem, including scalability, but a major motivation to think about architecture is to pick a way to organise the source code so that you don't have to rewrite from scratch every 2-3 years. Tautologically, then, it takes years before you know whether or not you succeeded. </p> <p> While waiting for feedback, you may continue doing what you believe is right: micro-services versus monoliths, unit tests versus acceptance tests, etcetera. Once you discover that a particular way to work has problems, you may overcompensate by going too far in the other direction. </p> <p> Once you discover the problem with that, you may begin to pull back towards the original position. Because feedback is delayed, the pendulum once more swings too far. </p> <p> If we manage to learn from our mistakes, one could hope that the oscillations we currently observe will dampen until we reach equilibrium in the future. The industry is still so young, though, that the pendulum makes wide swings. Perhaps it'll takes decades, or even centuries, before the oscillations die down. </p> <p> The more cynic interpretation is that most software developers have only a few years of professional experience, and aren't taught the experiences of past generations. <blockquote> <p> "Those who cannot remember the past are condemned to repeat it." </p> <footer><cite>George Santayana</cite></footer> </blockquote> In this light, the industry keeps regurgitating the same ideas over and over, never learning from past mistakes. </p> <p> The truth is probably a mix of both explanations. </p> <h3 id="36d029a90bfa4d35a7e8fc10048b8bcc"> Personal pendulum <a href="#36d029a90bfa4d35a7e8fc10048b8bcc" title="permalink">#</a> </h3> <p> I've noticed a similar tendency in myself. I work in a particular way until I run into the limitations of that way. Then, after a time of frustration, I change direction. </p> <p> As an example, I'm an autodidact programmer. In the beginning of my career, I'd just throw together code until I thought it worked, then launch the software with the debugger attached only to discover that it didn't, then go back and tweak some more, and so on. </p> <p> Then I discovered test-driven development (TDD) and for years, it was the only way I could conceive of working. As my experience with TDD grew, I started to notice that it wasn't the panacea that I believed when it was all new. <a href="/2010/12/22/TheTDDApostate">I wrote about that as early as late 2010</a>. Knowing myself, I'd probably started to notice problems with TDD before that. I have cognitive biases just like the next person. You can lie to yourself for years before the problems become so blatant that you can no longer ignore them. </p> <p> To be clear, I never lost faith in TDD, but I began to glimpse the contours of its limitations. It's good for many circumstances, and it's still my preferred technique for developing new production code, but I use other techniques for e.g. prototyping. </p> <p> In 2020 I wrote a code base of middling complexity, and I noticed that I'd started to change my position on some other long-standing practices. As I've tried to explain, it may look like pendulum swings, but I hope that they are, at least, dampened swings. I intend to observe what happens so that I can learn from these new directions. </p> <p> In the following, I'll be writing about these new approaches that I'm trying on, and so far like: <ul> <li><a href="/2021/03/01/pendulum-swing-internal-by-default">Pendulum swing: internal by default</a></li> <li><a href="/2021/03/08/pendulum-swing-sealed-by-default">Pendulum swing: sealed by default</a></li> <li><a href="/2021/03/15/pendulum-swing-pure-by-default">Pendulum swing: pure by default</a></li> </ul> I'd be naive if I believed these to be my final words on any of these topics. I'm currently trying them out for size; in a few decades I'll know more about how it all turns out. </p> <h3 id="fdb1ddeb6709428ca1f0e7f441085b3d"> Conclusion <a href="#fdb1ddeb6709428ca1f0e7f441085b3d" title="permalink">#</a> </h3> <p> One year TDD is all the rage; a few years later, it's BDD. One year it's SOA, then it's <a href="https://alistair.cockburn.us/hexagonal-architecture/">ports and adapters</a> (which implies consolidated deployment), then it's micro-services. One year, it's XML, then it's JSON, then it's YAML. One decade it's structured programming, then it's object-orientation, then it's functional programming, and so on ad nauseam. </p> <p> Hopefully, this is just a symptom of growing pains. Hopefully, we'll learn from all these wild swings so that we don't have to rewrite applications when older developers leave. </p> <p> The only course of action that I can see for myself here is to document how I work so that I, and others, can learn from those experiences. </p> <p> <strong>Next:</strong> <a href="/2021/03/01/pendulum-swing-internal-by-default">Pendulum swing: internal by default</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. When properties are easier than examples https://blog.ploeh.dk/2021/02/15/when-properties-are-easier-than-examples 2021-02-15T07:33:00+00:00 Mark Seemann <div id="post"> <p> <em>Sometimes, describing the properties of a function is easier than coming up with examples.</em> </p> <p> Instead of the term <em>test-driven development</em> you may occasionally encounter the phrase <em>example-driven development</em>. The idea is that each test is an example of how the system under test ought to behave. As you add more tests, you add more examples. </p> <p> I've noticed that beginners often find it difficult to come up with good examples. This is the reason I've developed the <a href="/2019/10/07/devils-advocate">Devil's advocate</a> technique. It's meant as a heuristic that may help you identify the next good example. It's particularly effective if you combine it with the <a href="https://blog.cleancoder.com/uncle-bob/2013/05/27/TheTransformationPriorityPremise.html">Transformation Priority Premise</a> (TPP) and <a href="https://en.wikipedia.org/wiki/Equivalence_partitioning">equivalence partitioning</a>. </p> <p> I've noticed, however, that translating concrete examples into code is not always straightforward. In the following, I'll describe an experience I had in 2020 while developing an online restaurant reservation system. </p> <h3 id="83a8dd13c9694ebc99b5fa4ae049b3b4"> Problem outline <a href="#83a8dd13c9694ebc99b5fa4ae049b3b4" title="permalink">#</a> </h3> <p> I'm going to start by explaining what it was that I was trying to do. I wanted to present the <a href="https://en.wikipedia.org/wiki/Ma%C3%AEtre_d%27h%C3%B4tel">maître d'</a> (or other restaurant staff) with a schedule of a day's reservations. It should take the form of a list of time entries, one entry for every time one or more new reservations would start. I also wanted to list, for each entry, all reservations that were currently ongoing, or would soon start. Here's a simple example, represented as JSON: </p> <p> <pre><span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2023-08-23&quot;</span>, <span style="color:#2e75b6;">&quot;entries&quot;</span>:&nbsp;[ &nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;reservations&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;af5feb35f62f475cb02df2a281948829&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2023-08-23T20:00:00.0000000&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;crystalmeth@example.net&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Crystal&nbsp;Metheney&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;3 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;eae39bc5b3a7408eb2049373b2661e32&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2023-08-23T20:30:00.0000000&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;x.benedict@example.org&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Benedict&nbsp;Xavier&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;}, &nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;reservations&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;af5feb35f62f475cb02df2a281948829&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2023-08-23T20:00:00.0000000&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;crystalmeth@example.net&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Crystal&nbsp;Metheney&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;3 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;eae39bc5b3a7408eb2049373b2661e32&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2023-08-23T20:30:00.0000000&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;x.benedict@example.org&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Benedict&nbsp;Xavier&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;} ]</pre> </p> <p> To keep the example simple, there are only two reservations for that particular day: one for 20:00 and one for 20:30. Since something happens at both of these times, both time has an entry. My intent isn't necessarily that a user interface should show the data in this way, but I wanted to make the relevant data available so that a user interface could show it if it needed to. </p> <p> The first entry for 20:00 shows both reservations. It shows the reservation for 20:00 for obvious reasons, and it shows the reservation for 20:30 to indicate that the staff can expect a party of four at 20:30. Since this restaurant runs with a single seating per evening, this effectively means that although the reservation hasn't started yet, it still reserves a table. This gives a user interface an opportunity to show the state of the restaurant at that time. The table for the 20:30 party isn't active yet, but it's effectively reserved. </p> <p> For restaurants with shorter seating durations, the schedule should reflect that. If the seating duration is, say, two hours, and someone has a reservation for 20:00, you can sell that table to another party at 18:00, but not at 18:30. I wanted the functionality to take such things into account. </p> <p> The other entry in the above example is for 20:30. Again, both reservations are shown because one is ongoing (and takes up a table) and the other is just starting. </p> <h3 id="63e308b1c630441e8f55fe3ecaa5a03f"> Desired API <a href="#63e308b1c630441e8f55fe3ecaa5a03f" title="permalink">#</a> </h3> <p> A major benefit of test-driven development (TDD) is that you get fast feedback on the API you intent for the system under test (SUT). You write a test against the intended API, and besides a pass-or-fail result, you also learn something about the interaction between client code and the SUT. You often learn that the original design you had in mind isn't going to work well once it meets the harsh realities of an actual programming language. </p> <p> In TDD, you often have to revise the design multiple times during the process. </p> <p> This doesn't mean that you can't have a plan. You can't write the initial test if you have no inkling of what the API should look like. For the schedule feature, I did have a plan. It turned out to hold, more or less. I wanted the API to be a method on a class called <code>MaitreD</code>, which already had these four fields and the constructors to support them: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;OpensAt&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;LastSeating&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>&nbsp;SeatingDuration&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;Tables&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;}</pre> </p> <p> I planned to implement the new feature as a new instance method on that class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>)</pre> </p> <p> This plan turned out to hold in general, although I ultimately decided to simplify the return type by getting rid of the <code>Occurrence</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a>. It's going to be present throughout this article, however, so I need to briefly introduce it. I meant to use it as a generic container of anything, but with an time-stamp associated with the value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>(<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="color:#1f377f;">at</span>,&nbsp;<span style="color:#2b91af;">T</span>&nbsp;<span style="color:#1f377f;">value</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;<span style="color:#1f377f;">at</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Value&nbsp;=&nbsp;<span style="color:#1f377f;">value</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;At&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;Value&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;<span style="color:#74531f;">Select</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;<span style="color:#1f377f;">selector</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(<span style="color:#1f377f;">selector</span>&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(<span style="color:#1f377f;">selector</span>)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(At,&nbsp;<span style="color:#1f377f;">selector</span>(Value)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;<span style="color:#74531f;">Equals</span>(<span style="color:blue;">object</span>?&nbsp;<span style="color:#1f377f;">obj</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#1f377f;">obj</span>&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;<span style="color:#1f377f;">occurrence</span>&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;<span style="color:#74531f;">==</span>&nbsp;<span style="color:#1f377f;">occurrence</span>.At&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">EqualityComparer</span>&lt;<span style="color:#2b91af;">T</span>&gt;.Default.<span style="color:#74531f;">Equals</span>(Value,&nbsp;<span style="color:#1f377f;">occurrence</span>.Value); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#74531f;">GetHashCode</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">HashCode</span>.<span style="color:#74531f;">Combine</span>(At,&nbsp;Value); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> You may notice that due to the presence of the <code>Select</code> method this is a <a href="/2018/03/22/functors">functor</a>. </p> <p> There's also a little extension method that we may later encounter: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;<span style="color:#74531f;">At</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;<span style="color:#1f377f;">value</span>,&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="color:#1f377f;">at</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">T</span>&gt;(<span style="color:#1f377f;">at</span>,&nbsp;<span style="color:#1f377f;">value</span>); }</pre> </p> <p> The plan, then, is to return a collection of occurrences, each of which may contain a collection of tables that are relevant to include at that time entry. </p> <h3 id="d121920365874005a3bf46d0555bd008"> Examples <a href="#d121920365874005a3bf46d0555bd008" title="permalink">#</a> </h3> <p> When I embarked on developing this feature, I thought that it was a good fit for example-driven development. Since the input for <code>Schedule</code> requires a collection of <code>Reservation</code> objects, each of which comes with some data, I expected the test cases to become verbose. So I decided to bite the bullet right away and define test cases using <a href="https://xunit.net">xUnit.net</a>'s <code>[ClassData]</code> feature. I wrote this test: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>,&nbsp;<span style="color:#2b91af;">ClassData</span>(<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">ScheduleTestCases</span>))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">Schedule</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">MaitreD</span>&nbsp;<span style="color:#1f377f;">sut</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">Table</span>[]&gt;&gt;&nbsp;<span style="color:#1f377f;">expected</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sut</span>.<span style="color:#74531f;">Schedule</span>(<span style="color:#1f377f;">reservations</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">expected</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">ts</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">ts</span>.<span style="color:#74531f;">AsEnumerable</span>())), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>); }</pre> </p> <p> This is almost as simple as it can be: Call the method and verify that <code>expected</code> is equal to <code>actual</code>. The only slightly complicated piece is the nested projection of <code>expected</code> from <code>IEnumerable&lt;Occurrence&lt;Table[]&gt;&gt;</code> to <code>IEnumerable&lt;Occurrence&lt;IEnumerable&lt;Table&gt;&gt;&gt;</code>. There are ugly reasons for this that I don't want to discuss here, since they have no bearing on the actual topic, which is coming up with tests. </p> <p> I also added the <code>ScheduleTestCases</code> class and a single test case: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ScheduleTestCases</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TheoryData</span>&lt;<span style="color:#2b91af;">MaitreD</span>,&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;,&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">Table</span>[]&gt;&gt;&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ScheduleTestCases</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;No&nbsp;reservations,&nbsp;so&nbsp;no&nbsp;occurrences:</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#74531f;">Add</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(6), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Array</span>.<span style="color:#74531f;">Empty</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Array</span>.<span style="color:#74531f;">Empty</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">Table</span>[]&gt;&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The simplest implementation that passed that test was this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">break</span>; }</pre> </p> <p> Okay, hardly rocket science, but this was just a test case to get started. So I added another one: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">SingleReservationCommunalTable</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">table</span>&nbsp;=&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r</span>&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#74531f;">Add</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(6), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">table</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#1f377f;">r</span>&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#1f377f;">table</span>.<span style="color:#74531f;">Reserve</span>(<span style="color:#1f377f;">r</span>)&nbsp;}.<span style="color:#74531f;">At</span>(<span style="color:#1f377f;">r</span>.At)&nbsp;}); }</pre> </p> <p> This test case adds a single reservation to a restaurant with a single communal table. The <code>expected</code> result is now a single occurrence with that reservation. In true TDD fashion, this new test case caused a test failure, and I now had to adjust the <code>Schedule</code> method to pass all tests: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">if</span>&nbsp;(<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Any</span>()) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">r</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">First</span>(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12).<span style="color:#74531f;">Reserve</span>(<span style="color:#1f377f;">r</span>)&nbsp;}.<span style="color:#74531f;">AsEnumerable</span>().<span style="color:#74531f;">At</span>(<span style="color:#1f377f;">r</span>.At); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">yield</span>&nbsp;<span style="color:#8f08c4;">break</span>; }</pre> </p> <p> You might have wanted to jump to something prettier right away, but I wanted to proceed according to the Devil's advocate technique. I was concerned that I was going to mess up the implementation if I moved too fast. </p> <p> And that was when I basically hit a wall. </p> <h3 id="c8fd655d50414215b04f891676a09bd8"> Property-based testing to the rescue <a href="#c8fd655d50414215b04f891676a09bd8" title="permalink">#</a> </h3> <p> I couldn't figure out how to proceed from there. Which test case ought to be the next? I wanted to follow the spirit of the TPP and pick a test case that would cause another incremental step in the right direction. The sheer number of possible combinations overwhelmed me, though. Should I adjust the reservations? The table configuration for the <code>MaitreD</code> class? The <code>SeatingDuration</code>? </p> <p> It's possible that you'd be able to conjure up the perfect next test case, but I couldn't. I actually let it stew for a couple of days before I decided to give up on the example-driven approach. While I couldn't see a clear path forward with concrete examples, I had a vivid vision of how to proceed with <a href="/property-based-testing-intro">property-based testing</a>. </p> <p> I left the above tests in place and instead added a new test class to my code base. Its only purpose: to test the <code>Schedule</code> method. The test method itself is only a composition of various data definitions and the actual test code: </p> <p> <pre>[<span style="color:#2b91af;">Property</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Property</span>&nbsp;<span style="color:#74531f;">Schedule</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">Prop</span>.<span style="color:#74531f;">ForAll</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GenReservation.<span style="color:#74531f;">ArrayOf</span>().<span style="color:#74531f;">ToArbitrary</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#74531f;">ScheduleImp</span>); }</pre> </p> <p> This uses <a href="https://fscheck.github.io/FsCheck">FsCheck</a> 2.14.3, which is written in <a href="https://fsharp.org">F#</a> and composes better if you also write the tests in F#. In order to make things a little more palatable for C# developers, I decided to implement the building blocks for the property using methods and class properties. </p> <p> The <code>ScheduleImp</code> method, for example, actually implements the test. This method runs a hundred times (FsCheck's default value) with randomly generated input values: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ScheduleImp</span>(<span style="color:#2b91af;">Reservation</span>[]&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Create&nbsp;a&nbsp;table&nbsp;for&nbsp;each&nbsp;reservation,&nbsp;to&nbsp;ensure&nbsp;that&nbsp;all</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;reservations&nbsp;can&nbsp;be&nbsp;allotted&nbsp;a&nbsp;table.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">tables</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Standard</span>(<span style="color:#1f377f;">r</span>.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">sut</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(6), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">tables</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sut</span>.<span style="color:#74531f;">Schedule</span>(<span style="color:#1f377f;">reservations</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">r</span>.At).<span style="color:#74531f;">Distinct</span>().<span style="color:#74531f;">Count</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Count</span>()); }</pre> </p> <p> The step you see in the first line of code is an example of a trick that I find myself doing often with property-based testing: instead of trying to find some good test values for a particular set of circumstances, I create a set of circumstances that fits the randomly generated test values. As the code comment explains, given a set of <code>Reservation</code> values, it creates a table that fits each reservation. In that way I ensure that all the reservations can be allocated a table. </p> <p> I'll soon return to how those random <code>Reservation</code> values are generated, but first let's discuss the rest of the test body. Given a valid <code>MaitreD</code> object it calls the <code>Schedule</code> method. In the <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">assertion phase</a>, it so far only verifies that there's as many time entries in <code>actual</code> as there are distinct <code>At</code> values in <code>reservations</code>. </p> <p> That's hardly a comprehensive description of the SUT, but it's a start. The following implementation passes both the new property, as well as the two examples above. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12).<span style="color:#74531f;">Reserve</span>(r)&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;g.<span style="color:#74531f;">AsEnumerable</span>().<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> I know that many C# programmers don't like query syntax, but I've always had a soft spot for it. I liked it, but wasn't sure that I'd be able to keep it up as I added more constraints to the property. </p> <h3 id="5670f47a815542bfac848645e9a3ccf4"> Generators <a href="#5670f47a815542bfac848645e9a3ccf4" title="permalink">#</a> </h3> <p> Before we get to that, though, I promised to show you how the random <code>reservations</code> are generated. FsCheck has an API for that, and it's also query-syntax-friendly: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Email</span>&gt;&nbsp;GenEmail&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;s&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Arb</span>.<span style="color:#2b91af;">Default</span>.<span style="color:#74531f;">NonWhiteSpaceString</span>().Generator &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(s.Item); <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Name</span>&gt;&nbsp;GenName&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;s&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Arb</span>.<span style="color:#2b91af;">Default</span>.<span style="color:#74531f;">StringWithoutNullChars</span>().Generator &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(s.Item); <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;GenReservation&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;id&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Arb</span>.<span style="color:#2b91af;">Default</span>.<span style="color:#74531f;">Guid</span>().Generator &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;d&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Arb</span>.<span style="color:#2b91af;">Default</span>.<span style="color:#74531f;">DateTime</span>().Generator &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;e&nbsp;<span style="color:blue;">in</span>&nbsp;GenEmail &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;n&nbsp;<span style="color:blue;">in</span>&nbsp;GenName &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;q&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Arb</span>.<span style="color:#2b91af;">Default</span>.<span style="color:#74531f;">PositiveInt</span>().Generator &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(id,&nbsp;d,&nbsp;e,&nbsp;n,&nbsp;q.Item);</pre> </p> <p> <code>GenReservation</code> is a generator of <code>Reservation</code> values (for a simplified explanation of how such a generator might work, see <a href="/2017/09/18/the-test-data-generator-functor">The Test Data Generator functor</a>). It's composed from smaller generators, among these <code>GenEmail</code> and <code>GenName</code>. The rest of the generators are general-purpose generators defined by FsCheck. </p> <p> If you refer back to the <code>Schedule</code> property above, you'll see that it uses <code>GenReservation</code> to produce an array generator. This is another general-purpose combinator provided by FsCheck. It turns any single-object generator into a generator of arrays containing such objects. Some of these arrays will be empty, which is often desirable, because it means that you'll automatically get coverage of that edge case. </p> <h3 id="ad13aac51d654a7f988b1106579f42fd"> Iterative development <a href="#ad13aac51d654a7f988b1106579f42fd" title="permalink">#</a> </h3> <p> As I already <a href="/2015/01/10/diamond-kata-with-fscheck">discovered in 2015</a> some problems are just much better suited for property-based development than example-driven development. As I expected, this one turned out to be just such a problem. (Recently, <a href="https://www.hillelwayne.com">Hillel Wayne</a> identified a set of problems with no clear properties as <a href="https://buttondown.email/hillelwayne/archive/cross-branch-testing/">rho problems</a>. I wonder if we should pick another Greek letter for this type of problems that almost ooze properties. Sigma problems? Maybe we should just call them <em>describable problems</em>...) </p> <p> For the next step, I didn't have to write a completely new property. I only had to add a new assertion, and thereby strengthening the postconditions of the <code>Schedule</code> method: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At).<span style="color:#74531f;">OrderBy</span>(<span style="color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">d</span>), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At));</pre> </p> <p> I added the above assertion to <code>ScheduleImp</code> after the previous assertion. It simply states that <code>actual</code> should be sorted in ascending order. </p> <p> To pass this new requirement I added an ordering clause to the implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12).<span style="color:#74531f;">Reserve</span>(r)&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">orderby</span>&nbsp;g.Key &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;g.<span style="color:#74531f;">AsEnumerable</span>().<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> It passes all tests. Commit to Git. Next. </p> <h3 id="040edba86dfd4bfe8a57cef15b4a9ff6"> Table configuration <a href="#040edba86dfd4bfe8a57cef15b4a9ff6" title="permalink">#</a> </h3> <p> If you consider the current implementation, there's much not to like. The worst offence, I think, is that it conjures a hard-coded communal table out of thin air. The method ought to use the table configuration passed to the <code>MaitreD</code> object. This seems like an obvious flaw to address. I therefore added this to the property: </p> <p> <pre>&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">All</span>(<span style="color:#1f377f;">actual</span>,&nbsp;<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">AssertTables</span>(<span style="color:#1f377f;">tables</span>,&nbsp;<span style="color:#1f377f;">o</span>.Value)); } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">AssertTables</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;<span style="color:#1f377f;">expected</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;<span style="color:#1f377f;">actual</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>(<span style="color:#1f377f;">expected</span>.<span style="color:#74531f;">Count</span>(),&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Count</span>()); }</pre> </p> <p> It's just another assertion that uses the helper assertion also shown. As a first pass, it's not enough to cheat the Devil, but it sets me up for my next move. The plan is to assert that no tables are generated out of thin air. Currently, <code>AssertTables</code> only verifies that the actual count of tables in each occurrence matches the expected count. </p> <p> The Devil easily foils that plan by generating a table for each reservation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">tables</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Communal</span>(12).<span style="color:#74531f;">Reserve</span>(<span style="color:#1f377f;">r</span>)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;r&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">orderby</span>&nbsp;g.Key &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:#1f377f;">tables</span>.<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> This (unfortunately) passes all tests, so commit to Git and move on. </p> <p> The next move I made was to add an assertion to <code>AssertTables</code>: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">expected</span>.<span style="color:#74531f;">Sum</span>(<span style="color:#1f377f;">t</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">t</span>.Capacity), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Sum</span>(<span style="color:#1f377f;">t</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">t</span>.Capacity));</pre> </p> <p> This new requirement states that the total capacity of the actual tables should be equal to the total capacity of the allocated tables. It doesn't prevent the Devil from generating tables out of thin air, but it makes it harder. At least, it makes it so hard that I found it more reasonable to use the supplied table configuration: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">tables</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Zip</span>(Tables,&nbsp;(<span style="color:#1f377f;">r</span>,&nbsp;<span style="color:#1f377f;">t</span>)&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">t</span>.<span style="color:#74531f;">Reserve</span>(<span style="color:#1f377f;">r</span>)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;r&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">orderby</span>&nbsp;g.Key &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:#1f377f;">tables</span>.<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> The implementation of <code>Schedule</code> still cheats because it 'knows' that no tests (except for the degenerate test where there are no reservations) have surplus tables in the configuration. It takes advantage of that knowledge to zip the two collections, which is really not appropriate. </p> <p> Still, it seems that things are moving in the right direction. </p> <h3 id="5891a00f231f46b68afb0b6aff43b0b0"> Generated SUT <a href="#5891a00f231f46b68afb0b6aff43b0b0" title="permalink">#</a> </h3> <p> Until now, <code>ScheduleImp</code> has been using a hard-coded <code>sut</code>. It's time to change that. </p> <p> To keep my steps as small as possible, I decided to start with the <code>SeatingDuration</code> since it was currently not being used by the implementation. This meant that I could start randomising it without affecting the SUT. Since this was a code change of middling complexity in the test code, I found it most prudent to move in such a way that I didn't have to change the SUT as well. </p> <p> I completely extracted the initialisation of the <code>sut</code> to a method argument of the <code>ScheduleImp</code> method, and adjusted it accordingly: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ScheduleImp</span>(<span style="color:#2b91af;">MaitreD</span>&nbsp;<span style="color:#1f377f;">sut</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>[]&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sut</span>.<span style="color:#74531f;">Schedule</span>(<span style="color:#1f377f;">reservations</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">r</span>.At).<span style="color:#74531f;">Distinct</span>().<span style="color:#74531f;">Count</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Count</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At).<span style="color:#74531f;">OrderBy</span>(<span style="color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">d</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">All</span>(<span style="color:#1f377f;">actual</span>,&nbsp;<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">AssertTables</span>(<span style="color:#1f377f;">sut</span>.Tables,&nbsp;<span style="color:#1f377f;">o</span>.Value)); }</pre> </p> <p> This meant that I also had to adjust the calling property: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Property</span>&nbsp;<span style="color:#74531f;">Schedule</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">Prop</span>.<span style="color:#74531f;">ForAll</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GenReservation &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">ArrayOf</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">SelectMany</span>(<span style="color:#1f377f;">rs</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">GenMaitreD</span>(<span style="color:#1f377f;">rs</span>).<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">m</span>&nbsp;=&gt;&nbsp;(<span style="color:#1f377f;">m</span>,&nbsp;<span style="color:#1f377f;">rs</span>))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">ToArbitrary</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">ScheduleImp</span>(<span style="color:#1f377f;">t</span>.m,&nbsp;<span style="color:#1f377f;">t</span>.rs)); }</pre> </p> <p> You've already seen <code>GenReservation</code>, but <code>GenMaitreD</code> is new: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">MaitreD</span>&gt;&nbsp;<span style="color:#74531f;">GenMaitreD</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Create&nbsp;a&nbsp;table&nbsp;for&nbsp;each&nbsp;reservation,&nbsp;to&nbsp;ensure&nbsp;that&nbsp;all</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;reservations&nbsp;can&nbsp;be&nbsp;allotted&nbsp;a&nbsp;table.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">tables</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Standard</span>(<span style="color:#1f377f;">r</span>.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;seatingDuration&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Choose</span>(1,&nbsp;6) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(seatingDuration), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">tables</span>); }</pre> </p> <p> The only difference from before is that the new <code>MaitreD</code> object is now initialised from within a generator expression. The duration is randomly picked from the range of one to six hours (those numbers are my arbitrary choices). </p> <p> Notice that it's possible to base one generator on values randomly generated by another generator. Here, <code>reservations</code> are randomly produced by <code>GenReservation</code> and merged to a tuple with <code>SelectMany</code>, as you can see above. </p> <p> This in itself didn't impact the SUT, but set up the code for my next move, which was to generate <em>more</em> tables than reservations, so that there'd be some free tables left after the schedule allocation. I first added a more complex table generator: </p> <p> <pre><span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">summary</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;Generate&nbsp;a&nbsp;table&nbsp;configuration&nbsp;that&nbsp;can&nbsp;at&nbsp;minimum&nbsp;accomodate&nbsp;all</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;reservations.</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;/</span><span style="color:gray;">summary</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">param</span><span style="color:gray;">&nbsp;name</span><span style="color:gray;">=</span><span style="color:gray;">&quot;</span><span style="color:#1f377f;">reservations</span><span style="color:gray;">&quot;</span><span style="color:gray;">&gt;</span><span style="color:green;">The&nbsp;reservations&nbsp;to&nbsp;accommodate</span><span style="color:gray;">&lt;/</span><span style="color:gray;">param</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">returns</span><span style="color:gray;">&gt;</span><span style="color:green;">A&nbsp;generator&nbsp;of&nbsp;valid&nbsp;table&nbsp;configurations.</span><span style="color:gray;">&lt;/</span><span style="color:gray;">returns</span><span style="color:gray;">&gt;</span> <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&nbsp;<span style="color:#74531f;">GenTables</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Create&nbsp;a&nbsp;table&nbsp;for&nbsp;each&nbsp;reservation,&nbsp;to&nbsp;ensure&nbsp;that&nbsp;all</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;reservations&nbsp;can&nbsp;be&nbsp;allotted&nbsp;a&nbsp;table.</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">tables</span>&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Standard</span>(<span style="color:#1f377f;">r</span>.Quantity)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;moreTables&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Choose</span>(1,&nbsp;12).<span style="color:#74531f;">Select</span>(<span style="color:#2b91af;">Table</span>.<span style="color:#74531f;">Standard</span>).<span style="color:#74531f;">ArrayOf</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;allTables&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Shuffle</span>(<span style="color:#1f377f;">tables</span>.<span style="color:#74531f;">Concat</span>(moreTables)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;allTables.<span style="color:#74531f;">AsEnumerable</span>(); }</pre> </p> <p> This function first creates standard tables that exactly accommodate each reservation. It then generates an array of <code>moreTables</code>, each fitting between one and twelve people. It then mixes those tables together with the ones that fit a reservation and returns the sequence. Since <code>moreTables</code> can be empty, it's possible that the entire sequence of tables only just accommodates the <code>reservations</code>. </p> <p> I then modified <code>GenMaitreD</code> to use <code>GenTables</code>: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">MaitreD</span>&gt;&nbsp;<span style="color:#74531f;">GenMaitreD</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;seatingDuration&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Choose</span>(1,&nbsp;6) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;tables&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#74531f;">GenTables</span>(<span style="color:#1f377f;">reservations</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromHours</span>(seatingDuration), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;tables); }</pre> </p> <p> This provoked a change in the SUT: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;r&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">orderby</span>&nbsp;g.Key &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:#74531f;">Allocate</span>(g).<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> The <code>Schedule</code> method now calls a private helper method called <code>Allocate</code>. This method already existed, since it supports the algorithm used to decide whether or not to accept a reservation request. </p> <h3 id="79e43322721b45a390c68caeaa1a3dcf"> Rinse and repeat <a href="#79e43322721b45a390c68caeaa1a3dcf" title="permalink">#</a> </h3> <p> I hope that a pattern starts to emerge. I kept adding more and more randomisation to the data generators, while I also added more and more assertions to the property. Here's what it looked like after a few more iterations: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ScheduleImp</span>(<span style="color:#2b91af;">MaitreD</span>&nbsp;<span style="color:#1f377f;">sut</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>[]&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">actual</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sut</span>.<span style="color:#74531f;">Schedule</span>(<span style="color:#1f377f;">reservations</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">r</span>.At).<span style="color:#74531f;">Distinct</span>().<span style="color:#74531f;">Count</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Count</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At).<span style="color:#74531f;">OrderBy</span>(<span style="color:#1f377f;">d</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">d</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>.<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">o</span>.At)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">All</span>(<span style="color:#1f377f;">actual</span>,&nbsp;<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">AssertTables</span>(<span style="color:#1f377f;">sut</span>.Tables,&nbsp;<span style="color:#1f377f;">o</span>.Value)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">All</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">actual</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">o</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">AssertRelevance</span>(<span style="color:#1f377f;">reservations</span>,&nbsp;<span style="color:#1f377f;">sut</span>.SeatingDuration,&nbsp;<span style="color:#1f377f;">o</span>)); }</pre> </p> <p> While <code>AssertTables</code> didn't change further, I added another helper assertion called <code>AssertRelevance</code>. I'm not going to show it here, but it checks that each occurrence only contains reservations that overlaps that point in time, give or take the <code>SeatingDuration</code>. </p> <p> I also made the reservation generator more sophisticated. If you consider the one defined above, one flaw is that it generates reservations at random dates. The chance that it'll generate two reservations that are actually adjacent in time is minimal. To counter this problem, I added a function that would return a generator of adjacent reservations: </p> <p> <pre><span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">summary</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;Generate&nbsp;an&nbsp;adjacant&nbsp;reservation&nbsp;with&nbsp;a&nbsp;25%&nbsp;chance.</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;/</span><span style="color:gray;">summary</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">param</span><span style="color:gray;">&nbsp;name</span><span style="color:gray;">=</span><span style="color:gray;">&quot;</span><span style="color:#1f377f;">reservation</span><span style="color:gray;">&quot;</span><span style="color:gray;">&gt;</span><span style="color:green;">The&nbsp;candidate&nbsp;reservation</span><span style="color:gray;">&lt;/</span><span style="color:gray;">param</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">returns</span><span style="color:gray;">&gt;</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;A&nbsp;generator&nbsp;of&nbsp;an&nbsp;array&nbsp;of&nbsp;reservations.&nbsp;The&nbsp;generated&nbsp;array&nbsp;is</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;either&nbsp;a&nbsp;singleton&nbsp;or&nbsp;a&nbsp;pair.&nbsp;In&nbsp;75%&nbsp;of&nbsp;the&nbsp;cases,&nbsp;the&nbsp;input</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;</span><span style="color:gray;">paramref</span><span style="color:gray;">&nbsp;name</span><span style="color:gray;">=</span><span style="color:gray;">&quot;</span><span style="color:#1f377f;">reservation</span><span style="color:gray;">&quot;</span><span style="color:gray;">&nbsp;/&gt;</span><span style="color:green;">&nbsp;is&nbsp;returned&nbsp;as&nbsp;a&nbsp;singleton&nbsp;array.</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;In&nbsp;25%&nbsp;of&nbsp;the&nbsp;cases,&nbsp;the&nbsp;array&nbsp;contains&nbsp;two&nbsp;reservations:&nbsp;the&nbsp;input</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;reservation&nbsp;as&nbsp;well&nbsp;as&nbsp;another&nbsp;reservation&nbsp;adjacent&nbsp;to&nbsp;it.</span> <span style="color:gray;">///</span><span style="color:green;">&nbsp;</span><span style="color:gray;">&lt;/</span><span style="color:gray;">returns</span><span style="color:gray;">&gt;</span> <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Reservation</span>[]&gt;&nbsp;<span style="color:#74531f;">GenAdjacentReservations</span>(<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="color:#1f377f;">reservation</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;adjacent&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#74531f;">GenReservationAdjacentTo</span>(<span style="color:#1f377f;">reservation</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;useAdjacent&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Frequency</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">WeightAndValue</span>&lt;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:blue;">bool</span>&gt;&gt;(3,&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Constant</span>(<span style="color:blue;">false</span>)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">WeightAndValue</span>&lt;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:blue;">bool</span>&gt;&gt;(1,&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Constant</span>(<span style="color:blue;">true</span>))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;rs&nbsp;=&nbsp;useAdjacent&nbsp;? &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#1f377f;">reservation</span>,&nbsp;adjacent&nbsp;}&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:#1f377f;">reservation</span>&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;rs; } <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#74531f;">GenReservationAdjacentTo</span>(<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="color:#1f377f;">reservation</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;minutes&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Choose</span>(-6&nbsp;*&nbsp;4,&nbsp;6&nbsp;*&nbsp;4)&nbsp;<span style="color:green;">//&nbsp;4:&nbsp;quarters/h</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;GenReservation &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;r.<span style="color:#74531f;">WithDate</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">reservation</span>.At&nbsp;<span style="color:#74531f;">+</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>.<span style="color:#74531f;">FromMinutes</span>(minutes)); }</pre> </p> <p> Now that I look at it again, I wonder whether I could have expressed this in a simpler way... It gets the job done, though. </p> <p> I then defined a generator that would either create entirely random reservations, or some with some adjacent ones mixed in: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Gen</span>&lt;<span style="color:#2b91af;">Reservation</span>[]&gt;&nbsp;GenReservations { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">normalArrayGen</span>&nbsp;=&nbsp;GenReservation.<span style="color:#74531f;">ArrayOf</span>(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">adjacentReservationsGen</span>&nbsp;=&nbsp;GenReservation.<span style="color:#74531f;">ArrayOf</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">SelectMany</span>(<span style="color:#1f377f;">rs</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Gen</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">Sequence</span>(<span style="color:#1f377f;">rs</span>.<span style="color:#74531f;">Select</span>(<span style="color:#74531f;">GenAdjacentReservations</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">SelectMany</span>(<span style="color:#1f377f;">rss</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">Shuffle</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">rss</span>.<span style="color:#74531f;">SelectMany</span>(<span style="color:#1f377f;">rs</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">rs</span>)))); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">Gen</span>.<span style="color:#74531f;">OneOf</span>(<span style="color:#1f377f;">normalArrayGen</span>,&nbsp;<span style="color:#1f377f;">adjacentReservationsGen</span>); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> I changed the property to use this generator instead: </p> <p> <pre>[<span style="color:#2b91af;">Property</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Property</span>&nbsp;<span style="color:#74531f;">Schedule</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:#2b91af;">Prop</span>.<span style="color:#74531f;">ForAll</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GenReservations &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">SelectMany</span>(<span style="color:#1f377f;">rs</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">GenMaitreD</span>(<span style="color:#1f377f;">rs</span>).<span style="color:#74531f;">Select</span>(<span style="color:#1f377f;">m</span>&nbsp;=&gt;&nbsp;(<span style="color:#1f377f;">m</span>,&nbsp;<span style="color:#1f377f;">rs</span>))) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.<span style="color:#74531f;">ToArbitrary</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">t</span>&nbsp;=&gt;&nbsp;<span style="color:#74531f;">ScheduleImp</span>(<span style="color:#1f377f;">t</span>.m,&nbsp;<span style="color:#1f377f;">t</span>.rs)); }</pre> </p> <p> I could have kept at it longer, but this turned out to be good enough to bring about the change in the SUT that I was looking for. </p> <h3 id="f34220b7400d464c9fce7c2b65454971"> Implementation <a href="#f34220b7400d464c9fce7c2b65454971" title="permalink">#</a> </h3> <p> These incremental changes iteratively brought me closer and closer to an implementation that I think has the correct behaviour: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Occurrence</span>&lt;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&gt;&gt;&nbsp;<span style="color:#74531f;">Schedule</span>(<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;<span style="color:#1f377f;">reservations</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">from</span>&nbsp;r&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#1f377f;">reservations</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">group</span>&nbsp;r&nbsp;<span style="color:blue;">by</span>&nbsp;r.At&nbsp;<span style="color:blue;">into</span>&nbsp;g &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">orderby</span>&nbsp;g.Key &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;seating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;g.Key) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;overlapping&nbsp;=&nbsp;<span style="color:#1f377f;">reservations</span>.<span style="color:#74531f;">Where</span>(seating.<span style="color:#74531f;">Overlaps</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">select</span>&nbsp;<span style="color:#74531f;">Allocate</span>(overlapping).<span style="color:#74531f;">At</span>(g.Key); }</pre> </p> <p> Contrary to my initial expectations, I managed to keep the implementation to a single query expression all the way through. </p> <h3 id="c64408017700494a808fae9388341e87"> Conclusion <a href="#c64408017700494a808fae9388341e87" title="permalink">#</a> </h3> <p> This was a problem that I was stuck on for a couple of days. I could describe the properties I wanted the function to have, but I had a hard time coming up with a good set of examples for unit tests. </p> <p> You may think that using property-based testing looks even more complicated, and I admit that it's far from trivial. The problem itself, however, isn't easy, and while the property-based approach may look daunting, it turned an intractable problem into a manageable one. That's a win in my book. </p> <p> It's also worth noting that this would all have looked more elegant in F#. There's an object-oriented tax to be paid when using FsCheck from C#. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Accountability and free speech https://blog.ploeh.dk/2021/02/09/accountability-and-free-speech 2021-02-09T07:53:00+00:00 Mark Seemann <div id="post"> <p> <em>A most likely naive suggestion.</em> </p> <p> A few years ago, my mother (born 1940) went to Paris on vacation with her older sister. She was a little concerned if she'd be able to navigate the <a href="https://en.wikipedia.org/wiki/Paris_M%C3%A9tro">Métro</a>, so she asked me to print out all sorts of maps in advance. I wasn't keen on another encounter with my nemesis, the printer, so I instead showed her how she could use Google Maps to get on-demand routes. Google Maps now include timetables and line information for many metropolitan subway lines around the world. I've successfully used it to find my way around London, Paris, New York, Tokyo, Sydney, and Melbourne. </p> <p> It even works in my backwater home-town: </p> <p> <img src="/content/binary/cph-metro-map.png" alt="Snapshot of Copenhagen Metro route generated by Google Maps."> </p> <p> It's rapidly turning into indispensable digital infrastructure, and that's beginning to trouble me. </p> <p> You can still get paper maps of the various rapid transit systems around the world, but for how long? </p> <h3 id="e5296cbd33f04db3b79890c09d8e3146"> Digital infrastructure <a href="#e5296cbd33f04db3b79890c09d8e3146" title="permalink">#</a> </h3> <p> If you're old enough, you may remember <a href="https://en.wikipedia.org/wiki/Telephone_directory">phone books</a>. In the days of land lines and analogue phones, you'd look up a number and then dial it. As the internet became increasingly ubiquitous, phone directories went online as well. Paper phone books were seen as waste, and ultimately disappeared from everyday life. </p> <p> You can think of paper phone books as a piece of infrastructure that's now gone. I don't miss them, but it's worth reflecting on what impact their disappearing has. Today, if I need to find a phone number, Google is often the place I go. The physical infrastructure is gone, replaced by a digital infrastructure. </p> <p> Google, in particular, now provides important infrastructure for modern society. Not only web search, but also maps, video sharing, translation, emails, and much more. </p> <p> Other companies offer other services that verge on being infrastructure. Facebook is more than just updates to friends and families. Many organisations, including schools and universities, coordinate activities via Facebook, and the political discourse increasingly happens there and on Twitter. </p> <p> We've come to rely on these 'free' services to a degree that resembles our reliance on physical infrastructure like the power grid, running water, roads, telephones, etcetera. </p> <h3 id="bf364e0f0ca84fbbb0536b462743993b"> TANSTAAFL <a href="#bf364e0f0ca84fbbb0536b462743993b" title="permalink">#</a> </h3> <p> <a href="https://en.wikipedia.org/wiki/Robert_A._Heinlein">Robert A. Heinlein</a> coined the phrase <em>There ain't no such thing as a free lunch</em> (TANSTAAFL) in his excellent book <a href="https://amzn.to/3pWi7aM">The Moon is a Harsh Mistress</a>. Indeed, the digital infrastructure isn't free. </p> <p> Recent events have magnified some of the cost we're paying. In January, Facebook and Twitter suspended the then-president of the United States from the platforms. I've little sympathy for Donald Trump, who strikes me as an uncouth narcissist, but since I'm a Danish citizen living in Denmark, I don't think that I ought to have much of an opinion about American politics. </p> <p> I do think, on the other hand, that the suspension sets an uncomfortable precedent. </p> <p> Should we let a handful of tech billionaires control essential infrastructure? This time, the victim was someone half the world was glad to see go, but who's next? Might these companies suspend the accounts of politicians who work against them? </p> <p> Never in the history of the world has decision power over so many people been so concentrated. I'll remind my readers that Facebook, Twitter, Google, etcetera are in worldwide use. The decisions of a handful of majority shareholders can now affect billions of people. </p> <p> If you're suspended from one of these platforms, you may lose your ability to participate in school activities, or from finding your way through a foreign city, or from taking part of the democratic discussion. </p> <h3 id="23e71b25789c465e9530a51a1cceac21"> The case for regulation <a href="#23e71b25789c465e9530a51a1cceac21" title="permalink">#</a> </h3> <p> In this article, I mainly want to focus on the free-speech issue related mostly to Facebook and Twitter. These companies make money from ads. The longer you stay on the platform, the more ads they can show you, and they've discovered that nothing pulls you in like anger. These incentives are incredibly counter-productive for society. </p> <p> Users are implicitly encouraged to create or spread anger, yet few are held accountable. One reason is that you can easily create new accounts without disclosing your real-world identity. This makes it difficult to hold users accountable for their utterances. </p> <p> Instead of clear rules, users are suspended for inexplicable and arbitrary reasons. It seems that these are mostly the verdicts of algorithms, but as we've seen in the case of Donald Trump, it can also be the result of an undemocratic, but political decision. </p> <p> Everyone can lose their opportunities for self-expression for arbitrary reasons. This is a free-speech issue. </p> <p> Yes, free speech. </p> <p> I'm well aware that Facebook and Twitter are private companies, and no-one has any <em>right</em> to an account on those platforms. That's why I started this essay by discussing how these services are turning into infrastructure. </p> <p> More than a century ago, the telephone was new technology operated by private companies. I know the Danish history best, but it serves well as an example. <a href="https://da.wikipedia.org/wiki/KTAS">KTAS</a>, one of the world's first telephone companies, was a private company. Via mergers and acquisitions, it still is, but as it grew and became the backbone of the country's electronic communications network, the government introduced legislation. For decades, it and its sister companies were regional monopolies. </p> <p> The government allowed the monopoly in exchange for various obligations. Even when the monopoly ended in 1996, the original monopoly companies inherited these legal obligations. For instance, every Danish citizen has the right to get a land-line installed, even if that installation by itself is a commercial loss for the company. This could be the case on certain remote islands or other rural areas. The Danish state compensates the telephone operator for this potential loss. </p> <p> Many other utility companies run in a similar fashion. Some are semi-public, some are private, but common to water, electricity, garbage disposal, heating, and other operators is that they are regulated by government. </p> <p> When a utility becomes important to the functioning of society, a sensible government steps in to regulate it to ensure the further smooth functioning of society. </p> <p> I think that the services offered by Google, Facebook, and Twitter are fast approaching a level of significance that ought to trigger government regulation. </p> <p> I don't say that lightly. I'm actually quite libertarian in my views, but I'm no anarcho-libertarian. I do acknowledge that a state offers essential services such as protection of property rights, a judicial system, and so on. </p> <h3 id="f4c78c79e1eb46d2941611d443f89ea1"> Accountability <a href="#f4c78c79e1eb46d2941611d443f89ea1" title="permalink">#</a> </h3> <p> How should we regulate social media? I think we should start by exchanging accountability for the right to post. </p> <p> Let's take another step back for a moment. For generations, it's been possible to send a letter to the editor of a regular newspaper. If the editor deems it worthy for publication, it'll be printed in the next issue. You don't have any right to get your letter published; this happens at the discretion of the editor. </p> <p> Why does it work that way? It works that way because the editor is accountable for what's printed in the paper. Ultimately, he or she can go to jail for what's printed. </p> <p> Freedom of speech is more complicated than it looks at first glance. For instance, censorship in Denmark was abolished with the 1849 constitution. This doesn't mean that you can freely say whatever you'd like; it only means that government has no right to <em>prevent</em> you from saying or writing something. You can still be prosecuted after the fact if you say or write something libellous, or if you incite violence, or if you disclose secrets that you've agreed to keep, etcetera. </p> <p> This is accountability. It works when the person making the utterance is known and within reach of the law. </p> <p> Notice, particularly, that an editor-in-chief is accountable for a newspaper's contents. Why isn't Facebook or Twitter accountable for content? </p> <p> These companies have managed to spin a story that they're <em>platforms</em> rather than publishers. This argument might have had some legs ten years ago. When I started using Twitter in 2009, the algorithm was easy to understand: My feed showed the tweets from the accounts that I followed, in the order that they were published. This was easy to understand, and Twitter didn't, as far as I could tell, edit the feed. Since no editing took place, I find the platform argument applicable. </p> <p> Later, the social networks began editing the feeds. No humans were directly involved, but today, some posts are amplified while others get little attention. Since this is done by 'algorithms' rather than editors, the companies have managed to convince lawmakers that they're still only platforms. Let's be real, though. Apparently, the public needs a programmer to say the following, and I'm a programmer. </p> <p> Programmers, employees of those companies, wrote the algorithms. They experimented and tweaked those algorithms to maximise 'engagement'. Humans are continually involved in this process. Editing takes place. </p> <p> I think it's time to stop treating social media networks as platforms, and start treating them as publishers. </p> <h3 id="ab23d2f377db45bf99970479238dd982"> Practicality <a href="#ab23d2f377db45bf99970479238dd982" title="permalink">#</a> </h3> <p> Facebook and Twitter will protest that it's practically impossible for them to 'edit' what happens on the networks. I find this argument unconvincing, since editing already takes place. </p> <p> I'd like to suggest a partial way out, though. This brings us back to regulation. </p> <p> I think that each country or jurisdiction should make it possible for users to opt in to being accountable. Users should be able to verify their accounts as real, legal persons. If they do that, their activity on the platform should be governed by the jurisdiction in which they reside, instead of by the arbitrary and ill-defined 'community guidelines' offered by the networks. You'd be accountable for your utterances according to local law where you live. The advantage, though, is that this accountability buys you the <em>right</em> to be present on the platform. You can be sued for your posts, but you can't be kicked off. </p> <p> I'm aware that not everyone lives in a benign welfare state such as Denmark. This is why I suggest this as an option. Even if you live in a state that regulates social media as outlined above, the option to stay anonymous should remain. This is how it already works, and I imagine that it should continue to work like that for this group of users. The cost of staying anonymous, though, is that you submit to the arbitrary and despotic rules of those networks. For many people, including minorities and citizens of oppressive states, this is likely to remain a better trade-off. </p> <h3 id="84eccfad9fe649c3b38d9e4d6ddb2698"> Problems <a href="#84eccfad9fe649c3b38d9e4d6ddb2698" title="permalink">#</a> </h3> <p> This suggestion is in no way perfect. I can already identify one problem with it. </p> <p> A country could grant its citizens the right to conduct infowar on an adversary; think troll armies and the like. If these citizens are verified and 'accountable' to their local government, but this government encourages rather than punishes incitement to violence in a foreign country, then how do we prevent that? </p> <p> I have a few half-baked ideas, but I'd rather leave the problem here in the hope that it might inspire other people to start thinking about it, too. </p> <h3 id="f25f8486eed54d3d89fd607ef41e3ac7"> Conclusion <a href="#f25f8486eed54d3d89fd607ef41e3ac7" title="permalink">#</a> </h3> <p> This is a post that I've waited for a long time for someone else to write. If someone already did, I'm not aware of it. I can think of three reasons: <ul> <li>It's a really stupid idea.</li> <li>It's a common idea, but no-one talks about it because of <a href="https://en.wikipedia.org/wiki/Pluralistic_ignorance">pluralistic ignorance</a>.</li> <li>It's an original idea that no-one else have had.</li> </ul> I don't believe much in the last option, and I'm afraid that the first is the most likely. I've no desire to look stupid, but on the other hand, I can't help keep thinking about this. </p> <p> The idea is, in short, to make it optional for users to 'buy' the right to stay on a social network for the price of being held legally accountable. This requires some national as well as international regulation of the digital infrastructure. </p> <p> Could it work? I don't know, but isn't it worth discussing? </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="c9bf9249f05342dcae02d9581b7e7e5c"> <div class="comment-author"><a href="https://jeremiahflaga.github.io/">Jboy Flaga</a></div> <div class="comment-content"> <p> Hi Mark, I would just like to say that I like the idea :) </p> <p> Thank you for writing this. </p> </div> <div class="comment-date">2021-02-18 13:34 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. ASP.NET POCO Controllers: an experience report https://blog.ploeh.dk/2021/02/01/aspnet-poco-controllers-an-experience-report 2021-02-01T08:10:00+00:00 Mark Seemann <div id="post"> <p> <em>Controllers don't have to derive from a base class.</em> </p> <p> In most tutorials about ASP.NET web APIs you'll be told to let Controller classes derive from a base class. It may be convenient if you believe that productivity is measured by how fast you can get an initial version of the software into production. Granted, sometimes that's the case, but usually there's a price to be paid. Did you produce legacy code in the process? </p> <p> One <a href="http://bit.ly/working-effectively-with-legacy-code">common definition of legacy code</a> is that it's code without tests. With ASP.NET I've repeatedly found that when Controllers derive from <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase">ControllerBase</a> they become harder to unit test. It may be convenient to have access to the <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.url#Microsoft_AspNetCore_Mvc_ControllerBase_Url">Url</a> and <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.user#Microsoft_AspNetCore_Mvc_ControllerBase_User">User</a> properties, but this tends to make the <em>arrange</em> phase of a unit test much more complex. Not impossible; just more complex than I like. In short, inheriting from <code>ControllerBase</code> just to get access to, say, <code>Url</code> or <code>User</code> violates the <a href="https://en.wikipedia.org/wiki/Interface_segregation_principle">Interface Segregation Principle</a>. That <code>ControllerBase</code> is too big a dependency for my taste. </p> <p> I already use <a href="/2021/01/25/self-hosted-integration-tests-in-aspnet">self-hosting in integration tests</a> so that I can interact with my REST APIs via HTTP. When I want to test how my API reacts to various HTTP-specific circumstances, I do that via integration tests. So, in a recent code base I decided to see if I could write an entire REST API in ASP.NET Core without inheriting from <code>ControllerBase</code>. </p> <p> The short answer is that, yes, this is possible, and I'd do it again, but you have to jump through some hoops. I consider that hoop-jumping a fine price to pay for the benefits of simpler unit tests and (it turns out) better separation of concerns. </p> <p> In this article, I'll share what I've learned. </p> <h3 id="fa68860febe6482287eadc65cf3e8a46"> POCO Controllers <a href="#fa68860febe6482287eadc65cf3e8a46" title="permalink">#</a> </h3> <p> Just so that we're on the same page: A <a href="https://en.wikipedia.org/wiki/Plain_old_CLR_object">POCO</a> Controller is a Controller class that doesn't inherit from any base class. In my code base, they're defined like this: </p> <p> <pre>[<span style="color:#2b91af;">Route</span>(<span style="color:#a31515;">&quot;&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">HomeController</span></pre> </p> <p> or </p> <p> <pre>[<span style="color:#2b91af;">Authorize</span>(Roles&nbsp;=&nbsp;<span style="color:#a31515;">&quot;MaitreD&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ScheduleController</span></pre> </p> <p> As you can tell, they don't inherit from <code>ControllerBase</code> or any other base class. They <em>are</em> annotated with attributes, like <code>[Route]</code>, <code>[Authorize]</code>, or <code>[ApiController]</code>. Strictly speaking, that may disqualify them as true POCOs, but in practice I've found that those attributes don't impact the sustainability of the code base in a negative manner. </p> <p> Dependency Injection is still possible, and in use: </p> <p> <pre>[<span style="color:#2b91af;">ApiController</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IClock</span>&nbsp;clock, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IRestaurantDatabase</span>&nbsp;restaurantDatabase, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;repository) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Clock&nbsp;=&nbsp;clock; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;RestaurantDatabase&nbsp;=&nbsp;restaurantDatabase; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Repository&nbsp;=&nbsp;repository; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IClock</span>&nbsp;Clock&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IRestaurantDatabase</span>&nbsp;RestaurantDatabase&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;Repository&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Controller&nbsp;actions&nbsp;and&nbsp;other&nbsp;members&nbsp;go&nbsp;here...</span></pre> </p> <p> I consider Dependency Injection nothing but an application of polymorphism, so I think that still fits the POCO label. After all, <a href="/2017/01/27/dependency-injection-is-passing-an-argument">Dependency Injection is just argument-passing</a>. </p> <h3 id="b1672f04c9774dca9f221a301c191d95"> HTTP responses and status codes <a href="#b1672f04c9774dca9f221a301c191d95" title="permalink">#</a> </h3> <p> The first thing I had to figure out was how to return various HTTP status codes. If you just return some model object, the default status code is <code>200 OK</code>. That's fine in many situations, but if you're implementing a proper REST API, you should be using headers and status codes to communicate with the client. </p> <p> The <code>ControllerBase</code> class defines many helper methods to return various other types of responses, such as <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.badrequest#Microsoft_AspNetCore_Mvc_ControllerBase_BadRequest">BadRequest</a> or <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.notfound#Microsoft_AspNetCore_Mvc_ControllerBase_NotFound">NotFound</a>. I had to figure out how to return such responses without access to the base class. </p> <p> Fortunately, ASP.NET Core is now open source, so it isn't too hard to look at the source code for those helper methods to see what they do. It turns out that they are just discoverable methods that create and return various result objects. So, instead of calling the <code>BadRequest</code> helper method, I can just return a <code>BadRequestResult</code>: </p> <p> <pre><span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>();</pre> </p> <p> Some of the responses were a little more involved, so I created my own domain-specific helper methods for them: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;Reservation201Created(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;r) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CreatedAtActionResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;actionName:&nbsp;<span style="color:blue;">nameof</span>(Get), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controllerName:&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;routeValues:&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;restaurantId,&nbsp;id&nbsp;=&nbsp;r.Id.ToString(<span style="color:#a31515;">&quot;N&quot;</span>)&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;value:&nbsp;r.ToDto()); }</pre> </p> <p> Apparently, the <code>controllerName</code> argument isn't required, so should be <code>null</code>. I had to experiment with various combinations to get it right, but I have self-hosted integration tests that cover this result. If it doesn't work as intended, tests will fail. </p> <p> Returning this <code>CreatedAtActionResult</code> object produces an HTTP response like this: </p> <p> <pre>HTTP/1.1 201 Created Content-Type: application/json; charset=utf-8 Location: https://example.net:443/restaurants/2112/reservations/276d124f20cf4cc3b502f57b89433f80 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;276d124f20cf4cc3b502f57b89433f80&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2022-01-14T19:45:00.0000000&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;donkeyman@example.org&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Don&nbsp;Keyman&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;4 }</pre> </p> <p> I've edited and coloured the response for readability. The <code>Location</code> URL actually also <a href="/2020/10/26/fit-urls">includes a digital signature</a>, which I've removed here just to make the example look a little prettier. </p> <p> In any case, returning various HTTP headers and status codes is less discoverable when you don't have a base class with all the helper methods, but once you've figured out the objects to return, it's straightforward, and the code is simple. </p> <h3 id="ecbc913d450d49b7ab92d2e0c193230e"> Generating links <a href="#ecbc913d450d49b7ab92d2e0c193230e" title="permalink">#</a> </h3> <p> A true (<a href="https://martinfowler.com/articles/richardsonMaturityModel.html">level 3</a>) REST API uses <a href="https://en.wikipedia.org/wiki/HATEOAS">hypermedia as the engine of application state</a>; that is, <em>links</em>. This means that a HTTP response will typically include a JSON or XML representation with several links. <a href="/2020/10/26/fit-urls">This article</a> includes several examples. </p> <p> ASP.NET provides <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.iurlhelper">IUrlHelper</a> for exactly that purpose, and if you inherit from <code>ControllerBase</code> the above-mentioned <code>Url</code> property gives you convenient access to just such an object. </p> <p> When a class <em>doesn't</em> inherit from <code>ControllerBase</code>, you don't have convenient access to an <code>IUrlHelper</code>. Then what? </p> <p> It's possible to get an <code>IUrlHelper</code> via the framework's built-in Dependency Injection engine, but if you add such a dependency to a POCO Controller, you'll have to jump through all sorts of hoops to configure it in unit tests. That was exactly the situation I wanted to avoid, so that got me thinking about design alternatives. </p> <p> That was the real, underlying reason I came up with the idea to instead <a href="/2020/08/24/adding-rest-links-as-a-cross-cutting-concern">add REST links as a cross-cutting concern</a>. The Controllers are wonderfully free of that concern, which helps keeping the complexity down of the unit tests. </p> <p> I still have test coverage of the links, but I prefer testing HTTP-related behaviour via the HTTP API instead of relying on implementation details: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;Nono&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;The&nbsp;Vatican&nbsp;Cellar&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;RestaurantReturnsCorrectLinks(<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;client&nbsp;=&nbsp;api.CreateClient(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;response&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;client.GetRestaurant(name); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HashSet</span>&lt;<span style="color:blue;">string</span>?&gt;(<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;response.ParseJsonContent&lt;<span style="color:#2b91af;">RestaurantDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actualRels&nbsp;=&nbsp;actual.Links.Select(l&nbsp;=&gt;&nbsp;l.Rel).ToHashSet(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Superset(expected,&nbsp;actualRels); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.All(actual.Links,&nbsp;AssertHrefAbsoluteUrl); }</pre> </p> <p> This test verifies that the representation returned in <code>response</code> contains four labelled links. Granted, this particular test only verifies that each link contains an absolute URL (which could, in theory, be <code>http://www.example.com</code>), but the whole point of <a href="/2020/10/26/fit-urls">fit URLs</a> is that they should be opaque. I've other tests that follow links to verify that the API affords the desired behaviour. </p> <h3 id="cd0769917ec441f786877bba0e630f00"> Authorisation <a href="#cd0769917ec441f786877bba0e630f00" title="permalink">#</a> </h3> <p> The final kind of behaviour that caused me a bit of trouble was authorisation. It's easy enough to annotate a Controller with an <code>[Authorize]</code> attribute, which is fine as long all you need is role-based authorisation. </p> <p> I did, however, run into one situation where I needed something closer to an <a href="https://en.wikipedia.org/wiki/Access-control_list">access control list</a>. The system that includes all these code examples is a multi-tenant restaurant reservation system. There's one protected resource: a day's schedule, intended for use by the restaurant's staff. Here's a simplified example: </p> <p> <pre>GET /restaurants/2112/schedule/2021/2/23 HTTP/1.1 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInCI6IkpXVCJ9.eyJ... HTTP/1.1 200 OK Content-Type: application/json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;year&quot;</span>:&nbsp;2021, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;month&quot;</span>:&nbsp;2, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;day&quot;</span>:&nbsp;23, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;days&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2021-02-23&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;entries&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;reservations&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2c7ace4bbee94553950afd60a86c530c&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2021-02-23T19:45:00.0000000&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;anarchi@example.net&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Ann&nbsp;Archie&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> Since this resource contains personally identifiable information (email addresses) it's protected. You have to present a valid <a href="https://en.wikipedia.org/wiki/JSON_Web_Token">JSON Web Token</a> with required claims. Role claims, however, aren't enough. A minimum bar is that the token must contain a sufficient role claim like <code>"MaitreD"</code> shown above, but that's not enough. This is, after all, a multi-tenant system, and we don't want one restaurant's <em>MaitreD</em> to be able to see another restaurant's schedule. </p> <p> If ASP.NET can address that kind of problem with annotations, I haven't figured out how. The Controller needs to check an access control list against the resource being accessed. The above-mentioned <code>User</code> property of <code>ControllerBase</code> would be <em>really</em> convenient here. </p> <p> Again, there are ways to inject an entire <a href="https://docs.microsoft.com/dotnet/api/system.security.claims.claimsprincipal">ClaimsPrincipal</a> class into the Controller that needs it, but once more I felt that that would violate the Interface Segregation Principle. I didn't need an entire <code>ClaimsPrincipal</code>; I just needed a list of restaurant IDs that a particular JSON Web Token allows access to. </p> <p> As is often my modus operandi in such situations, I started by writing the code I wished to use, and then figured out how to make it work: </p> <p> <pre>[<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/schedule/{year}/{month}/{day}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month,&nbsp;<span style="color:blue;">int</span>&nbsp;day) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!AccessControlList.Authorize(restaurantId)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ForbidResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Do&nbsp;the&nbsp;real&nbsp;work&nbsp;here...</span></pre> </p> <p> <code>AccessControlList</code> is a <a href="/2012/08/31/ConcreteDependencies">Concrete Dependency</a>. It's just a wrapper around a collection of IDs: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">AccessControlList</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">IReadOnlyCollection</span>&lt;<span style="color:blue;">int</span>&gt;&nbsp;restaurantIds;</pre> </p> <p> I had to create a new class in order to keep the built-in ASP.NET DI Container happy. Had I been doing <a href="/2014/06/10/pure-di">Pure DI</a> I could have just injected <code>IReadOnlyCollection&lt;int&gt;</code> directly into the Controller, but in this code base I used the built-in DI Container, which I had to configure like this: </p> <p> <pre>services.AddHttpContextAccessor(); services.AddTransient(sp&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">AccessControlList</span>.FromUser( &nbsp;&nbsp;&nbsp;&nbsp;sp.GetService&lt;<span style="color:#2b91af;">IHttpContextAccessor</span>&gt;().HttpContext.User));</pre> </p> <p> Apart from having to wrap <code>IReadOnlyCollection&lt;int&gt;</code> in a new class, I found such an implementation preferable to inheriting from <code>ControllerBase</code>. The Controller in question only depends on the services it needs: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ScheduleController</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IRestaurantDatabase</span>&nbsp;restaurantDatabase, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IReservationsRepository</span>&nbsp;repository, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">AccessControlList</span>&nbsp;accessControlList)</pre> </p> <p> The <code>ScheduleController</code> uses its <code>restaurantDatabase</code> dependency to look up a specific restaurant based on the <code>restaurantId</code>, the <code>repository</code> to read the schedule, and <code>accessControlList</code> to implement authorisation. That's what it needs, so that's its dependencies. It follows the Interface Segregation Principle. </p> <p> The <code>ScheduleController</code> class is easy to unit test, since a test can just create a new <code>AccessControlList</code> object whenever it needs to: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;GetScheduleForAbsentRestaurant() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ScheduleController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Some</span>.Restaurant.WithId(2)), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">AccessControlList</span>(3)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Get(3,&nbsp;2089,&nbsp;12,&nbsp;9); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">NotFoundResult</span>&gt;(actual); }</pre> </p> <p> This test requests the schedule for a restaurant with the ID <code>3</code>, and the access control list <em>does</em> include that ID. The restaurant, however, doesn't exist, so despite correct permissions, the expected result is <code>404 Not Found</code>. </p> <h3 id="882c69dacd17476d899d3910a35aa62d"> Conclusion <a href="#882c69dacd17476d899d3910a35aa62d" title="permalink">#</a> </h3> <p> ASP.NET has supported POCO Controllers for some time now, but it's clearly not a mainstream scenario. The documentation and Visual Studio tooling assumes that your Controllers inherit from one of the framework base classes. </p> <p> You do, therefore, have to jump through a few hoops to make POCO Controllers work. The result, however, is more lightweight Controllers and better separation of concerns. I find the jumping worthwhile. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="2d2c4c2ddf6845bba2cfe55a513ac3f4"> <div class="comment-author"><a href="https://taeguk.co.uk">Dave Shaw</a></div> <div class="comment-content"> <p>Hi Mark, as I read the part about <code>BadRequest</code> and <code>NotFound</code> being less discoverable, I wondered if you had considered creating a <code>ControllerHelper</code> (terrible name <a href="https://blog.ploeh.dk/2020/11/23/good-names-are-skin-deep/">I know</a>) class with <code>BadRequest</code>, etc? <br /> Then by adding <code>using static Resteraunt.ControllerHelper</code> as a little bit of boilerplate code to the top of each POCO controller, you would be able to do <code>return BadRequest();</code> again. </p> </div> <div class="comment-date">2021-02-03 22:00 UTC</div> </div> <div class="comment" id="99c5531b27ff4dddb98847063b7e670e"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Dave, thank you for writing. I hadn't thought of that, but it's a useful idea. Someone would have to figure out how to write such a helper class, but once it exists, it would offer the same degree of discoverability. I'd suggest a name like <code>Result</code> or <code>HttpResult</code>, so that you could write e.g. <code>Result.BadRequest()</code>. </p> <p> Wouldn't adding a static import defy the purpose, though? As I see it, <a href="/2012/05/25/Designpatternsacrossparadigms#ebe4a8c5ba664c6fb5ea07c8b7e18555">C# discoverability is enabled by 'dot-driven development'</a>. Don't you lose that by a static import? </p> </div> <div class="comment-date">2021-02-04 7:18 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Self-hosted integration tests in ASP.NET https://blog.ploeh.dk/2021/01/25/self-hosted-integration-tests-in-aspnet 2021-01-25T07:45:00+00:00 Mark Seemann <div id="post"> <p> <em>A way to self-host a REST API and test it through HTTP.</em> </p> <p> In 2020 I developed a sizeable code base for an online restaurant REST API. In the spirit of <a href="/outside-in-tdd">outside-in TDD</a>, I found it best to test the HTTP behaviour of the API by actually interacting with it via HTTP. </p> <p> Sometimes ASP.NET offers more than one way to achieve the same end result. For example, to return <code>200 OK</code>, you can use both <code>OkObjectResult</code> and <code>ObjectResult</code>. I don't want my tests to be coupled to such implementation details, so by testing an API via HTTP instead of using the ASP.NET object model, I decouple the two. </p> <p> You can easily self-host an ASP.NET web API and test it using an <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpclient">HttpClient</a>. In this article, I'll show you how I went about it. </p> <h3 id="a070ceb1f9a84f2988aa7e4c59f38397"> Reserving a table <a href="#a070ceb1f9a84f2988aa7e4c59f38397" title="permalink">#</a> </h3> <p> In true outside-in fashion, I'll first show you the test. Then I'll break it down to show you how it works. </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;<span style="color:#74531f;">ReserveTableAtNono</span>() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">api</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">client</span>&nbsp;=&nbsp;<span style="color:#1f377f;">api</span>.<span style="color:#74531f;">CreateClient</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">at</span>&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Today.<span style="color:#74531f;">AddDays</span>(434).<span style="color:#74531f;">At</span>(20,&nbsp;15); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">dto</span>&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation.<span style="color:#74531f;">WithDate</span>(<span style="color:#1f377f;">at</span>).<span style="color:#74531f;">WithQuantity</span>(6).<span style="color:#74531f;">ToDto</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">response</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">PostReservation</span>(<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;<span style="color:#1f377f;">dto</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#74531f;">AssertRemainingCapacity</span>(<span style="color:#1f377f;">client</span>,&nbsp;<span style="color:#1f377f;">at</span>,&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;4); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#74531f;">AssertRemainingCapacity</span>(<span style="color:#1f377f;">client</span>,&nbsp;<span style="color:#1f377f;">at</span>,&nbsp;<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>,&nbsp;10); }</pre> </p> <p> This test uses <a href="https://xunit.net">xUnit.net</a> 2.4.1 to make a reservation at the restaurant named <em>Nono</em>. The first line that creates the <code>api</code> variable spins up a self-hosted instance of the REST API. The next line creates an <code>HttpClient</code> configured to communicate with the self-hosted instance. </p> <p> The test proceeds to create a Data Transfer Object that it posts to the <em>Nono</em> restaurant. It then asserts that the remaining capacity at the <em>Nono</em> and <em>Hipgnosta</em> restaurants are as expected. </p> <p> You'll see the implementation details soon, but I first want to discuss this high-level test. As is the case with most of my code, it's far from perfect. If you're not familiar with this code base, you may have plenty of questions: <ul> <li>Why does it make a reservation 434 days in the future? Why not 433, or 211, or 1?</li> <li>Is there anything significant about the quantity <em>6?</em></li> <li>Why is the expected remaining capacity at <em>Nono 4?</em></li> <li>Why is the expected remaining capacity at <em>Hipgnosta 10?</em> Why does it even verify that?</li> </ul> To answer the easiest question first: There's nothing special about 434 days. The only <a href="/2021/01/11/waiting-to-happen">requirement is that it's a positive number</a>, so that the reservation is in the future. That makes this test a great candidate for a <a href="/property-based-testing-intro">property-based test</a>. </p> <p> The three other questions are all related. A bit of background is in order. I wrote this test during a process where I turned the system into a multi-tenant system. Before that change, there was only one restaurant, which was <em>Hipgnosta</em>. I wanted to verify that if you make a reservation at another restaurant (here, <em>Nono</em>) it changes the observable state of that restaurant, and not of <em>Hipgnosta</em>. </p> <p> The way these two restaurants are configured, <em>Hipgnosta</em> has <a href="/2020/01/27/the-maitre-d-kata">a single communal table that seats ten guests</a>. This explains why the expected capacity of <em>Hipgnosta</em> is <em>10</em>. Making a reservation at <em>Nono</em> shouldn't affect <em>Hipgnosta</em>. </p> <p> <em>Nono</em> has a more complex table configuration. It has both standard and communal tables, but the largest table is a six-person communal table. There's only one table of that size. The next-largest tables are four-person tables. Thus, a reservation for six people reserves the largest table that day, after which only four-person and two-person tables are available. Therefore the remaining capacity ought to be <em>4</em>. </p> <p> The above test knows all this. You are welcome to criticise such hard-coded knowledge. There's a real risk that it might make it more difficult to maintain the test suite in the future. </p> <p> Certainly, had this been a unit test, and not an integration test, I wouldn't have accepted so much implicit knowledge - particularly because I mostly apply <a href="/2018/11/19/functional-architecture-a-definition">functional architecture</a>, and <a href="/2015/05/07/functional-design-is-intrinsically-testable">pure functions should have isolation</a>. Functions shouldn't depend on implicit global state; they should return a value based on input arguments. That's a bit of digression, though. </p> <p> These are integration tests, which I mostly use for smoke tests and to verify HTTP-specific behaviour. I have unit tests for fine-grained testing of edge cases and variations of input. While I wouldn't accept so much implicit knowledge from a unit test, I find that it so far works well with integration tests. </p> <h3 id="db1185165dc748aba67c8daa6278f523"> Self-hosting <a href="#db1185165dc748aba67c8daa6278f523" title="permalink">#</a> </h3> <p> It only takes a <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.testing.webapplicationfactory-1">WebApplicationFactory</a> to self-host an ASP.NET API. You can use it directly, but if you want to modify the hosted service in some way, you can also inherit from it. </p> <p> I want my self-hosted integration tests to run as <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based tests</a> that use an in-memory database instead of SQL Server. I've defined <code>SelfHostedApi</code> for that purpose: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>&nbsp;:&nbsp;<span style="color:#2b91af;">WebApplicationFactory</span>&lt;<span style="color:#2b91af;">Startup</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">protected</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">void</span>&nbsp;<span style="color:#74531f;">ConfigureWebHost</span>(<span style="color:#2b91af;">IWebHostBuilder</span>&nbsp;<span style="color:#1f377f;">builder</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">builder</span>.<span style="color:#74531f;">ConfigureServices</span>(<span style="color:#1f377f;">services</span>&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">services</span>.<span style="color:#74531f;">RemoveAll</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">services</span>.<span style="color:#74531f;">AddSingleton</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> The way that <code>WebApplicationFactory</code> works, its <code>ConfigureWebHost</code> method runs <em>after</em> the <code>Startup</code> class' <code>ConfigureServices</code> method. Thus, when <code>ConfigureWebHost</code> runs, the <code>services</code> collection is already configured to use SQL Server. As <a href="/2020/04/20/unit-bias-against-collections#e6675033a3a9dc8a21c64650dff91b8432a9a151">Julius H so kindly pointed out to me</a>, the <code>RemoveAll</code> extension method removes all existing registrations of a service. I use it to remove the SQL Server dependency from the system, after which I replace it with a test-specific in-memory implementation. </p> <p> Since the in-memory database is configured with Singleton lifetime, that instance is going to be around for the lifetime of the service. While it's only keeping track of things in memory, it'll keep state until the service shuts down, which happens when the above <code>api</code> variable goes out of scope. </p> <p> Notice that <a href="/2020/11/30/name-by-role">I named the class by the role it plays</a> rather than which base class it derives from. </p> <h3 id="af89655ac4d948f8a749aa20fffe2b12"> Posting a reservation <a href="#af89655ac4d948f8a749aa20fffe2b12" title="permalink">#</a> </h3> <p> The <code>PostReservation</code> method is an extension method on <code>HttpClient</code>: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">HttpResponseMessage</span>&gt;&nbsp;<span style="color:#74531f;">PostReservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">HttpClient</span>&nbsp;<span style="color:#1f377f;">client</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">name</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">object</span>&nbsp;<span style="color:#1f377f;">reservation</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">json</span>&nbsp;=&nbsp;<span style="color:#2b91af;">JsonSerializer</span>.<span style="color:#74531f;">Serialize</span>(<span style="color:#1f377f;">reservation</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">content</span>&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">StringContent</span>(<span style="color:#1f377f;">json</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">content</span>.Headers.ContentType.MediaType&nbsp;=&nbsp;<span style="color:#a31515;">&quot;application/json&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">resp</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">GetRestaurant</span>(<span style="color:#1f377f;">name</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">resp</span>.<span style="color:#74531f;">EnsureSuccessStatusCode</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">rest</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">resp</span>.<span style="color:#74531f;">ParseJsonContent</span>&lt;<span style="color:#2b91af;">RestaurantDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">address</span>&nbsp;=&nbsp;<span style="color:#1f377f;">rest</span>.Links.<span style="color:#74531f;">FindAddress</span>(<span style="color:#a31515;">&quot;urn:reservations&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">PostAsync</span>(<span style="color:#1f377f;">address</span>,&nbsp;<span style="color:#1f377f;">content</span>); }</pre> </p> <p> It's part of a larger set of methods that enables an <code>HttpClient</code> to interact with the REST API. Three of those methods are visible here: <code>GetRestaurant</code>, <code>ParseJsonContent</code>, and <code>FindAddress</code>. These, and many other, methods form a client API for interacting with the REST API. While this is currently test code, it's ripe for being extracted to a reusable client SDK library. </p> <p> I'm not going to show all of them, but here's <code>GetRestaurant</code> to give you a sense of what's going on: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">HttpResponseMessage</span>&gt;&nbsp;<span style="color:#74531f;">GetRestaurant</span>(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">HttpClient</span>&nbsp;<span style="color:#1f377f;">client</span>,&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">name</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">homeResponse</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">GetAsync</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(<span style="color:#a31515;">&quot;&quot;</span>,&nbsp;<span style="color:#2b91af;">UriKind</span>.Relative)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">homeResponse</span>.<span style="color:#74531f;">EnsureSuccessStatusCode</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">homeRepresentation</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">homeResponse</span>.<span style="color:#74531f;">ParseJsonContent</span>&lt;<span style="color:#2b91af;">HomeDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">restaurant</span>&nbsp;=&nbsp;<span style="color:#1f377f;">homeRepresentation</span>.Restaurants.<span style="color:#74531f;">First</span>(<span style="color:#1f377f;">r</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">r</span>.Name&nbsp;==&nbsp;<span style="color:#1f377f;">name</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">address</span>&nbsp;=&nbsp;<span style="color:#1f377f;">restaurant</span>.Links.<span style="color:#74531f;">FindAddress</span>(<span style="color:#a31515;">&quot;urn:restaurant&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">GetAsync</span>(<span style="color:#1f377f;">address</span>); }</pre> </p> <p> The REST API has only a single documented address, which is the 'home' resource at the relative URL <code>""</code>; i.e. the root of the API. In this incarnation of the API, the home resource responds with a JSON array of restaurants. The <code>GetRestaurant</code> method finds the restaurant with the desired name and finds its address. It then issues another <code>GET</code> request against that address, and returns the response. </p> <h3 id="00d69af4ad6b43538b86f3e27fb7c4ee"> Verifying state <a href="#00d69af4ad6b43538b86f3e27fb7c4ee" title="permalink">#</a> </h3> <p> The verification phase of the above test calls a private helper method: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;<span style="color:#74531f;">AssertRemainingCapacity</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">HttpClient</span>&nbsp;<span style="color:#1f377f;">client</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;<span style="color:#1f377f;">date</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;<span style="color:#1f377f;">name</span>, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">expected</span>) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">response</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">client</span>.<span style="color:#74531f;">GetDay</span>(<span style="color:#1f377f;">name</span>,&nbsp;<span style="color:#1f377f;">date</span>.Year,&nbsp;<span style="color:#1f377f;">date</span>.Month,&nbsp;<span style="color:#1f377f;">date</span>.Day); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">day</span>&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#1f377f;">response</span>.<span style="color:#74531f;">ParseJsonContent</span>&lt;<span style="color:#2b91af;">CalendarDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">All</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">day</span>.Days.<span style="color:#74531f;">Single</span>().Entries, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">e</span>&nbsp;=&gt;&nbsp;<span style="color:#2b91af;">Assert</span>.<span style="color:#74531f;">Equal</span>(<span style="color:#1f377f;">expected</span>,&nbsp;<span style="color:#1f377f;">e</span>.MaximumPartySize)); }</pre> </p> <p> It uses another of the above-mentioned client API extension methods, <code>GetDay</code>, to inspect the REST API's calendar entry for the restaurant and day in question. Each calendar contains a series of time entries that lists the largest party size the restaurant can accept at that time slot. The two restaurants in question only have single seatings, so once you've booked a six-person table, you have it for the entire evening. </p> <p> Notice that verification is done by interacting with the system itself. No <a href="http://xunitpatterns.com/Back%20Door%20Manipulation.html">Back Door Manipulation</a> is required. I favour this if at all possible, since I believe that it offers better confidence that the system behaves as it should. </p> <h3 id="e93c5eb19e1d4f358197b4a1c048f01e"> Conclusion <a href="#e93c5eb19e1d4f358197b4a1c048f01e" title="permalink">#</a> </h3> <p> It's been possible to self-host .NET REST APIs for testing purposes at least since 2012, but it's only become easier over the years. All you need to get started is the <code>WebApplicationFactory&lt;TEntryPoint&gt;</code> class, although you're probably going to need a derived class to override some of the system configuration. </p> <p> From there, you can interact with the self-hosted system using the standard <code>HttpClient</code> class. </p> <p> Since I configure these tests to run on an in-memory database, the execution time is comparable to 'normal' unit tests. I admit that I haven't measured it, but that's because I haven't felt the need to do so. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="b2a9eb23454748c18b85927346d33ba9"> <div class="comment-author">Grzegorz Gałęzowski</div> <div class="comment-content"> <p>Hi Mark, 2 questions from me: <ol> <li>What kind of integration does this test verify? The integration between the HTTP request processing part and the service logic? If so, why fake the database only and not the whole logic?</li> <li>If you fake the database here, then would you have a separate test for testing integration with DB (e.g. some kind of "adapter test", as described in the GOOS book)?</li> </ol> </p> </div> <div class="comment-date">2021-01-25 20:34 UTC</div> </div> <div class="comment" id="05077196c8394879829e61d8c08fff08"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Grzegorz, thank you for writing. </p> <p> 1. As I already hinted at, the motivation for this test was that I was expanding the code from a single-tenant to a multi-tenant system. I did this in small steps, using tests to drive the augmented behaviour. Before I added the above test, I'd introduced the concept of a <em>restaurant ID</em> to identify each restaurant, and initially added a method overload to my data access interface that required such an ID: </p> <p> <pre><span style="color:#2b91af;">Task</span>&nbsp;<span style="color:#74531f;">Create</span>(<span style="color:blue;">int</span>&nbsp;<span style="color:#1f377f;">restaurantId</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;<span style="color:#1f377f;">reservation</span>);</pre> </p> <p> I was using the <a href="https://martinfowler.com/bliki/StranglerFigApplication.html">Strangler pattern</a> to be able to move in small steps. One step was to add overloads like the above. Subsequent steps would be to move existing callers from the 'legacy' overload to the new overload, and then finally delete the 'legacy' overload. </p> <p> When moving from a single-tenant system to a multi-tenant system, I had to <a href="https://en.wikipedia.org/wiki/Grandfather_clause">grandfather in</a> the existing restaurant, which I arbitrarily chose to give the restaurant ID <em>1</em>. During this process, I kept the existing system working so as to not break existing clients. </p> <p> I gradually introduced method overloads that took a restaurant ID as a parameter (as above), and then deleted the legacy methods that didn't take a restaurant ID. In order to not break anything, I'd initially be calling the new methods with the hard-coded restaurant ID <em>1</em>. </p> <p> The above test was one in a series of tests that followed. Their purpose was to verify that the system had become truly multi-tenant. Before I added that test, an attempt to make a reservation at <em>Nono</em> would result in a call to <code>Create(1, reservation)</code> because there was still hard-coded restaurant IDs left in the code base. </p> <p> (To be honest, the above is a simplified account of events. It was actually more complex than that, since it involved hypermedia controls. I also, for reasons that I may divulge in the future, didn't perform this work behind a feature flag, which under other circumstances would have been appropriate.) </p> <p> What the above test verifies, then, is that the reservation gets associated with the correct restaurant, instead of the restaurant that was grandfathered in (<em>Hipgnosta</em>). An in-memory (<a href="http://xunitpatterns.com/Fake%20Object.html">Fake</a>) database was sufficient to demonstrate that. </p> <p> Why didn't I use the real, relational database? Read on. </p> <p> 2. As I've <a href="/2020/07/20/closing-database-connections-during-test-teardown">recently discussed</a>, integration tests that involve SQL Server are certainly possible. They tend to be orders-of-magnitudes slower than unit tests that run entirely in memory, so I only add them if I deem them necessary. </p> <p> I interact with databases according to the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion Principle</a>. The above <code>Create</code> method is an example of that. The method is defined by the needs of the client code rather than on implementation details. </p> <p> The actual implementation of the data access interface is, as you imply, an <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a>. It adapts the SQL Server SDK (i.e. ADO.NET) to my data access interface. As long as I can keep such Adapters <a href="http://xunitpatterns.com/Humble%20Object.html">Humble Objects</a> I don't cover them by tests. </p> <p> Such Adapters are often just mappings: This class field maps to that table column, that field to that other column, and so on. If there's no logic, there isn't much to test. </p> <p> This doesn't mean that bugs can't appear in database Adapters. If that happens, I may introduce regression tests that involve the database. I prefer to do this in a separate test suite, however, since such tests tend to be slow, and <a href="/2012/05/24/TDDtestsuitesshouldrunin10secondsorless">TDD test suites should be fast</a>. </p> </div> <div class="comment-date">2021-01-26 8:32 UTC</div> </div> <div class="comment" id="3e56b76abc8e48e3a24174e9ba55d616"> <div class="comment-author">Brian Elgaard Bennett</div> <div class="comment-content"> <p>Hi Mark, I have been a fan of "integration" style testing since I found <a href="https://andrewlock.net/should-you-unit-test-controllers-in-aspnetcore/">Andrew Locks's post on the topic</a>. In fact, I personally find that it makes sense to <a href="https://medium.com/swlh/should-you-unit-test-in-asp-net-core-793de767ac68">take that idea to one extreme</a>. </p> <p> As far as I can see, your test assumes something like "Given that no reservations have been made". Did you consider to make that "Given" explicitly recognizable in your test? </p> <p> I am asking because this is a major pet topic of mine and I would love to hear your thoughts. I am not suggesting writing Gherkin instead of C# code, just if and how you would make the "Given" assumption explicit. </p> <p> A more practical comment is on your use of services.RemoveAll<IReservationsRepository>(); It seems extreme to remove all, as all you want to achieve is to overwrite a few service declarations. Andrew Lock demonstrates how to keep all the ASP.NET middleware intact by keeping most of the service registrations. </p> <p> I usually use ConfigureTestServices on WebApplicationFactory, something like, <pre style="font-family:Consolas;font-size:13px;color:black;background:white;"><span style="color:#1f377f;">factory</span>.<span style="color:#74531f;">WithWebHostBuilder</span>(<span style="color:#1f377f;">builder</span>&nbsp;=&gt;&nbsp;<span style="color:#1f377f;">builder</span>.<span style="color:#74531f;">ConfigureTestServices</span>(<span style="color:#2b91af;">MyTestCompositionRoot</span>.<span style="color:#74531f;">Initialize</span>)); </pre> </p> </div> <div class="comment-date">2021-02-22 10:40 UTC</div> </div> <div class="comment" id="0078721da53e4aa792698fae4a6ab325"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Brian, thank you for writing. You're correct that one implicit assumption is that no prior reservations have been made to <em>Nono</em> on that date. The reality is, as you suggest, that there are no reservations <em>at all</em>, but the test doesn't require that. It only relies on the weaker assumption that there are prior reservations to neither <em>Nono</em> nor <em>Hipgnosta</em> on the date in question. </p> <p> I often find it tricky to make 'negative' assumptions explicit. How much should one highlight? Is it also relevant to highlight that <code>SelfHostedApi</code> isn't going to send notification emails? That it doesn't write to a persistent log? That it doesn't have the capability to <a href="https://stackoverflow.com/q/2773004/126014">launch missiles</a>? </p> <p> I'm not saying that it's irrelevant to highlight certain assumptions, but I'm not sure that I have a clear heuristic for it. After all, it depends on what the reader cares about, and how clear the assumptions are. </p> <p> If it became clear to me that the assumption of 'no prior reservations' was important to the reader, I might consider to embed that in the name of the test, or perhaps by adding a comment. Another option is to hide the initialisation of <code>SelfHostedApi</code> behind a required factory method so that you'd have to initialise it as <code>using var api = SelfHostedApi.StartEmpty();</code> </p> <p> In this particular code base, though, that <code>SelfHostedApi</code> is used repeatedly. A programmer who regularly works with that code base will quickly internalise the assumptions embedded in that self-hosted service. </p> <p> Granted, it can be a dangerous strategy to rely too much on implicit assumptions, because it can make it more difficult to change things if those assumptions change in the future. I do, after all, favour the wisdom from <a href="https://www.python.org/dev/peps/pep-0020/">the Zen of Python</a>: <em>Explicit is better than implicit.</em> </p> <p> Making such things as we discuss here explicit does, however, tend to make the tests more verbose. Pragmatically, then, there's a balance to strike. I'm not claiming that the test shown here is perfect. In a a living code base, this would be a concern that I would keep a constant eye on. I'd fiddle with it, trying out making things more explicit, less explicit, and so on, to see what works best in context. </p> <p> Your other point about <code>services.RemoveAll&lt;IReservationsRepository&gt;()</code> I don't understand. Why is that extreme? It's literally the most minimal, precise change I can make. It removes only the <code>IReservationsRepository</code>, of which there's exactly one Singleton instance: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">connStr</span>&nbsp;=&nbsp;Configuration.<span style="color:#74531f;">GetConnectionString</span>(<span style="color:#a31515;">&quot;Restaurant&quot;</span>); <span style="color:#1f377f;">services</span>.<span style="color:#74531f;">AddSingleton</span>&lt;<span style="color:#2b91af;">IReservationsRepository</span>&gt;(<span style="color:#1f377f;">sp</span>&nbsp;=&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">logger</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sp</span>.<span style="color:#74531f;">GetService</span>&lt;<span style="color:#2b91af;">ILogger</span>&lt;<span style="color:#2b91af;">LoggingReservationsRepository</span>&gt;&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;<span style="color:#1f377f;">postOffice</span>&nbsp;=&nbsp;<span style="color:#1f377f;">sp</span>.<span style="color:#74531f;">GetService</span>&lt;<span style="color:#2b91af;">IPostOffice</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#8f08c4;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">EmailingReservationsRepository</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">postOffice</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LoggingReservationsRepository</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#1f377f;">logger</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SqlReservationsRepository</span>(<span style="color:#1f377f;">connStr</span>))); });</pre> </p> <p> The integration test throws away that Singleton object graph, but nothing else. </p> </div> <div class="comment-date">2021-02-23 9:38 UTC</div> </div> <div class="comment" id="8f23293c06234053853c0374b9200dd3"> <div class="comment-author">Brian Elgaard Bennett</div> <div class="comment-content"> <p>Hi Mark, thanks for a thorough answer.</p> <p>Alas, my second question was not thought through - I somehow thought you threw away all service registrations, which you obviously do not. In fact, I do exactly what you do, usually with Replace and thereby <i>implicitly</i> assume that there is only one registration. My Bad. <p>We very much agree in the importance of favouring <i>explicit</i> assumptions.</p> <p>The solution to stating 'negative' assumptions, as long as we talk about the 'initial context' (the 'Given') is actually quite simple if your readers know that you always state assumptions explicitly. If none are stated, it's the same as saying 'the world is empty' - at least when it comes to the bounded context of whatever you are testing. I find that the process of identifying a minimal set of assumptions per test is a worthwhile exercise.</p> <p>So, IMHO it boils down to consistently stating all assumptions and making the relevant bounded context clear.</p> </div> <div class="comment-date">2021-02-23 12:18 UTC</div> </div> <div class="comment" id="60934ff25b0e45bebca4f00749a6b44c"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Brian, thank you for writing. I agree that it's a good rule of thumb to expect the world to be empty unless explicitly stated otherwise. This is the reason that I love functional programming. </p> <p> One question, however, is whether a statement is sufficiently explicit. The integration test shown here is a good example. What's actually happening is that <code>SelfHostedApi</code> sets up a self-hosted service configured in a certain way. It contains three restaurants, of which two are relevant in this particular test. Each restaurant comes with quite a bit of configuration: opening time, closing time, seating duration, and the configuration of tables. Just consider the configuration of <em>Nono:</em> </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Id&quot;</span>:&nbsp;2112, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;OpensAt&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;LastSeating&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;21:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;SeatingDuration&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;6:00&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Tables&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Communal&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;6 &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Communal&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Standard&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Standard&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Standard&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;TableType&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Standard&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;Seats&quot;</span>:&nbsp;4 &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> The code base also enables a test writer to express the above configuration as code instead of JSON, but the point I'm trying to make is that you probably don't want that amount of detail to appear in the <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">Arrange phase</a> of each test, regardless of how explicit it'd be. </p> <p> To work around an issue like this, you might instead define an <a href="/2017/09/11/test-data-without-builders">immutable test data variable</a> called <code>nono</code>. Then, in each test that uses these restaurants, you might include something like <code>AddRestaurant(nono)</code> or <code>AddRestaurant(hipgnosta)</code> in the Arrange phase (hypothetical API). That's not <em>quite</em> as explicit, because it hides the actual values away, but is probably still good enough. </p> <p> Then you find yourself always adding the same restaurants, so you say to yourself: <em>It'd be nice if I had a helper method like <code>AddStandardRestaurants</code></em>. So, you may add such a helper method. </p> <p> But you find that for the integration tests, you <em>always</em> call that helper method, so ultimately, you decide to just roll it into the definition of <code>SelfHostedApi</code>. That's basically what's going on here. </p> <p> I don't recall whether I explicitly stated the following in the article, but I use integration tests consistent with the <a href="https://martinfowler.com/bliki/TestPyramid.html">test pyramid</a>. The purpose of the integration tests is to verify that components integrate correctly, and that the HTTP API behaves according to contract. In my experience, I can let the 'givens' be somewhat implicit and still keep the code maintainable. For integration tests, that is. </p> <p> On the unit level, I favour pure functions, which are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. Due to <em>isolation</em>, everything that affects the behaviour of a pure function must be explicitly passed as arguments, and as a bi-product, they become explicit as part of a test's Arrange phase. </p> <p> But, again, let me reiterate that I agree with you. I don't claim that the code shown here is perfect. It's so implicit that it makes me uncomfortable too, but on the other hand, I think that a test like the above communicates intent well. The more explicit details one adds, the more they may drown out the intent. It's a difficult balance to strike. </p> </div> <div class="comment-date">2021-02-25 12:34 UTC</div> </div> <div class="comment" id="8f23293c06234053853c0374b9200d42"> <div class="comment-author">Brian Elgaard Bennett</div> <div class="comment-content"> <p>Hi Mark, I'm happy that we agree. Still, this is a pet topic of mine so please allow me one additional comment.</p> <p>I wish all code had at least a functional core, as that would make testing so much easier. You could say that explicitly stating initial context resembles testing pure functions.</p> <p>I do believe that it is possible to state initial context without being overwhelmed by details. It can by no means be fully automated - this is where the mind of a tester must be used. Also, it will never be perfect, but at least it can be documented somewhat systematically and kept under version control, so the reasoning can be challenged.</p> <p>In your example, I would <i>not</i> add Nono or Hipgnosta or a standard restaurant, as those names say noting about what makes my test pass.</p> <p>Rather, I would add e.g. a "restaurant with a 6 and a 4 person table", as this seems to be what makes your test pass, if I understood it correctly. In another test I might have "restaurant which is open from 6 PM with last seating at 9 PM" if I want to test time aspects of booking. It may be the same Json file used in those two tests, because what's most important is that you document what makes each individual test pass.</p> <p>However, if it's not too much trouble (it sometimes is) I would even try to minimize the initial context. For example, I would test timing aspects with a minimal restaurant with a single table. Unless, timing and tables interact - maybe late bookings are OK if the restaurant is starved for customers? This is where the "tester mind" comes in handy.</p> <p>In your restaurant example, that would mean having quite a few restaurant definitions, but only definitions which include the needed context for each test. My experience is that minimizing the initial context will teach you a lot about the code. This is similar to unit tests which focus on a small part of code, but "minimizing initial context" allows for less fragile tests than when "minimizing code".</p> <p>We started using this approach because our tests did not give sufficient confidence. Tests were flaky and it was difficult to see what was tested and why. I can safely say that this is much better now.</p> <p>My examples of initial context above may appear like they could grow out of control, but our experience so far is that it is possible to find a relatively small set of equivalence classes of initial context items, even for fairly complex functionality.</p> </div> <div class="comment-date">2021-02-28 15:24 UTC</div> </div> <div class="comment" id="76bca252379c4f73a676ea505f16a550"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Brian, I agree that in unit testing, being able to reduce test cases to minimal examples is useful. Once you have those minimal examples, being explicit about all preconditions is more attainable. This is why, when I coach teams, I try to teach them about <a href="https://en.wikipedia.org/wiki/Equivalence_partitioning">equivalence class partitioning</a>. If, within each equivalence class, you can impose some sort of (partial) ordering, you may be able to pick a canonical representation that constitutes the minimal example. </p> <p> This is basically what the shrinking process of property-based testing frameworks attempt to do. </p> <p> Thus, when my testing goal is to explore the boundaries of the system under test, I go to great lengths to ensure that the arrange phase is as explicit as possible. I <a href="/2021/02/15/when-properties-are-easier-than-examples">recently published an example of how I use property-based testing</a> to such an end. </p> <p> The goal of the integration test in the present article, however, is <em>not</em> to explore boundary cases or verify the correctness of algorithms in question. I have unit tests for that. The goal is to examine whether things may be correctly 'clicked together'. </p> <p> Expressing an explicit minimal context for the above test is more involved than you outline. Having two restaurants, with two differently sized tables, is only the beginning. You must also ensure that they are open at the same time, so that an attempted reservation would fit both. Furthermore, you must then pick the date and time of the reservation to fit within that time window. This is still not hopelessly difficult, and might make for a good exercise, but it could easily add 4-5 extra lines of code to the test. </p> <p> And again: had this been a unit test, I'd happily added those extra lines, but for an integration test, I felt that it was less important. </p> </div> <div class="comment-date">2021-03-02 11:24 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Parametrised test primitive obsession code smell https://blog.ploeh.dk/2021/01/18/parametrised-test-primitive-obsession-code-smell 2021-01-18T06:30:00+00:00 Mark Seemann <div id="post"> <p> <em>Watch out for this code smell with some unit testing frameworks.</em> </p> <p> In a <a href="/2021/01/11/waiting-to-happen">previous article</a> you saw this <a href="/2019/04/01/an-example-of-state-based-testing-in-c">state-based integration test</a>: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> This was the test <em>after</em> I improved it. Still, I wasn't satisfied with it. It has several problems. Take a few moments to consider it. Can you identify any problems? Which ones? </p> <h3 id="6da86d1040e544a3a67f509c0f3aade7"> Size <a href="#6da86d1040e544a3a67f509c0f3aade7" title="permalink">#</a> </h3> <p> I know that you're not familiar with all the moving parts. You don't know how <code>ReservationDto</code> or <code>Reservation</code> are implemented. You don't know what <code>InMemoryRestaurantDatabase</code> is, or how <code>ReservationsController</code> behaves. Still, the issues I have in mind aren't specific to a particular code base. </p> <p> I feel that the method is verging on being too big. Quantifiably, it doesn't fit in an <a href="/2019/11/04/the-80-24-rule">80x24 box</a>, but that's just an arbitrary <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a> anyway. Still, I think it's grown to a size that makes me uncomfortable. </p> <p> If you aren't convinced, think of this code example as a stand-in for something larger. In the above test, a reservation contains five smaller values (<code>Id</code>, <code>At</code>, <code>Email</code>, <code>Name</code>, and <code>Quantity</code>). How would a similar test look if the object in question contains ten or twenty values? </p> <p> In the decades I've been programming and consulting, I've seen plenty of code bases. Data objects made from twenty fields are hardly unusual. </p> <p> What would a similar test look like if the <code>dto</code> and the <code>expected</code> object required twenty smaller values? </p> <p> The test would be too big. </p> <h3 id="e251bee2df414c5cb0d489f31183c664"> Primitive obsession <a href="#e251bee2df414c5cb0d489f31183c664" title="permalink">#</a> </h3> <p> A test like this one contains a mix of essential behaviour and implementation details. The behaviour that it verifies is that when you <code>Post</code> a valid <code>dto</code>, the data makes it all the way to the database. </p> <p> Exactly how the <code>dto</code> or the <code>expected</code> value are constructed is less relevant for the test. Yet it's intermingled with the test of behaviour. The signal-to-noise ratio in the test isn't all that great. What can you do to improve things? </p> <p> As given, it seems difficult to do much. The problem is <a href="/2011/05/25/DesignSmellPrimitiveObsession">primitive obsession</a>. While this is a <a href="http://xunitpatterns.com/Parameterized%20Test.html">parametrised test</a>, all the method parameters are primitives: integers and strings. The makes it hard to introduce useful abstractions. </p> <p> In C# (and probably other languages as well) parametrised tests often suffer from primitive obsession. The most common data source is an attribute (AKA <em>annotation</em>), like <a href="https://xunit.net">xUnit.net</a>'s <code>[InlineData]</code> attribute. This isn't a limitation of xUnit.net, but rather of .NET attributes. You can only create attributes with primitive values and arrays. </p> <p> What <em>is</em> a limitation of xUnit.net (and the other mainstream .NET testing frameworks, as far as I know) is that tests aren't first-class values. In <a href="https://www.haskell.org">Haskell</a>, by contrast, it's <a href="/2018/04/30/parametrised-unit-tests-in-haskell">easy to write parametrised tests using the normal language constructs</a> exactly because tests are first-class values. (I hope that the next version of xUnit.net will support tests as first-class values.) </p> <p> Imagine that instead of only five constituent fields, you'd have to write a parametrised test for objects with twenty primitive values. As long as you stick with attribute-based data sources, you'll be stuck with primitive values. </p> <p> Granted, attributes like <code>[InlineData]</code> are lightweight, but over the years, my patience with them has grown shorter. They lock me into primitive obsession, and I don't appreciate that. </p> <h3 id="9342d4306e174fe79946a131b9c6894c"> Essential test <a href="#9342d4306e174fe79946a131b9c6894c" title="permalink">#</a> </h3> <p> While tests as first-class values aren't an option in xUnit.net, you can provide other data sources for the <code>[Theory]</code> attribute than <code>[InlineData]</code>. It's not as lightweight, but it breaks the primitive obsession and re-enables normal code design techniques. It enables you to reduce the test itself to its essence. You no longer have to think in primitives, but can instead express the test unshackled by constraints. As a first pass, I'd like the test to look like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">ClassData</span>(<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;validDto,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;expected) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(validDto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> This version of the test eliminates the noise. How <code>validDto</code> and <code>expected</code> are constructed is an implementation detail that has little bearing on the behaviour being tested. </p> <p> For a reader of the code, it's should now be clearer what's at stake here: If you <code>Post</code> a <code>validDto</code> the <code>expected</code> reservation should appear in the database. </p> <p> Reducing the test code to its essentials made me realise something that hitherto had escaped me: that I could <a href="/2020/11/30/name-by-role">name the DTO by role</a>. Instead of just <code>dto</code>, I could call the parameter <code>validDto</code>. </p> <p> Granted, I could also have done that previously, but I didn't think of it. There's was so much noise in that test that I didn't stop to consider whether <code>dto</code> sufficiently communicated the role of that variable. </p> <p> The less code, the easier it becomes to think such things through, I find. </p> <p> In any case, the test code now much more succinctly expresses the essence of the desired behaviour. Notice how I started my refactoring by writing the desired test code. I've yet to implement the data source. Now that the data source expresses test data as full objects, I'm not so concerned with whether or not that's going to be possible. Of course it's possible. </p> <h3 id="469ee6ae8f264e21a544f11be2476111"> Object data source <a href="#469ee6ae8f264e21a544f11be2476111" title="permalink">#</a> </h3> <p> You can define data sources for xUnit.net as classes or methods. In C# I usually reach for the <code>[ClassData]</code> option, since an object (in C#, that is) gives me better options for further decomposition. For example, I can define a class and delegate the details to helper methods: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TheoryData</span>&lt;<span style="color:#2b91af;">ReservationDto</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithoutName(956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;2); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;members&nbsp;here...</span></pre> </p> <p> Here, I'm taking advantage of xUnit.net's built-in <code>TheoryData&lt;T1, T2&gt;</code> base class, but that's just a convenience. All you have to do is to implement <code>IEnumerable&lt;object[]&gt;</code>. </p> <p> As you can see, the constructor adds the four test cases by calling two private helper methods. Here's the first of those: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>; <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithName( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(name), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity)); }</pre> </p> <p> The other helper method is almost identical, although it has a slight variation when it comes to the reservation name: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithoutName( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;quantity)); }</pre> </p> <p> In total, this refactoring results in <em>more</em> code, so how is this an improvement? </p> <h3 id="9722aade54194b1a87c7312d16ed7fc8"> The paradox of decomposition <a href="#9722aade54194b1a87c7312d16ed7fc8" title="permalink">#</a> </h3> <p> In object-oriented design, decomposition tends to lead to more code. If you want to isolate and make reusable a particular piece of behaviour, you'll usually introduce an interface or a base class. Even stateless functions need a static class to define them. (To be fair, functional programming isn't entirely devoid of such overhead associated with decomposition, but the cost tends to smaller.) This leads to more code, compared with the situation before decomposition. </p> <p> This is a situation you may also encounter if you attempt to refactor to design patterns, or follow the <a href="/encapsulation-and-solid">SOLID principles</a>. You'll have more code than when you started. This often leads to resistance to such 'code bloat'. </p> <p> It's fine to resist code bloat. It's also fine to dislike 'complexity for complexity's sake'. Try to evaluate each potential code change based on advantages and disadvantages. I'm not insisting that the above refactoring is objectively better. I did feel, however, that I had a problem that I ought to address, and that this was a viable alternative. The result is more code, but each piece of code is smaller and simpler. </p> <p> You can, conceivably, read the test method itself to get a feel for what it tests, even if you don't know all the implementation details. You can read the four statements in the <code>PostValidReservationWhenDatabaseIsEmptyTestCases</code> constructor without, I hope, understanding all the details about the two helper methods. And you <em>can</em> read <code>AddWithName</code> without understanding how <code>AddWithoutName</code> works, and vice versa, because these two methods don't depend on each other. </p> <h3 id="27b546f17ba44571a11ed44db9d45a09"> Conclusion <a href="#27b546f17ba44571a11ed44db9d45a09" title="permalink">#</a> </h3> <p> In this article, I've described how the use of code annotations for parametrised tests tend to pull in the direction of primitive obsession. This is a force worth keeping an eye on, I think. </p> <p> You saw how to refactor to class-based test data generation. This enables you to use objects instead of primitives, thus opening your design palette. You can now use all your usual object-oriented or functional design skills to factor the code in a way that's satisfactory. </p> <p> Was it worth it in this case? Keep in mind that the original problem was already marginal. While the code didn't fit in a 80x24 box, it was only 33 lines of code (excluding the test data). Imagine, however, that instead of a five-field reservation, you'd be dealing with a twenty-field data class, and such a refactoring begins to look more compelling. </p> <p> Is the code now perfect? It still isn't. I'm a little put off by the similarity of <code>AddWithName</code> and <code>AddWithoutName</code>. I'm also aware that there's a trace of production code duplicated in the test case, in the way that the test code duplicates how a valid <code>ReservationDto</code> relates to a <code>Reservation</code>. I'm on the fence whether I should do anything about this. </p> <p> At the moment I'm inclined to heed <a href="https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)">the rule of three</a>. The duplication is still too insubstantial to warrant refactoring, but it's worth keeping an eye on. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Waiting to happen https://blog.ploeh.dk/2021/01/11/waiting-to-happen 2021-01-11T06:31:00+00:00 Mark Seemann <div id="post"> <p> <em>A typical future test maintenance problem.</em> </p> <p> In <a href="/2020/12/07/branching-tests">a recent article</a> I showed a unit test and parenthetically mentioned that it might have a future maintenance problem. Here's a more recent version of the same test. Can you tell what the future issue might be? </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2022-03-18&nbsp;17:30&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> To be honest, there's more than one problem with this test, but presently I'm going to focus on one of them. </p> <p> Since you don't know the details of the implementation, you may not be able to tell what the problem might be. It's not a trick question. On the other hand, you might still be able to guess, just from the clues available in the above code listing. </p> <h3 id="970cada688cc4a04abbb1c11328931f1"> Sooner or later <a href="#970cada688cc4a04abbb1c11328931f1" title="permalink">#</a> </h3> <p> Here are some clues to consider: I'm writing this article in the beginning of 2021. Consider the dates supplied via the <code>[InlineData]</code> attributes. Seen from 2021, they're all in the future. </p> <p> Notice, as well, that the <code>sut</code> takes a <code>SystemClock</code> dependency. You don't know the <code>SystemClock</code> class (it's a proprietary class in this code base), but from the name I'm sure that you can imagine what it represents. </p> <p> From the perspective of early 2021, all dates are going to be in the future for more than a year. What is going to happen, though, once the test runs after March 18, 2022? </p> <p> That test case is going to fail. </p> <p> You can't tell from the above code listing, but the system under test rejects reservations in the past. Once March 18, 2022 has come and gone, the reservation at <code>"2022-03-18 17:30"</code> is going to be in the past. The <code>sut</code> will reject the reservation, and the assertion will fail. </p> <p> You have to be careful with tests that rely on the system clock. </p> <h3 id="61e0c1225dd34c40a0ceb19270ae9007"> Test Double? <a href="#61e0c1225dd34c40a0ceb19270ae9007" title="permalink">#</a> </h3> <p> The fundamental problem is that the system clock is non-deterministic. A typical reaction to non-determinism in unit testing is to introduce a <a href="https://martinfowler.com/bliki/TestDouble.html">Test Double</a> of some sort. Instead of using the system clock, you could use a <a href="/2013/10/23/mocks-for-commands-stubs-for-queries">Stub</a> as a stand-in for the real time. </p> <p> This is possible here as well. The <code>ReservationsController</code> class actually depends on an <code>IClock</code> interface that <code>SystemClock</code> implements. You could define a test-specific <code>ConstantClock</code> implementation that would always return a constant date and time. This would actually work, but would rely on an implementation detail. </p> <p> At the moment, the <code>ReservationsController</code> only calls <code>Clock.GetCurrentDateTime()</code> a <em>single time</em> to get the current time. As soon as it has that value, it passes it to a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a>, which implements <a href="/2020/01/27/the-maitre-d-kata">the business logic</a>: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); <span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError();</pre> </p> <p> A <code>ConstantClock</code> would work, but only as long as the <code>ReservationsController</code> only calls <code>Clock.GetCurrentDateTime()</code> once. If it ever began to call this method multiple times to detect the passing of time, using a constant time value would mostly likely again break the test. This seems brittle, so I don't want to go that way. </p> <h3 id="29f9100aa1104cd2a1db18146384bc17"> Relative time <a href="#29f9100aa1104cd2a1db18146384bc17" title="permalink">#</a> </h3> <p> Working with the system clock in automated tests is easier if you deal with relative time. Instead of defining the test cases as absolute dates, express them as days into the future. Here's one way to refactor the test: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(1049,&nbsp;19,&nbsp;00,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(1130,&nbsp;18,&nbsp;15,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;956,&nbsp;16,&nbsp;55,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;433,&nbsp;17,&nbsp;30,&nbsp;<span style="color:#a31515;">&quot;shli@example.org&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Shanghai&nbsp;Li&quot;</span>,&nbsp;5)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;days, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;hours, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;minutes, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Date&nbsp;+&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSpan</span>(days,&nbsp;hours,&nbsp;minutes,&nbsp;0); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Grandfather</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;<span style="color:#a31515;">&quot;B50DF5B1-F484-4D99-88F9-1915087AF568&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Guid</span>.Parse(dto.Id), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Email</span>(dto.Email), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Name</span>(dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db.Grandfather); }</pre> </p> <p> The absolute dates always were fairly arbitrary, so I just took the current date and converted the dates to a number of days into the future. Now, the first test case will always be a date 1,049 days (not quite three years) into the future, instead of November 24, 2023. </p> <p> The test is no longer a failure waiting to happen. </p> <h3 id="906499c9b3d647d08d2f48eb5991cf43"> Conclusion <a href="#906499c9b3d647d08d2f48eb5991cf43" title="permalink">#</a> </h3> <p> Treating test cases that involve time and date as relative to the current time, instead of as absolute values, is usually a good idea if the system under test depends on the system clock. </p> <p> It's always a good idea to factor as much code as you can as pure functions, like the above <code>WillAccept</code> method. Pure functions don't depend on the system clock, so here you can safely pass absolute time and date values. Pure functions are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. </p> <p> Still, as the <a href="https://martinfowler.com/bliki/TestPyramid.html">test pyramid</a> suggests, relying exclusively on unit tests isn't a good idea. The test shown in this article isn't really a unit test, but rather a <a href="/2019/04/01/an-example-of-state-based-testing-in-c">state-based integration test</a>. It relies on both the system clock and a <a href="http://xunitpatterns.com/Fake%20Object.html">Fake</a> database. Expressing the test cases for this test as relative time values effectively addresses the problem introduced by the system clock. </p> <p> There are plenty of other problems with the above test. One thing that bothers me is that the 'fix' made the line count grow. It didn't quite fit into a <a href="/2019/11/04/the-80-24-rule">80x24 box</a> before, but now it's even worse! I should do something about that, but that's a topic for <a href="/2021/01/18/parametrised-test-primitive-obsession-code-smell">another article</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Dynamic test oracles for rho problems https://blog.ploeh.dk/2021/01/04/dynamic-test-oracles-for-rho-problems 2021-01-04T06:26:00+00:00 Mark Seemann <div id="post"> <p> <em>A proof of concept of cross-branch testing for compiled languages.</em> </p> <p> <a href="https://www.hillelwayne.com">Hillel Wayne</a> recently published an article called <a href="https://buttondown.email/hillelwayne/archive/cross-branch-testing/">Cross-Branch Testing</a>. It outlines an approach to a class of problems that are hard to test. He mentions computer vision and simulations, among others. I can add that it's also <a href="/2015/10/19/visual-value-verification">difficult to write intuitive tests of convex hulls</a> and <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Conway's game of life</a>. </p> <p> Hillel Wayne calls these <em>rho problems</em>, 'just because'. I'm totally going to run with that term. </p> <p> In the article, he outlines an approach where you test an iteration of rho code against a 'last known good' snapshot. He uses <code>git worktree</code> to set up a snapshot of the reference implementation. He then writes a property that compares the refactored code's behaviour against the reference. </p> <p> The example code is in <a href="https://www.python.org">Python</a>, which is a language that I don't know. As far as I can tell, it works because Python is 'lightweight' enough that you can load and execute source code directly. I found that the approach makes much sense, but I wondered how it would apply for statically typed, compiled languages. I decided to create a proof of concept in <a href="https://fsharp.org">F#</a>. </p> <h3 id="ab0c6139b2c84e148fe1173c2508ec62"> Test cases from Python <a href="#ab0c6139b2c84e148fe1173c2508ec62" title="permalink">#</a> </h3> <p> My first problem was to port Hillel Wayne's example rho problem to F#. The function <code>f</code> doesn't have any immediate mathematical properties; nor is its behaviour intuitive. While I think that I understand what each line of code in <code>f</code> means, I don't really know Python. Since one of the properties of rho problems is that bugs can be subtle, I didn't trust myself to be able to port the Python code to F# without some test cases. </p> <p> To solve that problem, I first found an online Python interpreter and pasted the <code>f</code> function into it. I then wrote code to print the output of a function call: </p> <p> <pre>print(f'1, 2, 3, { f(1, 2, 3) }')</pre> </p> <p> This line of code produces this output: </p> <p> <pre>1, 2, 3, True</pre> </p> <p> In other words, I could produce comma-separated values of input and actual output. </p> <p> Hillel Wayne wrote properties using <a href="https://hypothesis.works">Hypothesis</a>, which, <a href="https://hypothesis.works/articles/how-many-tests">it seems</a>, by default runs each property 200 times. </p> <p> In F# I'm going to use <a href="https://fscheck.github.io/FsCheck">FsCheck</a>, so I first used <em>F# Interactive</em> with FsCheck to produce 200 Python <code>print</code> statements like the above: </p> <p> <pre>&gt; Arb.Default.Int32().Generator |&gt; Gen.three |&gt; Gen.map (fun (x, y, z) -&gt; sprintf "print(f'%i, %i, %i, { f(%i, %i, %i) }')" x y z x y z) |&gt; Gen.sample 100 200 |&gt; List.iter (printfn "%s");; print(f'-77, 67, 84, { f(-77, 67, 84) }') print(f'58, -46, 3, { f(58, -46, 3) }') print(f'21, 13, 94, { f(21, 13, 94) }') ... </pre> </p> <p> This is a throwaway data pipeline that starts with an FsCheck integer generator, creates a triple from it, turns that triple into a Python <code>print</code> statement, and finally writes 200 of those to the console. The above code listing only shows the first three lines of output, while the rest are indicated by an ellipsis. </p> <p> I copied those 200 <code>print</code> statements over to the online Python interpreter and ran the code. That produced 200 comma-separated values like these: </p> <p> <pre>-77, 67, 84, False 58, -46, 3, False 21, 13, 94, True ...</pre> </p> <p> These can serve as test cases for porting the Python code to F#. </p> <h3 id="936410215a784d73ae9b5dcba9125a4c"> Port to F# <a href="#936410215a784d73ae9b5dcba9125a4c" title="permalink">#</a> </h3> <p> The next step is to write a parametrised test, using a provisional implementation of <code>f</code>: </p> <p> <pre>[&lt;Theory;&nbsp;MemberData(nameof&nbsp;fTestCases)&gt;] <span style="color:blue;">let</span>&nbsp;``test&nbsp;f``&nbsp;x&nbsp;y&nbsp;z&nbsp;expected&nbsp;=&nbsp;expected&nbsp;=!&nbsp;f&nbsp;x&nbsp;y&nbsp;z</pre> </p> <p> This test uses <a href="https://xunit.net">xUnit.net</a> 2.4.1 and <a href="https://github.com/SwensenSoftware/Unquote">Unquote</a> 5.0.0. As you can tell, apart from the annotations, it's a true one-liner. It calls the <code>f</code> function with the three supplied arguments <code>x</code>, <code>y</code>, and <code>z</code> and compares the return value with the <code>expected</code> value. </p> <p> The code uses the new <a href="https://docs.microsoft.com/dotnet/fsharp/language-reference/nameof">nameof</a> feature of F# 5. <code>fTestCases</code> is a function in the same module that holds the test: </p> <p> <pre><span style="color:green;">//&nbsp;unit&nbsp;-&gt;&nbsp;seq&lt;obj&nbsp;[]&gt;</span> <span style="color:blue;">let</span>&nbsp;fTestCases&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;strm&nbsp;=&nbsp;typeof&lt;Anchor&gt;.Assembly.GetManifestResourceStream&nbsp;streamName &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">use</span>&nbsp;rdr&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;StreamReader&nbsp;(strm) &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;s&nbsp;=&nbsp;rdr.ReadToEnd&nbsp;() &nbsp;&nbsp;&nbsp;&nbsp;s.Split&nbsp;Environment.NewLine&nbsp;|&gt;&nbsp;Seq.map&nbsp;csvToTestCase</pre> </p> <p> It reads an embedded resource stream of test cases, like the above comma-separated values. Even though the values are in a text file, it's easier to embed the file in the test assembly, because it nicely dispenses with the problem of copying a text file to the appropriate output directory when the code compiles. That would, however, be an valid alternative. </p> <p> <code>Anchor</code> is a dummy type to support <code>typeof</code>, and <code>streamName</code> is just a string constant that identifies the name of the stream. </p> <p> The <code>csvToTestCase</code> function converts each line of comma-separated values to test cases for the <code>[&lt;Theory&gt;]</code> attribute: </p> <p> <pre><span style="color:green;">//&nbsp;string&nbsp;-&gt;&nbsp;obj&nbsp;[]</span> <span style="color:blue;">let</span>&nbsp;csvToTestCase&nbsp;(csv&nbsp;:&nbsp;string)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;values&nbsp;=&nbsp;csv.Split&nbsp;<span style="color:#a31515;">&#39;,&#39;</span> &nbsp;&nbsp;&nbsp;&nbsp;[| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[0]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[1]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[2]&nbsp;|&gt;&nbsp;Convert.ToInt32&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values.[3]&nbsp;|&gt;&nbsp;Convert.ToBoolean&nbsp;|&gt;&nbsp;box &nbsp;&nbsp;&nbsp;&nbsp;|]</pre> </p> <p> It's not the safest code I could write, but this is, after all, a proof of concept. </p> <p> The most direct port of the Python code I could produce is this: </p> <p> <pre><span style="color:green;">//&nbsp;f&nbsp;:&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;f&nbsp;(x&nbsp;:&nbsp;int)&nbsp;(y&nbsp;:&nbsp;int)&nbsp;(z&nbsp;:&nbsp;int)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;mx&nbsp;=&nbsp;bigint&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;my&nbsp;=&nbsp;bigint&nbsp;y &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;mz&nbsp;=&nbsp;bigint&nbsp;z &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;<span style="color:blue;">mutable</span>&nbsp;out&nbsp;=&nbsp;0I &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">for</span>&nbsp;i&nbsp;<span style="color:blue;">in</span>&nbsp;[0I..9I]&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;out&nbsp;*&nbsp;mx&nbsp;+&nbsp;abs&nbsp;(my&nbsp;*&nbsp;mz&nbsp;-&nbsp;i&nbsp;*&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;x&#39;&nbsp;=&nbsp;mx &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;y&#39;&nbsp;=&nbsp;my &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;z&#39;&nbsp;=&nbsp;mz &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mx&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;y&#39;&nbsp;+&nbsp;1I &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;my&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;z&#39; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mz&nbsp;<span style="color:blue;">&lt;-</span>&nbsp;x&#39; &nbsp;&nbsp;&nbsp;&nbsp;abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;10I</pre> </p> <p> As F# code goes, it's disagreeable, but it passes all 200 test cases, so this will serve as an initial implementation. The <code>out</code> variable can grow to values that overflow even 64-bit integers, so I had to convert to <a href="https://docs.microsoft.com/dotnet/api/system.numerics.biginteger">bigint</a> to get all test cases to pass. </p> <p> If I make the same mutation to the code that Hillel Wayne did (<code>abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;9I</code>) two test cases fail. This gives me some confidence that I have a degree of problem coverage comparable to his. </p> <h3 id="289e30b4098c4e54b8390af0b672caf1"> Test oracle <a href="#289e30b4098c4e54b8390af0b672caf1" title="permalink">#</a> </h3> <p> Now that a reference implementation exists, we can use it as a <a href="https://en.wikipedia.org/wiki/Test_oracle">test oracle</a> for refactorings. You can, for example, add a little test-only utility to your program portfolio: </p> <p> <pre><span style="color:blue;">open</span>&nbsp;Prod <span style="color:blue;">open</span>&nbsp;FsCheck [&lt;EntryPoint&gt;] <span style="color:blue;">let</span>&nbsp;main&nbsp;argv&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;Arb.Default.Int32().Generator &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Gen.three &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Gen.sample&nbsp;100&nbsp;200 &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;List.iter&nbsp;(<span style="color:blue;">fun</span>&nbsp;(x,&nbsp;y,&nbsp;z)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;printfn&nbsp;<span style="color:#a31515;">&quot;%i,&nbsp;%i,&nbsp;%i,&nbsp;%b&quot;</span>&nbsp;x&nbsp;y&nbsp;z&nbsp;(f&nbsp;x&nbsp;y&nbsp;z)) &nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;<span style="color:green;">//&nbsp;return&nbsp;an&nbsp;integer&nbsp;exit&nbsp;code</span></pre> </p> <p> Notice that the last step in the pipeline is to output the values of each <code>x</code>, <code>y</code>, and <code>z</code>, as well as the result of calling <code>f x y z</code>. </p> <p> This is a command-line executable that uses FsCheck to produce new test cases by calling the <code>f</code> function. It looks similar to the above one-off script that produced Python code, but this one instead just produces comma-separated values. You can run it from the command line to produce a new sample of test cases: </p> <p> <pre>$ ./foracle 29, -48, -78, false -8, -25, 13, false -74, 34, -68, true ...</pre> </p> <p> As above, I've used an ellipsis to indicate that in reality, 200 lines of comma-separated values scroll by. </p> <p> When you use Bash, you can even pipe the output straight to a file: </p> <p> <pre>$ ./foracle > csv.txt</pre> </p> <p> You can now take the new comma-separated values and update the test values that the above <code>test f</code> test uses. </p> <p> In other words, you use version <em>n</em> of <code>f</code> as a test oracle for version <em>n + 1</em>. When iteration <em>n + 1</em> is a function of iteration <em>n</em>, you have a so-called <em>dynamic system</em>, so I think that we can call this technique <em>dynamic test oracles</em>. </p> <p> The above <code>foracle</code> program is just a proof of concept. You could make it more flexible by making it take command-line arguments that would let you control the sample size and FsCheck's <code>size</code> parameter (the hard-coded <code>100</code> in the above code listing). </p> <h3 id="b7cee6f3ed874b3390276dd3852850a6"> Refactoring <a href="#b7cee6f3ed874b3390276dd3852850a6" title="permalink">#</a> </h3> <p> With the confidence instilled by the test cases, we can now refactor the <code>f</code> function: </p> <p> <pre><span style="color:green;">//&nbsp;f&nbsp;:&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;int&nbsp;-&gt;&nbsp;bool</span> <span style="color:blue;">let</span>&nbsp;f&nbsp;(x&nbsp;:&nbsp;int)&nbsp;(y&nbsp;:&nbsp;int)&nbsp;(z&nbsp;:&nbsp;int)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;imp&nbsp;(x,&nbsp;y,&nbsp;z,&nbsp;out)&nbsp;i&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;out&nbsp;=&nbsp;out&nbsp;*&nbsp;x&nbsp;+&nbsp;abs&nbsp;(y&nbsp;*&nbsp;z&nbsp;-&nbsp;i&nbsp;*&nbsp;i) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;y&nbsp;+&nbsp;1I,&nbsp;z,&nbsp;x,&nbsp;out &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;(_,&nbsp;_,&nbsp;_,&nbsp;out)&nbsp;=&nbsp;List.fold&nbsp;imp&nbsp;(bigint&nbsp;x,&nbsp;bigint&nbsp;y,&nbsp;bigint&nbsp;z,&nbsp;0I)&nbsp;[0I..9I] &nbsp;&nbsp;&nbsp;&nbsp;abs&nbsp;out&nbsp;%&nbsp;100I&nbsp;&lt;&nbsp;10I</pre> </p> <p> Instead of all those mutable variables, the function is, after all, just a left fold. Phew, I feel better now. </p> <h3 id="f849e03594b448299ba4eef5a6a72b4e"> Conclusion <a href="#f849e03594b448299ba4eef5a6a72b4e" title="permalink">#</a> </h3> <p> This article demonstrated a proof of concept where you use a known good version (<em>n</em>) of the code as a test oracle for the next version (<em>n + 1</em>). In interpreted languages, you may be able to load two versions of the code base side by side, but that's rarely practical in a statically typed compiled language like F#. Instead, I used a utility program to generate test cases that can be used as a data source for a parametrised test. </p> <p> The example rho problem takes only integers as input, and returns a simple Boolean value, so in this case it's trivial to encode each test case as a line of comma-separated values. For (real) problems that may involve more complex types, it'd be better to use another serialisation format, such as JSON or XML. </p> <p> An outstanding issue is whether it's possible to implement shrinking behaviour when tests fail. Currently, the proof of concept just uses a set of serialised test cases. Normally, when a <a href="/property-based-testing-intro">property-based testing</a> framework like FsCheck detects a counter-example, it'll shrink the counter-example to values that are easier to understand than the original. This proof of concept doesn't do that. I'm not sure if a framework like FsCheck currently contains enough extensibility points to enable that sort of behaviour. I'll leave that question open for any reader interested in taking on that problem. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bb31c6eea41f4adcacf249d39a3798d2"> <div class="comment-author"><a href="https://github.com/dharmaturtle">Alex Nguyen</a></div> <div class="comment-content"> <p> Hi Mark! Thanks for another thought provoking post. </p> <p> I believe you and Hillel are writing <a href="https://en.wikipedia.org/wiki/Characterization_test">characterization tests</a>, which <a href="https://blog.ploeh.dk/2013/04/02/why-trust-tests/">you've mentioned in the past</a>. Namely, you're both using the behavior of existing code to verify the correctness of a refactor. The novel part to me is that Hillel is using code as the test oracle. Your solution serializes the oracle to a static file. The library I use for characterization tests (<a href="https://www.nuget.org/packages/ApprovalTests">ApprovalTests</a>) does this as well. </p> <p> I believe shrinking is impossible when the oracle is a static file. However with Hillel's solution the oracle may be consulted at any time, making shrinking viable. If only there was a practical way to combine the two... </p> </div> <div class="comment-date">2021-01-06 23:01 UTC</div> </div> <div class="comment" id="4aa4188124ca4f4fadf35d6881b9452e"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> A thought provoking post indeed! </p> <blockquote> In F# I'm going to use <a href="https://fscheck.github.io/FsCheck">FsCheck</a>... </blockquote> <p> I think that is a fine choice given the use case laid out in this post. In general though, I think <a href="https://github.com/hedgehogqa/fsharp-hedgehog">Hedgehog</a> is a better property-based testing library. Its killer feature is integrated shrinking, which means that all generators can also shrink and this extra power is essentially free. </p> <p> For the record (because this can be a point of confusion), Haskell has <a href="https://hackage.haskell.org/package/QuickCheck">QuickCheck</a> and <a href="https://hackage.haskell.org/package/hedgehog">(Haskell) Hedgehog</a> while F# has ports from Haskell called <a href="https://fscheck.github.io/FsCheck">FsCheck</a> and <a href="https://github.com/hedgehogqa/fsharp-hedgehog">(FSharp) Hedgehog</a>. </p> <p> <a href="https://twitter.com/jacobstanley">Jacob Stanley</a> gave <a href="https://www.youtube.com/watch?v=AIv_9T0xKEo">this excellent talk at YOW! Lambda Jam 2017</a> that explains the key idea that allows Hedgehog to have integrated shrinking. (Spoiler: A generic type that is invariant in its only type parameter is replaced by a different generic type that is monadic in its only type parameter. API design guided by functional programming for the win!) </p> <blockquote> Normally, when a property-based testing framework like FsCheck detects a counter-example, it'll shrink the counter-example to values that are easier to understand than the original. </blockquote> <p> In my experience, property-based tests written with QuickCheck / FsCheck do not normally shrink. I think this is because of the extra work required to enable shrinking. For an anecdotal example, consider <a href="https://frasertweedale.github.io/blog-fp/posts/2020-03-31-quickcheck-hedgehog.html">this post by Fraser Tweedale</a>. He believed that it would be faster to add (Haskell) Hedgehog as a dependency and create a generator for it than to add shrinking to his existing generator in QuickCheck. </p> <blockquote> In other words, you use version <em>n</em> of <code>f</code> as a test oracle for version <em>n + 1</em>. When iteration <em>n + 1</em> is a function of iteration <em>n</em>, you have a so-called <em>dynamic system</em>, so I think that we can call this technique <em>dynamic test oracles</em>. </blockquote> <p> I am confused by this paragraph. I interpret your word "When" at the start of the second sentence as a common-language synonym for the mathematical word "If". Here is roughly how I understand that paragraph, where <code>A</code> stands for "version / iteration <em>n</em> of <code>f</code>" and <code>B</code> stands for "version / iteration <em>n + 1</em> of <code>f</code>". "<code>A</code> depends on <code>B</code>. If <code>B</code> depends on <code>A</code>, then we have a dynamic system. Therefore, we have a dynamic system." I feel like the paragraph assumes (because it is obvious?) that version / iteration <em>n + 1</em> of <code>f</code> depends on version / iteration <em>n</em> of <code>f</code>. In what sense is that the case? </p> <blockquote> An outstanding issue is whether it's possible to implement shrinking behaviour when tests fail. [...] I'll leave that question open for any reader interested in taking on that problem. </blockquote> <p> I am interested! </p> <p> Quoting Mark and then Alex. </p> <blockquote> <p> Hillel Wayne [...] outlines an approach where you test an iteration of rho code against a 'last known good' snapshot. He uses <code>git worktree</code> to set up a snapshot of the reference implementation. He then writes a property that compares the refactored code's behaviour against the reference. </p> <p> The example code is in <a href="https://www.python.org">Python</a>, which is a language that I don't know. As far as I can tell, it works because Python is 'lightweight' enough that you can load and execute source code directly. I found that the approach makes much sense, but I wondered how it would apply for statically typed, compiled languages. I decided to create a proof of concept in <a href="https://fsharp.org">F#</a>. </p> </blockquote> <blockquote> I believe shrinking is impossible when the oracle is a static file. However with Hillel's solution the oracle may be consulted at any time, making shrinking viable. </blockquote> <p> I want to start by elaborating on this to make sure we are all on the same page. I think of shrinking as involving two parts. On the one hand, we have the "shrink tree", which contains the values to test during the shrinking process. On the other hand, for each input tested, we need to know if the output should cause the test to pass or fail. </p> <p> With Hedgehog, getting a shrink tree would not be too difficult. For a generator with type parameter <code>'a</code>, the current generator API returns a "random" shrink tree of type <code>'a</code> in which the root is an instance <code>a</code> of the type <code>'a</code> and the tree completely depends on <code>a</code>. It should be easy to expose an additional function that accepts inputs of type <code>'a Gen</code> and <code>'a</code> and returns <em>the</em> tree with the given <code>'a</code> as its root. </p> <p> The difficult part is being able to query the test oracle. As Mark said, this seems easy to do in a dynamically-typed language like Python. In contrast, the fundamental issue with a statically-typed language like F# is that the compiled code exists in an assembly and only one assembly of a given name can be loaded in a given process at the same time. </p> <p> This leads me to two ideas for workarounds. First, we could query the test oracle in a different process. I imagine an entry point could be generated that gives direct access to the test oracle. Then the test process could query the test oracle by executing this generated process. Second, we could generate a different assembly that exposes the test oracle. Then the test process could load this generated assembly to query the test oracle. The second approach seems like it would have a faster query time but be harder to implement. The first approach seems easier to implement but would probably have a slower query time. Maybe the query time would be fast enough though, especially if it was only queried when shrinking. </p> <p> But given such a solution, who wants to restrict access to the test oracle only to shrinking? If the test oracle is always available, then there is no need to store input-output pairs. Instead of always checking that the system under test works correctly for a previously selected set of inputs, the property-based test can check that the system under test has the expected behavior for a unique set of inputs each time the property-based test is executed. In my experience, this is the default behavior of a property-based test. </p> <p> One concern that some people might have is the idea of checking into the code repository the binary containing the test oracle. My first though is that <a href="https://blog.ploeh.dk/2014/01/29/nuget-package-restore-considered-harmful/">the size of this is likely so small that it does not matter</a>. My second thought is that the binary containing the test oracle does not have to be included in the repository. Instead, the workflow could be to (1) create the property-based test that uses the compiled test oracle, (2) refactor the system under test, (3) observe that the property-based test still passes, (4) commit the refactored code, and (5) discard the remaining changes, which will delete the property-based test and the compiled test oracle. </p> <p> Instead of completely removing that property-based test, it might be better to leave it there with input-output pairs stored in a file. Then the conversion from that state of the property-based test to the one that uses the compiled test oracle will be much smaller. </p> </div> <div class="comment-date">2021-01-07 19:27 UTC</div> </div> <div class="comment" id="39b559f49ecc426db7b3d9d1518556df"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Alex, thank you for writing. Yes, I think that calling this a Characterisation Test is correct. I wasn't aware of the <em>ApprovalTests</em> library; thank you for mentioning it. </p> <p> When I originally wrote the article, I was under the impression that shrinking might still be possible. I admit, though, that I hadn't thought things through. I think that <a href="#4aa4188124ca4f4fadf35d6881b9452e">Tyson Williams argues convincingly</a> that this isn't possible. </p> </div> <div class="comment-date">2021-01-15 13:42 UTC</div> </div> <div class="comment" id="a6ea5d7cbb75431ba6adf969599c2bd3"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I'm <a href="/2017/09/18/the-test-data-generator-functor#5bd990290ff048c2a7b55b740053831d">well aware of Hedgehog</a>, and I'm keen on the way it works. I rarely use it, however, as it so far doesn't quite seem to have the same degree of 'industrial strength' to it that FsCheck has. Additionally, I find that shrinking is less important in practice than it might seem in theory. </p> <p> I'm not sure that I understand your confusion about the term <em>dynamic</em>. You write: <blockquote> <p> "<code>A</code> depends on <code>B</code>." </p> </blockquote> Why do you write that? I don't think, in the way you've labelled iterations, that <code>A</code> depends on <code>B</code>. </p> <p> When it comes to shrinking, I think that you convincingly argues that it can't be done unless one is able to query the oracle. As long as all you have is a list of test cases, you can't do that... unless, perhaps, you were to also generate and run all the shrunk test cases when you capture the list of test cases... Again, I haven't thought this through, so there may be some obvious gotcha that I'm missing. </p> <p> I would be wary of trying to host the previous iteration in a different process. This is technically possible, but, in .NET at least, quite cumbersome. You'll have to deal with data marshalling and lifetime management of the second process. It was difficult enough in .NET framework back when <em>remoting</em> was a thing; I'm not even sure how one would go about such a problem in .NET Core - particularly if you want it to work on both Windows, Linux, and Mac. HTTP? </p> </div> <div class="comment-date">2021-01-16 13:24 UTC</div> </div> <div class="comment" id="7e4b9abea062491bb5a4040761902b38"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> [Hedgehog] so far doesn't quite seem to have the same degree of 'industrial strength' to it that FsCheck has. </blockquote> <p> That seems reasonable. I haven't used FsCheck, so I wouldn't know myself. Several of us are making many great improvements to F# Hedgehog right now. </p> <blockquote> <p> When it comes to shrinking, I think that you convincingly argues that it can't be done unless one is able to query the oracle. As long as all you have is a list of test cases, you can't do that... unless, perhaps, you were to also generate and run all the shrunk test cases when you capture the list of test cases... Again, I haven't thought this through, so there may be some obvious gotcha that I'm missing. </p> </blockquote> <p> That would be too many test cases. The shrinking process finds two values <code>n</code> and <code>n+1</code> such that the test passes for <code>n</code> and fails for <code>n+1</code>. This can be viewed as a constraint. The objective is to minimize the value of <code>n</code>. The only way to ensure that some value is the minimum is to test all values smaller than it. However, doing so is not practical. Property-based tests uses randomness precisely because it is not practical to test every possible value. </p> <p> Instead, the shrinking process uses binary search as a heuristic to find a value satisfying the constraint that is rather small but not necessarily the smallest. </p> <blockquote> Why do you write that? I don't think, in the way you've labelled iterations, that <code>A</code> depends on <code>B</code>. </blockquote> <p> Ok. I will go slower and ask smaller questions. </p> <blockquote> When iteration <em>n + 1</em> is a function of iteration <em>n</em> [...] </blockquote> <p> Does this phrase have the same meaning if "When" is replaced by "If"? </p> <blockquote> In other words, you use version <em>n</em> of <code>f</code> as a test oracle for version <em>n + 1</em>. When iteration <em>n + 1</em> is a function of iteration <em>n</em>, you have a so-called <em>dynamic system</em>, so I think that we can call this technique <em>dynamic test oracles</em>. </blockquote> <p> I understand how version <em>n</em> of <code>f</code> is being used as a test oracle for version <em>n + 1</em>. In this blog post, in what sense is something of iteration <em>n + 1</em> is a function of iteration <em>n</em>? </p> </div> <div class="comment-date">2021-01-30 16:36 UTC</div> </div> <div class="comment" id="078ef886b5ba4b818cf8909e9c25e83e"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. <blockquote> "Does this phrase have the same meaning if "When" is replaced by "If"?" </blockquote> I'm not sure that there's a big difference, but then, I don't know how you parse that. As Kevlin Henney puts it, <blockquote> <p> "The act of describing a program in unambiguous detail and the act of programming are one and the same." </p> <footer><cite><a href="https://twitter.com/KevlinHenney/status/3361631527">Kevlin Henney</a></cite></footer> </blockquote> It seems to me that you're attempting to parse my prose as though it was an unambiguous description, which it can't be. </p> <p> A dynamic system is a system such that <code>x<sub>t+1</sub> = f(x<sub>t</sub>)</sub></code>, where <code>x<sub>t</sub></code> is the value of <code>x</code> at time <code>t</code>, and <code>x<sub>t+1</sub></code> is the value of <code>x</code> at time <code>t+1</code>. For simplicity, this is the definition of a dynamic system in discrete time. Mathematically, you can also express it in continuous time using calculus. </p> </div> <div class="comment-date">2021-02-02 6:46 UTC</div> </div> <div class="comment" id="3fe354d5505b4dd2aa9c2c0e80cf9002"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> It seems to me that you're attempting to parse my prose as though it was an unambiguous description, which it can't be. </blockquote> <p> Oh, yes. My mistake. I meant to phrase in slightly differently thereby changing it from essentially an impossible question to one that only you can answer. Here is what I meant to ask. </p> <blockquote> Does this phrase have the same meaning <em>to you</em> if "When" is replaced by "If"? </blockquote> <p> No matter though. I simply misunderstood your description / defintion of a dynamical system. I understand now. Thanks for your patience and willingness to explain it to me again. </p> </div> <div class="comment-date">2021-03-25 03:47 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An F# demo of validation with partial data round trip https://blog.ploeh.dk/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip 2020-12-28T09:22:00+00:00 Mark Seemann <div id="post"> <p> <em>An F# port of the previous Haskell proof of concept.</em> </p> <p> This article is part of <a href="/2020/12/14/validation-a-solved-problem">a short article series</a> on <a href="/2018/11/05/applicative-validation">applicative validation</a> with a twist. The twist is that validation, when it fails, should return not only a list of error messages; it should also retain that part of the input that <em>was</em> valid. </p> <p> In the <a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">previous article</a> you saw a <a href="https://www.haskell.org">Haskell</a> proof of concept that demonstrated how to compose the appropriate <a href="/2018/10/01/applicative-functors">applicative functor</a> with a suitable <a href="/2017/11/27/semigroups">semigroup</a> to make validation work as desired. In this article, you'll see how to port that proof of concept to <a href="https://fsharp.org">F#</a>. </p> <h3 id="b2ea11fdb65343b5b60fcf2cffc62b1a"> Data definitions <a href="#b2ea11fdb65343b5b60fcf2cffc62b1a" title="permalink">#</a> </h3> <p> Like in the previous article, we're going to need some types. These are essentially direct translations of the corresponding Haskell types: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Input&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string&nbsp;option;&nbsp;DoB&nbsp;:&nbsp;DateTime&nbsp;option;&nbsp;Address&nbsp;:&nbsp;string&nbsp;option} <span style="color:blue;">type</span>&nbsp;ValidInput&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string;&nbsp;DoB&nbsp;:&nbsp;DateTime;&nbsp;Address&nbsp;:&nbsp;string&nbsp;}</pre> </p> <p> The <code>Input</code> type plays the role of the input we'd like to validate, while <code>ValidInput</code> presents validated data. </p> <p> If you're an F# fan, you can bask in the reality that F# records are terser than Haskell records. I like both languages, so I have mixed feelings about this. </p> <h3 id="2e1b689c57114b259b3b4019f4c3976d"> Computation expression <a href="#2e1b689c57114b259b3b4019f4c3976d" title="permalink">#</a> </h3> <p> Haskell's main workhorse is its type class system. F# doesn't have that, but it has <a href="https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/computation-expressions">computation expressions</a>, which in F# 5 got support for applicative functors. That's just what we need, and it turns out that there isn't a lot of code we have to write to make all of this work. </p> <p> To recap from the Haskell proof of concept: We need a <code>Result</code>-like <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> that returns a tuple for errors. One element of the tuple should be a an <a href="https://en.wikipedia.org/wiki/Endomorphism">endomorphism</a>, which <a href="/2017/11/13/endomorphism-monoid">forms a monoid</a> (and therefore also a semigroup). The other element should be a list of error messages - <a href="/2017/10/10/strings-lists-and-sequences-as-a-monoid">another monoid</a>. In F# terms we'll write it as <code>(('b -&gt; 'b) * 'c list)</code>. </p> <p> That's a tuple, and since <a href="/2017/10/30/tuple-monoids">tuples form monoids when their elements do</a> the <code>Error</code> part of <code>Result</code> <a href="/2017/11/20/monoids-accumulate">supports accumulation</a>. </p> <p> To support an applicative computation expression, we're going to need a a way to merge two results together. This is by far the most complicated piece of code in this article, all six lines of code: </p> <p> <pre><span style="color:blue;">module</span>&nbsp;Result&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;&#39;a&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;,((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;&nbsp;-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;&#39;d&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;,((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;&nbsp;-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;Result&lt;(&#39;a&nbsp;*&nbsp;&#39;d),((&#39;b&nbsp;-&gt;&nbsp;&#39;b)&nbsp;*&nbsp;&#39;c&nbsp;list)&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;merge&nbsp;x&nbsp;y&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;x,&nbsp;y&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;xres,&nbsp;Ok&nbsp;yres&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;(xres,&nbsp;yres) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;(f,&nbsp;e1s),&nbsp;Error&nbsp;(g,&nbsp;e2s)&nbsp;&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(f&nbsp;&gt;&gt;&nbsp;g,&nbsp;e2s&nbsp;@&nbsp;e1s) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Error&nbsp;e,&nbsp;Ok&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;e &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Ok&nbsp;_,&nbsp;Error&nbsp;e&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;e</pre> </p> <p> The <code>merge</code> function composes two input results together. The results have <code>Ok</code> types called <code>'a</code> and <code>'d</code>, and if they're both <code>Ok</code> values, the return value is an <code>Ok</code> tuple of <code>'a</code> and <code>'d</code>. </p> <p> If one of the results is an <code>Error</code> value, it beats an <code>Ok</code> value. The only moderately complex operations is when both are <code>Error</code> values. </p> <p> Keep in mind that an <code>Error</code> value in this instance contains a tuple of the type <code>(('b -&gt; 'b) * 'c list)</code>. The first element is an endomorphism <code>'b -&gt; 'b</code> and the other element is a list. The <code>merge</code> function composes the endomorphism <code>f</code> and <code>g</code> by standard function composition (the <code>&gt;&gt;</code> operator), and concatenates the lists with the standard <code>@</code> list concatenation operator. </p> <p> Because I'm emulating how the <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">original forum post</a>'s code behaves, I'm concatenating the two lists with the rightmost going before the leftmost. It doesn't make any other difference than determining the order of the error list. </p> <p> With the <code>merge</code> function in place, the computation expression is a simple matter: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidationBuilder&nbsp;()&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;_.BindReturn&nbsp;(x,&nbsp;f)&nbsp;=&nbsp;Result.map&nbsp;f&nbsp;x &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">member</span>&nbsp;_.MergeSources&nbsp;(x,&nbsp;y)&nbsp;=&nbsp;Result.merge&nbsp;x&nbsp;y</pre> </p> <p> The last piece is a <code>ValidationBuilder</code> value: </p> <p> <pre>[&lt;AutoOpen&gt;] <span style="color:blue;">module</span>&nbsp;ComputationExpressions&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;validation&nbsp;=&nbsp;ValidationBuilder&nbsp;()</pre> </p> <p> Now, whenever you use the <code>validation</code> computation expression, you get the desired functionality. </p> <h3 id="948b93fa6a1d47beac150180769731eb"> Validators <a href="#948b93fa6a1d47beac150180769731eb" title="permalink">#</a> </h3> <p> Before we can compose some validation functions, we'll need to have some validators in place. These are straightforward translations of the Haskell validation functions, starting with the name validator: </p> <p> <pre><span style="color:green;">//&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;string,((Input&nbsp;-&gt;&nbsp;Input)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateName&nbsp;({&nbsp;Name&nbsp;=&nbsp;name&nbsp;}&nbsp;:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;name&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;n&nbsp;<span style="color:blue;">when</span>&nbsp;n.Length&nbsp;&gt;&nbsp;3&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;n &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">fun</span>&nbsp;(args&nbsp;:&nbsp;Input)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;args&nbsp;<span style="color:blue;">with</span>&nbsp;Name&nbsp;=&nbsp;None&nbsp;}), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;no&nbsp;bob&nbsp;and&nbsp;toms&nbsp;allowed&quot;</span>]) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;name&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> When the name is too short, the endomorphism resets the <code>Name</code> field to <code>None</code>. </p> <p> The date-of-birth validation function works the same way: </p> <p> <pre><span style="color:green;">//&nbsp;DateTime&nbsp;-&gt;&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;DateTime,((Input&nbsp;-&gt;&nbsp;Input)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateDoB&nbsp;(now&nbsp;:&nbsp;DateTime)&nbsp;({&nbsp;DoB&nbsp;=&nbsp;dob&nbsp;}&nbsp;:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;dob&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;d&nbsp;<span style="color:blue;">when</span>&nbsp;d&nbsp;&gt;&nbsp;now.AddYears&nbsp;-12&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;d &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;_&nbsp;<span style="color:blue;">-&gt;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">fun</span>&nbsp;(args&nbsp;:&nbsp;Input)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;{&nbsp;args&nbsp;<span style="color:blue;">with</span>&nbsp;DoB&nbsp;=&nbsp;None&nbsp;}), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;get&nbsp;off&nbsp;my&nbsp;lawn&quot;</span>]) &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;dob&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> Again, like in the Haskell proof of concept, instead of calling <code>DateTime.Now</code> from within the function, I'm passing <code>now</code> as an argument to keep the function <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a>. </p> <p> The address validation concludes the set of validators: </p> <p> <pre><span style="color:green;">//&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;string,((&#39;a&nbsp;-&gt;&nbsp;&#39;a)&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateAddress&nbsp;({&nbsp;Address&nbsp;=&nbsp;address&nbsp;}:&nbsp;Input)&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">match</span>&nbsp;address&nbsp;<span style="color:blue;">with</span> &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;Some&nbsp;a&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Ok&nbsp;a &nbsp;&nbsp;&nbsp;&nbsp;|&nbsp;None&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;Error&nbsp;(id,&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>])</pre> </p> <p> The inferred endomorphism type here is the more general <code>'a -&gt; 'a</code>, but it's compatible with <code>Input -&gt; Input</code>. </p> <h3 id="836690a83da24da492f70c8eb756a52d"> Composition <a href="#836690a83da24da492f70c8eb756a52d" title="permalink">#</a> </h3> <p> All three functions have compatible <code>Error</code> types, so they ought to compose with the applicative computation expression to produce the desired behaviour: </p> <p> <pre><span style="color:green;">//&nbsp;DateTime&nbsp;-&gt;&nbsp;Input&nbsp;-&gt;&nbsp;Result&lt;ValidInput,(Input&nbsp;*&nbsp;string&nbsp;list)&gt;</span> <span style="color:blue;">let</span>&nbsp;validateInput&nbsp;now&nbsp;args&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;validation&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let!</span>&nbsp;name&nbsp;=&nbsp;validateName&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">and!</span>&nbsp;dob&nbsp;=&nbsp;validateDoB&nbsp;now&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">and!</span>&nbsp;address&nbsp;=&nbsp;validateAddress&nbsp;args &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;{&nbsp;Name&nbsp;=&nbsp;name;&nbsp;DoB&nbsp;=&nbsp;dob;&nbsp;Address&nbsp;=&nbsp;address&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;|&gt;&nbsp;Result.mapError&nbsp;(<span style="color:blue;">fun</span>&nbsp;(f,&nbsp;msgs)&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;f&nbsp;args,&nbsp;msgs)</pre> </p> <p> The <code>validation</code> expression alone produces a <code>Result&lt;ValidInput,((Input -&gt; Input) * string list)&gt;</code> value. To get an <code>Input</code> value in the <code>Error</code> tuple, we need to 'run' the <code>Input -&gt; Input</code> endomorphism. The <code>validateInput</code> function does that by applying the endomorphism <code>f</code> to <code>args</code> when mapping the error with <code>Result.mapError</code>. </p> <h3 id="df2c48e822ad48ce916e29da8638563a"> Tests <a href="#df2c48e822ad48ce916e29da8638563a" title="permalink">#</a> </h3> <p> To test that the <code>validateInput</code> works as intended, I first copied all the code from the original forum post. I then wrote eight <a href="https://en.wikipedia.org/wiki/Characterization_test">characterisation tests</a> against that code to make sure that I could reproduce the desired functionality. </p> <p> I then wrote a parametrised test against the new function: </p> <p> <pre>[&lt;Theory;&nbsp;ClassData(typeof&lt;ValidationTestCases&gt;)&gt;] <span style="color:blue;">let</span>&nbsp;``Validation&nbsp;works``&nbsp;input&nbsp;expected&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;now&nbsp;=&nbsp;DateTime.Now &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;validateInput&nbsp;now&nbsp;input &nbsp;&nbsp;&nbsp;&nbsp;expected&nbsp;=!&nbsp;actual</pre> </p> <p> The <code>ValidationTestCases</code> class is defined like this: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidationTestCases&nbsp;()&nbsp;<span style="color:blue;">as</span>&nbsp;this&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">inherit</span>&nbsp;TheoryData&lt;Input,&nbsp;Result&lt;ValidInput,&nbsp;Input&nbsp;*&nbsp;string&nbsp;list&gt;&gt;&nbsp;()</pre> </p> <p> This class produces a set of test cases, where each test case contains an <code>input</code> value and the <code>expected</code> output. To define the test cases, I copied the eight characterisation tests I'd already produced and adjusted them so that they fit the simpler API of the <code>validateInput</code> function. Here's a few examples: </p> <p> <pre><span style="color:blue;">let</span>&nbsp;eightYearsAgo&nbsp;=&nbsp;DateTime.Now.AddYears&nbsp;-8 <span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;None&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Error&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;None&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>])) <span style="color:blue;">do</span>&nbsp;this.Add&nbsp;( &nbsp;&nbsp;&nbsp;&nbsp;{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Ok&nbsp;({&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Alice&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}))</pre> </p> <p> The first case expects an <code>Error</code> value because the <code>Input</code> value has no address. The other test case expects an <code>Ok</code> value because all input is fine. </p> <p> I copied all eight characterisation tests over, so now I have those eight tests, as well as the modified eight tests for the applicative-based API shown here. All sixteen tests pass. </p> <h3 id="5a20682ce9494011b78d5a5f9c2c9bad"> Conclusion <a href="#5a20682ce9494011b78d5a5f9c2c9bad" title="permalink">#</a> </h3> <p> I find this solution to the problem elegant. It's always satisfying when you can implement what at first glance looks like custom behaviour using <a href="/2017/10/04/from-design-patterns-to-category-theory">universal abstractions</a>. </p> <p> Besides the aesthetic value, I also believe that this keeps a team more productive. These concepts of monoids, semigroups, applicative functors, and so on, are concepts that you only have to learn once. Once you know them, you'll recognise them when you run into them. This means that there's less code to understand. </p> <p> An ad-hoc implementation as the original forum post suggested (even though it looked quite decent) always puts the onus on a maintenance developer to read and understand even more one-off infrastructure code. </p> <p> With an architecture based on universal abstractions and well-documented language features, a functional programmer that knows these things will be able to pick up what's going on without much trouble. Specifically, (s)he will recognise that this is 'just' applicative validation with a twist. </p> <p> This article is the December 28 entry in the <a href="https://sergeytihon.com/2020/10/22/f-advent-calendar-in-english-2020">F# Advent Calendar in English 2020</a>. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. A Haskell proof of concept of validation with partial data round trip https://blog.ploeh.dk/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip 2020-12-21T06:54:00+00:00 Mark Seemann <div id="post"> <p> <em>Which Semigroup best addresses the twist in the previous article?</em> </p> <p> This article is part of <a href="/2020/12/14/validation-a-solved-problem">a short article series</a> on applicative validation with a twist. The twist is that validation, when it fails, should return not only a list of error messages; it should also retain that part of the input that <em>was</em> valid. </p> <p> In this article, I'll show how I did a quick proof of concept in <a href="https://www.haskell.org">Haskell</a>. </p> <h3 id="9417153ff45d4188a65470fa2d67ea2e"> Data definitions <a href="#9417153ff45d4188a65470fa2d67ea2e" title="permalink">#</a> </h3> <p> You can't use the regular <code>Either</code> instance of <code>Applicative</code> for validation because it short-circuits on the first error. In other words, you can't collect multiple error messages, even if the input has multiple issues. Instead, you need a custom <code>Applicative</code> instance. You can <a href="/2018/11/05/applicative-validation">easily write such an instance</a> yourself, but there are a couple of libraries that already do this. For this prototype, I chose the <a href="https://hackage.haskell.org/package/validation">validation</a> package. </p> <p> <pre><span style="color:blue;">import</span>&nbsp;Data.Bifunctor <span style="color:blue;">import</span>&nbsp;Data.Time <span style="color:blue;">import</span>&nbsp;Data.Semigroup <span style="color:blue;">import</span>&nbsp;Data.Validation </pre> </p> <p> Apart from importing <code>Data.Validation</code>, I also need a few other imports for the proof of concept. All of them are well-known. I used no language extensions. </p> <p> For the proof of concept, the input is a triple of a name, a date of birth, and an address: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;Input&nbsp;=&nbsp;Input&nbsp;{ &nbsp;&nbsp;<span style="color:#2b91af;">inputName</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:#2b91af;">String</span>, &nbsp;&nbsp;<span style="color:#2b91af;">inputDoB</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:blue;">Day</span>, &nbsp;&nbsp;<span style="color:#2b91af;">inputAddress</span>&nbsp;::&nbsp;<span style="color:#2b91af;">Maybe</span>&nbsp;<span style="color:#2b91af;">String</span>&nbsp;} &nbsp;&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) </pre> </p> <p> The goal is actually to <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">parse (not validate)</a> <code>Input</code> into a safer data type: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;ValidInput&nbsp;=&nbsp;ValidInput&nbsp;{ &nbsp;&nbsp;<span style="color:#2b91af;">validName</span>&nbsp;::&nbsp;<span style="color:#2b91af;">String</span>, &nbsp;&nbsp;<span style="color:#2b91af;">validDoB</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>, &nbsp;&nbsp;<span style="color:#2b91af;">validAddress</span>&nbsp;::&nbsp;<span style="color:#2b91af;">String</span>&nbsp;} &nbsp;&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) </pre> </p> <p> If parsing/validation fails, the output should report a collection of error messages <em>and</em> return the <code>Input</code> value with any valid data retained. </p> <h3 id="4ce26b2031084212a5fbab83dd848d4b"> Looking for a Semigroup <a href="#4ce26b2031084212a5fbab83dd848d4b" title="permalink">#</a> </h3> <p> My hypothesis was that validation, even with that twist, can be implemented elegantly with an <code>Applicative</code> instance. The <em>validation</em> package defines its <code>Validation</code> data type such that it's an <code>Applicative</code> instance as long as its error type is a <code>Semigroup</code> instance: </p> <p> <pre><span style="color:blue;">Semigroup</span> err =&gt; <span style="color:blue;">Applicative</span> (<span style="color:blue;">Validation</span> err)</pre> </p> <p> The question is: which <code>Semigroup</code> can we use? </p> <p> Since we need to return <em>both</em> a list of error messages <em>and</em> a modified <code>Input</code> value, it sounds like we'll need a product type of some sorts. A tuple will do; something like <code>(Input, [String])</code>. Is that a <code>Semigroup</code> instance, though? </p> <p> Tuples only form semigroups if both elements give rise to a semigroup: </p> <p> <pre>(<span style="color:blue;">Semigroup</span> a, <span style="color:blue;">Semigroup</span> b) =&gt; <span style="color:blue;">Semigroup</span> (a, b)</pre> </p> <p> The second element of my candidate is <code>[String]</code>, which is fine. Lists are <code>Semigroup</code> instances. But what about <code>Input</code>? Can we somehow combine two <code>Input</code> values into one? It's not entirely clear how we should do that, so that doesn't seem too promising. </p> <p> What we need to do, however, is to take the original <code>Input</code> and modify it by (optionally) resetting one or more fields. In other words, a series of functions of the type <code>Input -&gt; Input</code>. Aha! There's the semigroup we need: <a href="https://hackage.haskell.org/package/base/docs/Data-Semigroup.html#t:Endo"><code>Endo Input</code></a>. </p> <p> So the <code>Semigroup</code> instance we need is <code>(<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code>, and the validation output should be of the type <code><span style="color:blue;">Validation</span> (<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>]) a</code>. </p> <h3 id="5da9d89ac8414ad0bc9ebe322b831390"> Validators <a href="#5da9d89ac8414ad0bc9ebe322b831390" title="permalink">#</a> </h3> <p> Cool, we can now implement the validation logic; a function for each field, starting with the name: </p> <p> <pre><span style="color:#2b91af;">validateName</span>&nbsp;::&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span> validateName&nbsp;(Input&nbsp;(Just&nbsp;name)&nbsp;_&nbsp;_)&nbsp;|&nbsp;<span style="color:blue;">length</span>&nbsp;name&nbsp;&gt;&nbsp;3&nbsp;=&nbsp;Success&nbsp;name validateName&nbsp;(Input&nbsp;(Just&nbsp;_)&nbsp;_&nbsp;_)&nbsp;= &nbsp;&nbsp;Failure&nbsp;(Endo&nbsp;$&nbsp;\x&nbsp;-&gt;&nbsp;x&nbsp;{&nbsp;inputName&nbsp;=&nbsp;Nothing&nbsp;},&nbsp;[<span style="color:#a31515;">&quot;no&nbsp;bob&nbsp;and&nbsp;toms&nbsp;allowed&quot;</span>]) validateName&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;name&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> This function reproduces the validation logic implied by <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">the forum question that started it all</a>. Notice, particularly, that when the name is too short, the endomorphism resets <code>inputName</code> to <code>Nothing</code>. </p> <p> The date-of-birth validation function works the same way: </p> <p> <pre><span style="color:#2b91af;">validateDoB</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:blue;">Day</span> validateDoB&nbsp;now&nbsp;(Input&nbsp;_&nbsp;(Just&nbsp;dob)&nbsp;_)&nbsp;|&nbsp;addGregorianYearsRollOver&nbsp;(-12)&nbsp;now&nbsp;&lt;&nbsp;dob&nbsp;= &nbsp;&nbsp;Success&nbsp;dob validateDoB&nbsp;_&nbsp;(Input&nbsp;_&nbsp;(Just&nbsp;_)&nbsp;_)&nbsp;= &nbsp;&nbsp;Failure&nbsp;(Endo&nbsp;$&nbsp;\x&nbsp;-&gt;&nbsp;x&nbsp;{&nbsp;inputDoB&nbsp;=&nbsp;Nothing&nbsp;},&nbsp;[<span style="color:#a31515;">&quot;get&nbsp;off&nbsp;my&nbsp;lawn&quot;</span>]) validateDoB&nbsp;_&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;dob&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> Again, the validation logic is inferred from the forum question, although I found it better keep the function pure by requiring a <code>now</code> argument. </p> <p> The address validation is the simplest of the three validators: </p> <p> <pre><span style="color:#2b91af;">validateAddress</span>&nbsp;::&nbsp;<span style="color:blue;">Monoid</span>&nbsp;a&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(a,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span> validateAddress&nbsp;(Input&nbsp;_&nbsp;_&nbsp;(Just&nbsp;a))&nbsp;=&nbsp;Success&nbsp;a validateAddress&nbsp;_&nbsp;=&nbsp;Failure&nbsp;(mempty,&nbsp;[<span style="color:#a31515;">&quot;add1&nbsp;is&nbsp;required&quot;</span>]) </pre> </p> <p> This one's return type is actually more general than required, since I used <code>mempty</code> instead of <code>Endo id</code>. This means that it actually works for any <code>Monoid a</code>, which also includes <code>Endo Input</code>. </p> <h3 id="6d5502d178f143d58a4d3c5bef7c1f05"> Composition <a href="#6d5502d178f143d58a4d3c5bef7c1f05" title="permalink">#</a> </h3> <p> All three functions return <code><span style="color:blue;">Validation</span> (<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code>, which has an <code>Applicative</code> instance. This means that we should be able to compose them together to get the behaviour we're looking for: </p> <p> <pre><span style="color:#2b91af;">validateInput</span>&nbsp;::&nbsp;<span style="color:blue;">Day</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:#2b91af;">Either</span>&nbsp;(<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:blue;">ValidInput</span> validateInput&nbsp;now&nbsp;args&nbsp;= &nbsp;&nbsp;toEither&nbsp;$ &nbsp;&nbsp;first&nbsp;(first&nbsp;(`appEndo`&nbsp;args))&nbsp;$ &nbsp;&nbsp;ValidInput&nbsp;&lt;$&gt;&nbsp;validateName&nbsp;args&nbsp;&lt;*&gt;&nbsp;validateDoB&nbsp;now&nbsp;args&nbsp;&lt;*&gt;&nbsp;validateAddress&nbsp;args </pre> </p> <p> That compiles, so it probably works. </p> <h3 id="9ed5a5fe379244a3bd1e9206a79a1ea9"> Sanity check <a href="#9ed5a5fe379244a3bd1e9206a79a1ea9" title="permalink">#</a> </h3> <p> Still, it'd be prudent to check. Since this is only a proof of concept, I'm not going to set up a test suite. Instead, I'll just start GHCi for some ad-hoc testing: </p> <p> <pre>λ&gt; now &lt;- localDay &lt;&amp;&gt; zonedTimeToLocalTime &lt;&amp;&gt; getZonedTime λ&gt; validateInput now &amp; Input Nothing Nothing Nothing Left (Input {inputName = Nothing, inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["name is required","dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Bob") Nothing Nothing Left (Input {inputName = Nothing, inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["no bob and toms allowed","dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") Nothing Nothing Left (Input {inputName = Just "Alice", inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["dob is required","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2002 10 12) Nothing Left (Input {inputName = Just "Alice", inputDoB = Nothing, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["get off my lawn","add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2012 4 21) Nothing Left (Input {inputName = Just "Alice", inputDoB = Just 2012-04-21, inputAddress = Nothing}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;["add1 is required"]) λ&gt; validateInput now &amp; Input (Just "Alice") (Just &amp; fromGregorian 2012 4 21) (Just "x") Right (ValidInput {validName = "Alice", validDoB = 2012-04-21, validAddress = "x"})</pre> </p> <p> In order to make the output more readable, I've manually edited the GHCi session by adding line breaks to the output. </p> <p> It looks like it's working like it's supposed to. Only the last line successfully parses the input and returns a <code>Right</code> value. </p> <h3 id="8bbb1d8ca355495b95e5e5ed85a924f4"> Conclusion <a href="#8bbb1d8ca355495b95e5e5ed85a924f4" title="permalink">#</a> </h3> <p> Before I started this proof of concept, I had an inkling of the way this would go. Instead of making the prototype in <a href="https://fsharp.org">F#</a>, I found it more productive to do it in Haskell, since Haskell enables me to compose things together. I particularly appreciate how a composition of types like <code>(<span style="color:blue;">Endo Input</span>, [<span style="color:#2b91af;">String</span>])</code> is automatically a <code>Semigroup</code> instance. I don't have to do anything. That makes the language great for prototyping things like this. </p> <p> Now that I've found the appropriate semigroup, I know how to convert the code to F#. That's in the next article. </p> <p> <strong>Next:</strong> <a href="/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip">An F# demo of validation with partial data round trip</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="0ebea2f4d9c54072a5bb0c093a63fe14"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Great work and excellent post. I just had a few clarification quesitons. </p> <blockquote> <p> ...But what about <code>Input</code>? Can we somehow combine two <code>Input</code> values into one? It's not entirely clear how we should do that, so that doesn't seem too promising. </p> <p> What we need to do, however, is to take the original <code>Input</code> and modify it by (optionally) resetting one or more fields. In other words, a series of functions of the type <code>Input -&gt; Input</code>. Aha! There's the semigroup we need: <code>Endo Input</code>. </p> </blockquote> <p> How rhetorical are those questions? Whatever the case, I will take the bait. </p> <p> Any product type forms a semigroup if all of its elements do. You explicitly stated this for tuples of length 2; it also holds for records such as <code>Input</code>. Each field on that record has type <code> Maybe a</code> for some <code>a</code>, so it suffices to select a semigroup involving <code>Maybe a</code>. There are few different semigropus involving <code>Maybe</code> that have different functions. </p> <p> I think the most common semigroup for <code>Maybe a</code> has the function that returns the first <code>Just _</code> if one exists or else returns <code>Nothing</code>. Combining that with <code>Nothing</code> as the identity element gives the monoid that is typically associated with <code>Maybe a</code> (and I know by the name monoidal plus). Another monoid, and therefore a semigroup, is to return the last <code>Just _</code> instead of the first. </p> <p> Instead of the having a preference for <code>Just _</code>, the function could have a preference for <code>Nothing</code>. As before, when both inputs are <code>Just _</code>, the output could be either of the inputs. </p> <p> I think either of those last two semigroups will achieved the desired behavior in the problem at hand. Your code never replaces an instace of <code>Just a</code> with a different instance, so we don't need a preference for some input when they are both <code>Just _</code>. </p> <p> In the end though, I think the semigroup you derived from <code>Endo</code> leads to simpler code. </p> <p> At the end of the type signature for <code>validateName</code> / <code>validateDoB</code> / <code>validateAddress</code>, what does <code>String</code> / <code>Day</code> / <code>String</code> mean? </p> <p> Why did you pass all three arguments into every parsing/validation function? I think it is a bit simpler to only pass in the needed argument. Maybe you thought this was good enough for prototype code. </p> <p> Why did you use <code>add1</code> in your error message instead of <code>address</code>? Was it only for prototype code to make the message a bit shorter? </p> </div> <div class="comment-date">2020-12-21 14:21 UTC</div> </div> <div class="comment" id="510b9be50c1b43c18973008b89d2da38"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. The semigroup you suggest, I take it, would look something like this: </p> <p> <pre><span style="color:blue;">newtype</span>&nbsp;Perhaps&nbsp;a&nbsp;=&nbsp;Perhaps&nbsp;{&nbsp;runPerhaps&nbsp;::&nbsp;Maybe&nbsp;&nbsp;a&nbsp;}&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>) <span style="color:blue;">instance</span>&nbsp;<span style="color:blue;">Semigroup</span>&nbsp;(<span style="color:blue;">Perhaps</span>&nbsp;a)&nbsp;<span style="color:blue;">where</span> &nbsp;&nbsp;Perhaps&nbsp;Nothing&nbsp;&lt;&gt;&nbsp;_&nbsp;=&nbsp;Perhaps&nbsp;Nothing &nbsp;&nbsp;_&nbsp;&lt;&gt;&nbsp;Perhaps&nbsp;Nothing&nbsp;=&nbsp;Perhaps&nbsp;Nothing &nbsp;&nbsp;Perhaps&nbsp;(Just&nbsp;x)&nbsp;&lt;&gt;&nbsp;_&nbsp;=&nbsp;Perhaps&nbsp;(Just&nbsp;x)</pre> </p> <p> That might work, but it's an atypical semigroup. I <em>think</em> that it's lawful - at least, I can't come up with a counterexample against associativity. It seems reminiscent of Boolean <em>and</em> (the <em>All</em> monoid), but it isn't a monoid, as far as I can tell. </p> <p> Granted, a <code>Monoid</code> constraint isn't required to make the validation code work, but following the <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">principle of least surprise</a>, I still think that picking a well-known semigroup such as <code>Endo</code> is preferable. </p> <p> Regarding your second question, the type signature of e.g. <code>validateName</code> is: </p> <p> <pre><span style="color:#2b91af;">validateName</span>&nbsp;::&nbsp;<span style="color:blue;">Input</span>&nbsp;<span style="color:blue;">-&gt;</span>&nbsp;<span style="color:blue;">Validation</span>&nbsp;(<span style="color:blue;">Endo</span>&nbsp;<span style="color:blue;">Input</span>,&nbsp;[<span style="color:#2b91af;">String</span>])&nbsp;<span style="color:#2b91af;">String</span></pre> </p> <p> Like <code>Either</code>, <code>Validation</code> has two type arguments: <code>err</code> and <code>a</code>; it's defined as <code>data Validation err a</code>. In the above function type, the return value is a <code>Validation</code> value where the <code>err</code> type is <code>(Endo Input, [String])</code> and <code>a</code> is <code>String</code>. </p> <p> All three validation functions share a common <code>err</code> type: <code>(Endo Input, [String])</code>. On the other hand, they return various <code>a</code> types: <code>String</code>, <code>Day</code>, and <code>String</code>, respectively. </p> <p> Regarding your third question, I could also have defined the functions so that they would only have taken the values they'd need to validate. That would better fit <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>, so I should probably have done that... </p> <p> As for the last question, I was just following the 'spec' implied by <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">the original forum question</a>. </p> </div> <div class="comment-date">2020-12-22 15:05 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Validation, a solved problem? https://blog.ploeh.dk/2020/12/14/validation-a-solved-problem 2020-12-14T08:28:00+00:00 Mark Seemann <div id="post"> <p> <em>A validation problem with a twist.</em> </p> <p> Until recently, I thought that data validation was a solved problem: <a href="/2018/11/05/applicative-validation">Use an applicative functor</a>. I then encountered <a href="https://forums.fsharp.org/t/thoughts-on-input-validation-pattern-from-a-noob/1541">a forum question</a> that for a few minutes shook my faith. </p> <p> After brief consideration, though, I realised that all is good. Validation, even with a twist, is successfully modelled with an <a href="/2018/10/01/applicative-functors">applicative functor</a>. Faith in computer science restored. </p> <h3 id="891801e3802f49d8a51b56bebaacecb9"> The twist <a href="#891801e3802f49d8a51b56bebaacecb9" title="permalink">#</a> </h3> <p> Usually, when you see a demo of applicative validation, the result of validating is one of two: either <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">a parsed result</a>, or a collection of error messages. </p> <p> <pre>λ&gt; validateReservation $ ReservationJson "2017-06-30 19:00:00+02:00" 4 "Jane Doe" "j@example.com" Validation (Right (Reservation { &nbsp;&nbsp;&nbsp;&nbsp;reservationDate = 2017-06-30 19:00:00 +0200, &nbsp;&nbsp;&nbsp;&nbsp;reservationQuantity = 4, &nbsp;&nbsp;&nbsp;&nbsp;reservationName = "Jane Doe", &nbsp;&nbsp;&nbsp;&nbsp;reservationEmail = "j@example.com"})) λ&gt; validateReservation $ ReservationJson "2017/14/12 6pm" 4.1 "Jane Doe" "jane.example.com" Validation (Left ["Not a date.","Not a positive integer.","Not an email address."]) λ&gt; validateReservation $ ReservationJson "2017-06-30 19:00:00+02:00" (-3) "Jane Doe" "j@example.com" Validation (Left ["Not a positive integer."])</pre> </p> <p> (Example from <a href="/2018/11/05/applicative-validation">Applicative validation</a>.) </p> <p> What if, instead, you're displaying an input form? When users enter data, you want to validate it. Imagine, for the rest of this short series of articles that the input form has three fields: <em>name</em>, <em>date of birth</em>, and <em>address</em>. Each piece of data has associated validation rules. </p> <p> If you enter a valid name, but an invalid date of birth, you want to clear the input form's date of birth, but not the name. It's such a bother for a user having to retype valid data just because a single field turned out to be invalid. </p> <p> Imagine, for example, that you want to bind the form to a data model like this <a href="https://fsharp.org">F#</a> record type: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;Input&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string&nbsp;option;&nbsp;DoB&nbsp;:&nbsp;DateTime&nbsp;option;&nbsp;Address&nbsp;:&nbsp;string&nbsp;option}</pre> </p> <p> Each of these three fields is optional. We'd like validation to work in the following way: If validation fails, the function should return <em>both</em> a list of error messages, and <em>also</em> the <code>Input</code> object, with valid data retained, but invalid data cleared. </p> <p> One of the rules implied in the forum question is that names must be more than three characters long. Thus, input like this is invalid: </p> <p> <pre>{&nbsp;Name&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;Tom&quot;</span>;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}</pre> </p> <p> Both the <code>DoB</code> and <code>Address</code> fields, however, are valid, so, along with error messages, we'd like our validation function to return a partially wiped <code>Input</code> value: </p> <p> <pre>{&nbsp;Name&nbsp;=&nbsp;None;&nbsp;DoB&nbsp;=&nbsp;Some&nbsp;eightYearsAgo;&nbsp;Address&nbsp;=&nbsp;Some&nbsp;<span style="color:#a31515;">&quot;x&quot;</span>&nbsp;}</pre> </p> <p> Notice that both <code>DoB</code> and <code>Address</code> field values are retained, while <code>Name</code> has been reset. </p> <p> A final requirement: If validation succeeds, the return value should be a <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate">parsed value that captures that validation took place</a>: </p> <p> <pre><span style="color:blue;">type</span>&nbsp;ValidInput&nbsp;=&nbsp;{&nbsp;Name&nbsp;:&nbsp;string;&nbsp;DoB&nbsp;:&nbsp;DateTime;&nbsp;Address&nbsp;:&nbsp;string&nbsp;}</pre> </p> <p> That requirement is straightforward. That's how you'd usually implement application validation. It's the partial data round-trip that seems to throw a spanner in the works. </p> <p> How should we model such validation? </p> <h3 id="534fcc2d66f242a0ba10a9aca7827276"> Theory, applied <a href="#534fcc2d66f242a0ba10a9aca7827276" title="permalink">#</a> </h3> <p> There's a subculture of functional programming that draws heavily on <a href="https://en.wikipedia.org/wiki/Category_theory">category theory</a>. This is most prevalent in <a href="https://www.haskell.org">Haskell</a>. I've been studying category theory in an attempt to understand what it's all about. I even wrote <a href="/2017/10/04/from-design-patterns-to-category-theory">a substantial article series</a> about some design patterns and how they relate to theory. </p> <p> One thing I learned <em>after</em> I'd named that article series is that most of the useful theoretical concepts come from <a href="https://en.wikipedia.org/wiki/Abstract_algebra">abstract algebra</a>, with the possible exception of monads. </p> <p> People often ask me: does all that theory have any practical use? </p> <p> Yes, it does, as it turns out. It did, for example, enable me to identify a solution to the above twist in five to ten minutes. </p> <p> It's a discussion that I often have, particularly with the always friendly F# community. <em>Do you have to understand <a href="/2018/03/22/functors">functors</a>, monads, etcetera to be a productive F# developer?</em> </p> <p> To anyone who wants to learn F# I'd respond: Don't worry about that at the gate. Find a good learning resource and dive right in. It's a friendly language that you can learn gradually. </p> <p> Sooner or later, though, you'll run into knotty problems that you may struggle to address. I've seen this enough times that it looks like a pattern. The present forum question is just one example. A beginner or intermediate F# programmer will typically attempt to solve the problem in an ad-hoc manner that may or may not be easy to maintain. (The solution proposed by the author of that forum question doesn't, by the way, look half bad.) </p> <p> To be clear: there's nothing wrong with being a beginner. I was once a beginner programmer, and I'm <em>still</em> a beginner in multiple ways. What I'm trying to argue here is that there <em>is</em> value in knowing theory. With my knowledge of abstract algebra and how it applies to functional programming, it didn't take me long to identify a solution. I'll get to that later. </p> <p> Before I outline a solution, I'd like to round off the discussion of applied theory. That question about monads comes up a lot. <em>Do I have to understand functors, monads, etcetera to be a good F# developer?</em> </p> <p> I think it's like asking <em>Do I have to understand polymorphism, design patterns, the <a href="https://en.wikipedia.org/wiki/SOLID">SOLID principles</a>, etcetera to be a good object-oriented programmer?</em> </p> <p> Those are typically not the first topics people are taught about OOD. I would assert, however, that understanding such topics do help. They may not be required to get started with OOP, but knowing them makes you a better programmer. </p> <p> I think the same is true for functional programming. It's just a different skill set that makes you better in that paradigm. </p> <h3 id="78fdc446770a4ec48b556e0826a59ce9"> Solution outline <a href="#78fdc446770a4ec48b556e0826a59ce9" title="permalink">#</a> </h3> <p> When you know a bit of theory, you may know that validation can be implemented with an applicative sum type like <a href="/2019/01/14/an-either-functor">Either</a> (AKA <em>Option</em>), with one extra requirement. </p> <p> Either <a href="/2019/01/07/either-bifunctor">has two dimensions</a>, <em>left</em> or <em>right</em> (<em>success</em> or <em>failure</em>, <em>ok</em> or <em>error</em>, etcetera). The applicative nature of it already supplies a way to compose the successes, but what if there's more than one validation error? </p> <p> In my <a href="/2018/11/05/applicative-validation">article about applicative validation</a> I showed how to collect multiple error messages in a list. Lists, however, <a href="/2017/10/10/strings-lists-and-sequences-as-a-monoid">form a monoid</a>, so I typed the validation API to be that flexible. </p> <p> In fact, all you need is a <a href="/2017/11/27/semigroups">semigroup</a>. When I wrote the article on applicative validation, Haskell's <code>Semigroup</code> type class wasn't yet a supertype of <code>Monoid</code>, and I (perhaps without sufficient contemplation) just went with <code>Monoid</code>. </p> <p> What remains is that applicative validation can collect errors for <em>any</em> semigroup of errors. All we need to solve the above validation problem with a twist, then, is to identify a suitable semigroup. </p> <p> I don't want to give away everything in this article, so I'm going to leave you with this cliffhanger. Which semigroup solves the problem? Read on. <ul> <li><a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">A Haskell proof of concept of validation with partial data round trip</a></li> <li><a href="/2020/12/28/an-f-demo-of-validation-with-partial-data-round-trip">An F# demo of validation with partial data round trip</a></li> </ul> As is often my modus operandi, I first did a proof of concept in Haskell. With its type classes and higher-kinded polymorphism, it's much faster to prototype solutions than even in F#. In the next article, I'll describe how that turned out. </p> <p> After the Haskell article, I'll show how it translates to F#. You can skip the Haskell article if you like. </p> <h3 id="048492d078164a648dddf6c57dbaf490"> Conclusion <a href="#048492d078164a648dddf6c57dbaf490" title="permalink">#</a> </h3> <p> I still think that validation is a solved problem. It's always interesting when such a belief for a moment is challenged, and satisfying to discover that it still holds. </p> <p> This is, after all, not proof of anything. Perhaps tomorrow, someone will throw another curve ball that I can't catch. If that happens, I'll have to update my beliefs. Until then, I'll consider validation a solved problem. </p> <p> <strong>Next:</strong> <a href="/2020/12/21/a-haskell-proof-of-concept-of-validation-with-partial-data-round-trip">A Haskell proof of concept of validation with partial data round trip</a>. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Branching tests https://blog.ploeh.dk/2020/12/07/branching-tests 2020-12-07T06:25:00+00:00 Mark Seemann <div id="post"> <p> <em>Is it ever okay to branch and loop in a unit test?</em> </p> <p> When I coach development organisations about unit testing and test-driven development, there's often a sizeable group of developers who don't see the value of unit testing. Some of the arguments they typically use are worth considering. </p> <p> A common complaint is that it's difficult to see the wisdom in writing code to prevent defects in code. That's not an unreasonable objection. </p> <p> <a href="/2020/05/25/wheres-the-science">We have scant scientific knowledge about software engineering</a>, but the little we know suggests that the number of defects is proportional to lines of code. The more lines of code, the more defects. </p> <p> If that's true, adding more code - even when it's test code - seems like a bad idea. </p> <h3 id="88c3fa7503454d9f9e329db299f70881"> Reasons to trust test code <a href="#88c3fa7503454d9f9e329db299f70881" title="permalink">#</a> </h3> <p> First, we should consider the possibility that the correlation between lines of code and defects doesn't mean that defects are <em>evenly</em> distributed. As <a href="https://www.adamtornhill.com">Adam Tornhill</a> argues in <a href="https://amzn.to/36Pd5EE">Your Code as a Crime Scene</a>, defects tend to cluster in hotspots. </p> <p> You can have a large proportion of your code base which is, for all intents and purpose, bug-free, and hotspots where defects keep spawning. </p> <p> If this is true, adding test code isn't a problem if you can keep it bug-free. </p> <p> That, however, sounds like a chicken-and-the-egg kind of problem. How can you know that test code is bug-free without tests? </p> <p> I've <a href="/2013/04/02/why-trust-tests">previously answered that question</a>. In short, you can trust a test for two reasons: <ul> <li>You've seen it fail (haven't you?)</li> <li>It's simple</li> </ul> I usually think of the simplicity criterion as a limit on <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a>: it should be <em>1</em>. This means no branching and no loops in your tests. </p> <p> That's what this article is actually about. </p> <h3 id="0494e1e3524d46b186ad7fc039dc7c17"> What's in a name? <a href="#0494e1e3524d46b186ad7fc039dc7c17" title="permalink">#</a> </h3> <p> I was working with an online restaurant reservation system (example code), and had written this test: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> This is a <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based test</a> that verifies that a valid reservation makes it to the database. The test has a cyclomatic complexity of <em>1</em>, and I've seen it fail, so all is good. (It may, in fact, contain a future maintenance problem, but that's a topic for <a href="/2021/01/11/waiting-to-happen">another article</a>.) </p> <p> What constitutes a valid reservation? At the very least, we should demand that <code>At</code> is a valid date and time, and that <code>Quantity</code> is a positive number. The restaurant would like to be able to email a confirmation to the user, so an email address is also required. Email addresses are notoriously difficult to validate, so we'll just require that the the string isn't null. </p> <p> What about the <code>Name</code>? I thought about this a bit and decided that, according to <a href="https://en.wikipedia.org/wiki/Robustness_principle">Postel's law</a>, the system should accept null names. The name is only a convenience; the system doesn't need it, it's just there so that when you arrive at the restaurant, you can say <em>"I have a reservation for Julia"</em> instead of giving an email address to the <a href="https://en.wikipedia.org/wiki/Ma%C3%AEtre_d%27h%C3%B4tel">maître d'hôtel</a>. But then, if you didn't supply a name when you made the reservation, you can always state your email address when you arrive. To summarise, the name is just a convenience, not a requirement. </p> <p> This decision meant that I ought to write a test case with a null name. </p> <p> That turned out to present a problem. I'd defined the <code>Reservation</code> class so that it didn't accept <code>null</code> arguments, and I think that's the appropriate design. Null is just evil and has no place in my domain models. </p> <p> That's not a problem in itself. In this case, I think it's acceptable to convert a null name to the empty string. </p> <h3 id="f0caaae1c01d47378b0f23a8d6a95d72"> Copy and paste <a href="#f0caaae1c01d47378b0f23a8d6a95d72" title="permalink">#</a> </h3> <p> Allow me to summarise. If you consider the above unit test, I needed a third test case with a null <code>name</code>. In that case, <code>expected</code> should be a <code>Reservation</code> value with the name <code>""</code>. Not <code>null</code>, but <code>""</code>. </p> <p> As far as I can tell, you can't easily express that in <code>PostValidReservationWhenDatabaseIsEmpty</code> without increasing its cyclomatic complexity. Based on the above introduction, that seems like a no-no. </p> <p> What's the alternative? Should I copy the test and adjust the <em>single</em> line of code that differs? If I did, it would look like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithNullNameWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Apart from the values in the <code>[InlineData]</code> attribute and the method name, the <em>only</em> difference from <code>PostValidReservationWhenDatabaseIsEmpty</code> is that <code>expected</code> has a hard-coded name of <code>""</code>. </p> <p> This is not acceptable. </p> <p> There's a common misconception that the <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> principle doesn't apply to unit tests. I don't see why this should be true. The DRY principle exists because copy-and-paste code is difficult to maintain. Unit test code is also code that you have to maintain. All the rules about writing maintainable code also apply to unit test code. </p> <h3 id="1d10bd69a6364e6984cb808ab5d9a8f8"> Branching in test <a href="#1d10bd69a6364e6984cb808ab5d9a8f8" title="permalink">#</a> </h3> <p> What's the alternative? One option (that shouldn't be easily dismissed) is to introduce a <a href="http://xunitpatterns.com/Test%20Helper.html">Test Helper</a> to perform the conversion from a nullable name to a non-nullable name. Such a helper would have a cyclomatic complexity of <em>2</em>, but could be unit tested in isolation. It might even turn out that it'd be useful in the production code. </p> <p> Still, that seems like overkill, so I instead made the taboo move and added branching logic to the existing test to see how it'd look: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-08-23&nbsp;16:55&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;<span style="color:blue;">null</span>,&nbsp;2)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Name&nbsp;??&nbsp;<span style="color:#a31515;">&quot;&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Notice that the <code>expected</code> name is now computed as <code>dto.Name ?? ""</code>. Perhaps you think about branching instructions as relating exclusively to keywords such as <code>if</code> or <code>switch</code>, but the <code>??</code> operator is also a branching instruction. The test now has a cyclomatic complexity of <code>2</code>. </p> <p> Is that okay? </p> <h3 id="baf81334614241e7b36bbe7f8f1af821"> To branch or not to branch <a href="#baf81334614241e7b36bbe7f8f1af821" title="permalink">#</a> </h3> <p> I think that in this case, it's okay to slightly increase the cyclomatic complexity of the test. It's not something I just pull out of my hat, though. I think it's possible to adjust the above heuristics to embrace this sort of variation. </p> <p> To be clear, I consider this an <em>advanced</em> practice. If you're just getting started with unit testing, try to keep tests simple. Keep the cyclomatic complexity at <em>1</em>. </p> <p> Had I been in the above situation a couple of years ago, I might not have considered this option. About a year ago, though, I watched <a href="https://en.wikipedia.org/wiki/John_Hughes_(computer_scientist)">John Hughes'</a> presentation <a href="https://youtu.be/NcJOiQlzlXQ">Building on developers' intuitions to create effective property-based tests</a>. When he, about 15 minutes in, wrote a test with a branching instruction, I remember becoming quite uncomfortable. This lasted for a while until I understood where he was going with it. It's truly an inspiring and illuminating talk; I highly recommend it. </p> <p> How it relates to the problem presented here is through <em>coverage</em>. While the <code>PostValidReservationWhenDatabaseIsEmpty</code> test now has a cyclomatic complexity of <em>2</em>, it's a parametrised test with three test cases. Two of these cover one branch, and the third covers the other. </p> <p> What's more important is the process that produced the test. I added one test case at a time, and for each case, <em>I saw the test fail</em>. </p> <p> Specifically, when I added the third test case with the null name, I first added the branching expression <code>dto.Name ?? ""</code> and ran the two existing tests. They still both passed, which bolstered my belief that they both exercised the left branch of that expression. I then added the third case and saw that it (and only it) failed. This supported my belief that the third case exercised the right branch of <code>??</code>. </p> <p> Branching in unit tests isn't something I do lightly. I still believe that it could make the test more vulnerable to future changes. I'm particularly worried about making a future change that might shift one or more of these test cases into false negatives in the form of <a href="/2019/10/14/tautological-assertion">tautological assertions</a>. </p> <h3 id="bfa3a3d3f82044f289568efcc675c70f"> Conclusion <a href="#bfa3a3d3f82044f289568efcc675c70f" title="permalink">#</a> </h3> <p> As you can tell, when I feel that I'm moving onto thin ice, I move deliberately. If there's one thing I've learned from decades of professional programming it's that my brain loves jumping to conclusions. Moving slowly and deliberately is my attempt at countering this tendency. I believe that it enables me to go faster in the long run. </p> <p> I don't think that branching in unit tests should be common, but I believe that it may be occasionally valid. The key, I think, is to guarantee that each branch in the test is covered by a test case. The implication is that there must be <em>at least</em> as many test cases as the cyclomatic complexity. In other words, the test <em>must</em> be a parametrised test. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="cbbf719d6e1744c9879edb6242bbdf21"> <div class="comment-author"><a href="http://www.morcs.com">James Morcom</a></div> <div class="comment-content"> <p>Hi Mark, I guess there is implicit cyclomatic complexity in the testing framework itself (For example, it loops through the <code>InlineData</code> records). That feels fine though, does this somehow have less cost than cyclomatic complexity in the test code itself? I guess, as you mentioned, it's acceptable because the alternative is violation of DRY. </p> <p> With this in mind, I wonder how you feel about adding an <code>expectedName</code> parameter to the <code>InlineData</code> attributes, instead of the conditional in the test code? Maybe it's harder to read though when the test data includes input and output. </p> </div> <div class="comment-date">2020-12-07 08:36 UTC</div> </div> <div class="comment" id="1f5efe14d22441ffac85f5f5afc9b3b1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> James, thank you for writing. I consider <a href="/2019/12/09/put-cyclomatic-complexity-to-good-use#de927bfcc95d410bbfcd0adf7a63926b">the cyclomatic complexity of a method call to be <em>1</em></a>, and Visual Studio code metrics agree with me. Whatever happens in a framework should, in my opinion, likewise be considered as encapsulated abstraction that's none of our business. </p> <p> Adding an <code>expectedName</code> parameter to the method is definitely an option. I sometimes do that, and I could have done that here, too. In this situation, I think it's a toss-up. It'd make it harder for a later reader of the code to parse the test cases, but would simplify the test code itself, so that alternative comes with both advantages and disadvantages. </p> </div> <div class="comment-date">2020-12-08 11:02 UTC</div> </div> <div class="comment" id="e5a0f70c39f411ebadc10242ac120002"> <div class="comment-author">Romain Deneau <a href="https://twitter.com/DeneauRomain">@DeneauRomain</a></div> <div class="comment-content"> <p> Hi Mark. To build up on the additional <code>expectedName</code> parameter, instead of keeping a single test with the 3 cases but the last being a edge case, I prefer introduce a specific test for the last case. </p> <p> Then, to remove the duplication, we can extract a common method which will take this additional <code>expectedName</code> parameter: </p> <p> <pre> [<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;2024-02-13&nbsp;18:15&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;&nbsp;&nbsp;&nbsp;9)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithNameWhenDatabaseIsEmpty &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;(<span style="color:blue;">string</span>&nbsp;at,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;quantity)&nbsp;=> PostValidReservationWhenDatabaseIsEmpty(at, email, name, expectedName: name, quantity); [<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWithoutNameWhenDatabaseIsEmpty()&nbsp;=> PostValidReservationWhenDatabaseIsEmpty( at          : <span style="color:#a31515;">&quot;2023-11-24&nbsp;19:00&quot;</span>, email       : <span style="color:#a31515;">&quot;juliad@example.net&quot;</span>, name        : <span style="color:blue;">null</span>, expectedName: <span style="color:#a31515;">&quot;&quot;</span>, quantity    : 5); <span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;expectedName, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>   &nbsp;quantity) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;at, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity, &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Parse(dto.At,&nbsp;<span style="color:#2b91af;">CultureInfo</span>.InvariantCulture), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;expectedName,&nbsp;<span style="color:green;">// /!\ Not `dto.Name`</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); } </pre> </p> </div> <div class="comment-date">2020-12-09 8:44 UTC</div> </div> <div class="comment" id="2779280fced748b0879a4d7267f3d634"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Romain, thank you for writing. There are, indeed, many ways to skin that cat. If you're comfortable with distributing a test over more than one method, I instead prefer to use another data source for the <code>[Theory]</code> attribute: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>&nbsp;: &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TheoryData</span>&lt;<span style="color:#2b91af;">ReservationDto</span>,&nbsp;<span style="color:#2b91af;">Reservation</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2023,&nbsp;11,&nbsp;24,&nbsp;19,&nbsp;0,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;juliad@example.net&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Julia&nbsp;Domna&quot;</span>,&nbsp;5); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2024,&nbsp;2,&nbsp;13,&nbsp;18,&nbsp;15,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;x@example.com&quot;</span>,&nbsp;<span style="color:#a31515;">&quot;Xenia&nbsp;Ng&quot;</span>,&nbsp;9); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddWithoutName(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DateTime</span>(2023,&nbsp;8,&nbsp;23,&nbsp;16,&nbsp;55,&nbsp;0),&nbsp;<span style="color:#a31515;">&quot;kite@example.edu&quot;</span>,&nbsp;2); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithName(<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;email, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;quantity &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(at,&nbsp;email,&nbsp;name,&nbsp;quantity)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddWithoutName(<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:blue;">string</span>&nbsp;email,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;{&nbsp;At&nbsp;=&nbsp;at.ToString(<span style="color:#a31515;">&quot;O&quot;</span>),&nbsp;Email&nbsp;=&nbsp;email,&nbsp;Quantity&nbsp;=&nbsp;quantity&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(at,&nbsp;email,&nbsp;<span style="color:#a31515;">&quot;&quot;</span>,&nbsp;quantity)); &nbsp;&nbsp;&nbsp;&nbsp;} } [<span style="color:#2b91af;">Theory</span>,&nbsp;<span style="color:#2b91af;">ClassData</span>(<span style="color:blue;">typeof</span>(<span style="color:#2b91af;">PostValidReservationWhenDatabaseIsEmptyTestCases</span>))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PostValidReservationWhenDatabaseIsEmpty( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;expected) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>(db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Post(dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Contains(expected,&nbsp;db); }</pre> </p> <p> Whether you prefer one over the other is, I think, subjective. I like my alternative, using a <code>[ClassData]</code> source, better, because I find it a bit more principled and 'pattern-based', if you will. I also like how small the actual test method becomes. </p> <p> Your solution, on the other hand, is more portable, in the sense that you could also apply it in a testing framework that doesn't have the sort of capability that xUnit.net has. That's a definite benefit with your suggestion. </p> </div> <div class="comment-date">2020-12-10 20:05 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Name by role https://blog.ploeh.dk/2020/11/30/name-by-role 2020-11-30T06:31:00+00:00 Mark Seemann <div id="post"> <p> <em>Consider naming variables according to their role, instead of their type.</em> </p> <p> My <a href="/2020/11/23/good-names-are-skin-deep">recent article on good names</a> might leave you with the impression that I consider good names unimportant. Not at all. That article was an attempt at delineating the limits of naming. Good names aren't the panacea some people seem to imply, but they're still important. </p> <p> As the cliché goes, naming is one of the hardest problems in software development. Perhaps it's hard because you have to do it so frequently. Every time you create a variable, you have to name it. It's also an opportunity to add clarity to a code base. </p> <p> A common naming strategy is to name objects after their type: </p> <p> <pre><span style="color:#2b91af;">Reservation</span>?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(id);</pre> </p> <p> or: </p> <p> <pre><span style="color:#2b91af;">Restaurant</span>?&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase.GetRestaurant(restaurantId);</pre> </p> <p> There's nothing inherently wrong with a naming scheme like this. It often makes sense. The <code>reservation</code> variable is a <code>Reservation</code> object, and there's not that much more to say about it. The same goes for the <code>restaurant</code> object. </p> <p> In some contexts, however, objects play specific <em>roles</em>. This is particularly prevalent with primitive types, but can happen to any type of object. It may help the reader if you name the variables according to such roles. </p> <p> In this article, I'll show you several examples. I hope these examples are so plentiful and varied that they can inspire you to come up with good names. </p> <h3 id="895798815b414b368058ed8640236ff9"> A variable introduced only to be named <a href="#895798815b414b368058ed8640236ff9" title="permalink">#</a> </h3> <p> In a <a href="/2020/11/09/checking-signed-urls-with-aspnet">recent article</a> I showed this code snippet: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;signaturesMatch&nbsp;=&nbsp;computedSignature.SequenceEqual(receivedSignature); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;signaturesMatch; }</pre> </p> <p> Did you wonder about the <code>signaturesMatch</code> variable? Why didn't I just return the result of <code>SequenceEqual</code>, like the following? </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;computedSignature.SequenceEqual(receivedSignature); }</pre> </p> <p> Visual Studio even offers this as a possible refactoring that it'll do for you. </p> <p> The inclusion of the <code>signaturesMatch</code> variable was a conscious decision of mine. I felt that directly returning the result of <code>SequenceEqual</code> was a bit too implicit. It forces readers to make the inference themselves: <em>Ah, the two arrays contain the same sequence of bytes; that must mean that the signatures match!</em> </p> <p> Instead of asking readers to do that work themselves, I decided to do it for them. I hope that it improves readability. It doesn't change the behaviour of the code one bit. </p> <h3 id="e5f18d0e31264b29bf11cd817b8c7bfa"> Test roles <a href="#e5f18d0e31264b29bf11cd817b8c7bfa" title="permalink">#</a> </h3> <p> When it comes to unit testing, there's plenty of inconsistent terminology. One man's <em>mock object</em> is another woman's <em>test double</em>. Most of the jargon isn't even internally consistent. Do yourself a favour and adopt a consistent pattern language. I use the one presented in <a href="http://bit.ly/xunitpatterns">xUnit Test Patterns</a>. </p> <p> For instance, the thing that you're testing is the System Under Test (SUT). This can be a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a> or a static method, but when it's an object, you're going to create a variable. Consider <a href="https://docs.microsoft.com/en-us/archive/blogs/ploeh/naming-sut-test-variables">naming it <em>sut</em></a>. A typical test also defines other variables. Naming one of them <code>sut</code> clearly identifies which of them is the SUT. It also protects the tests against the class in question being renamed. </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ScheduleSingleReservationCommunalTable() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;table&nbsp;=&nbsp;<span style="color:#2b91af;">Table</span>.Communal(12); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(6), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;table); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;r&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.Schedule(<span style="color:blue;">new</span>[]&nbsp;{&nbsp;r&nbsp;}); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TimeSlot</span>(r.At,&nbsp;table.Reserve(r))&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(expected,&nbsp;actual); }</pre> </p> <p> The above test follows <a href="/2013/06/24/a-heuristic-for-formatting-code-according-to-the-aaa-pattern">my AAA.formatting heuristic</a>. In all, it defines five variables, but there can be little doubt about which one is the <code>sut</code>. </p> <p> The <code>table</code> and <code>r</code> variables follow the mainstream practice of naming variables after their type. They play no special role, so that's okay. You may balk at such a short variable name as <code>r</code>, and that's okay. In my defence, I follow <a href="http://amzn.to/XCJi9X">Clean Code</a>'s <em>N5</em> heuristic for long and short scopes. A variable name like <code>r</code> is fine when it only spans three lines of code (four, if you also count the blank line). </p> <p> Consider also using the variable names <code>expected</code> and <code>actual</code>, as in the above example. In many unit testing frameworks, those are the argument names for the assertion. For instance, in <a href="https://xunit.net">xUnit.net</a> (which the above test uses) the <code>Assert.Equals</code> overloads are defined as <code>Equal&lt;T&gt;(T expected, T actual)</code>. Using these names for variables makes the roles clearer, I think. </p> <h3 id="bb01b7e1d37f469e8a81f288b513248a"> The other <a href="#bb01b7e1d37f469e8a81f288b513248a" title="permalink">#</a> </h3> <p> The above assertion relies on structural equality. The <code>TimeSlot</code> class is immutable, so it can safely override <code>Equals</code> (and <code>GetHashCode</code>) to implement structural equality: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">override</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Equals(<span style="color:blue;">object</span>?&nbsp;obj) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;obj&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">TimeSlot</span>&nbsp;other&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;==&nbsp;other.At&nbsp;&amp;&amp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tables.SequenceEqual(other.Tables); }</pre> </p> <p> I usually call the downcast variable <code>other</code> because, from the perspective of the instance, it's the other object. I usually use that convention whenever an instance interacts with another object of the same type. Among other examples, this happens when you model objects as <a href="/2017/11/27/semigroups">semigroups</a> and <a href="/2017/10/06/monoids">monoids</a>. The <a href="/2018/07/16/angular-addition-monoid">Angle struct, for example, defines this binary operation</a>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Angle</span>&nbsp;Add(<span style="color:#2b91af;">Angle</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Angle</span>(<span style="color:blue;">this</span>.degrees&nbsp;+&nbsp;other.degrees); }</pre> </p> <p> Again, the method argument is in the role as the other object, so naming it <code>other</code> seems natural. </p> <p> Here's another example from a restaurant reservation code base: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Seating</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Start&nbsp;&lt;&nbsp;other.End&nbsp;&amp;&amp;&nbsp;other.Start&nbsp;&lt;&nbsp;End; }</pre> </p> <p> The <code>Overlaps</code> method is an instance method on the <code>Seating</code> class. Again, <code>other</code> seems natural. </p> <h3 id="8cf3583ad0ac446fa4b4dee2272ba054"> Candidates <a href="#8cf3583ad0ac446fa4b4dee2272ba054" title="permalink">#</a> </h3> <p> The <code>Overlaps</code> method looks like a <em>predicate</em>, i.e. a function that returns a Boolean value. In the case of that method, <code>other</code> indicates the role of being the other object, but it also plays another role. It makes sense to me to call predicate input <em>candidates</em>. Typically, you have some input that you want to evaluate as either true or false. I think it makes sense to think of such a parameter as a 'truth candidate'. You can see one example of that in the above <code>SignatureIsValid</code> method. </p> <p> There, the <code>string</code> parameter is a <code>candidate</code> for having a valid signature. </p> <p> Here's another restaurant-related example: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;WillAccept( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;now, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Reservation</span>&gt;&nbsp;existingReservations, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;candidate) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(existingReservations&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(existingReservations)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(candidate&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(candidate.At&nbsp;&lt;&nbsp;now) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsOutsideOfOpeningHours(candidate)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;seating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;candidate.At); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;relevantReservations&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;existingReservations.Where(seating.Overlaps); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;availableTables&nbsp;=&nbsp;Allocate(relevantReservations); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;availableTables.Any(t&nbsp;=&gt;&nbsp;t.Fits(candidate.Quantity)); }</pre> </p> <p> Here, the reservation in question is actually not yet a reservation. It might be rejected, so it's a <code>candidate</code> reservation. </p> <p> You can also use that name in <code>TryParse</code> methods, as shown in <a href="/2019/12/09/put-cyclomatic-complexity-to-good-use">this article</a>. </p> <h3 id="4301289931dc4f94abf46578ddcfd693"> Data Transfer Objects <a href="#4301289931dc4f94abf46578ddcfd693" title="permalink">#</a> </h3> <p> Another name that I like to use is <code>dto</code> for <a href="https://en.wikipedia.org/wiki/Data_transfer_object">Data Transfer Objects</a> (DTOs). The benefit here is that as long as <code>dto</code> is unambiguous in context, it makes it easier to distinguish between a DTO and the domain model you might want to turn it into: </p> <p> <pre>[<span style="color:#2b91af;">HttpPost</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/reservations&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;<span style="color:#2b91af;">Guid</span>.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;reservation&nbsp;=&nbsp;dto.Validate(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase.GetRestaurant(restaurantId); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(restaurant&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;TryCreate(restaurant,&nbsp;reservation); }</pre> </p> <p> By naming the input parameter <code>dto</code>, I keep the name <code>reservation</code> free for the domain object, which ought to be the more important object of the two. <blockquote> <p> "A Data Transfer Object is one of those objects our mothers told us never to write." </p> <footer><cite><a href="http://bit.ly/patternsofeaa">Martin Fowler</a></cite></footer> </blockquote> I could have named the input parameter <code>reservationDto</code> instead of <code>dto</code>, but that would diminish the 'mental distance' between <code>reservationDto</code> and <code>reservation</code>. I like to keep that distance, so that the roles are more explicit. </p> <h3 id="b8ece382f7554141aa741fc9ded9e02a"> Time <a href="#b8ece382f7554141aa741fc9ded9e02a" title="permalink">#</a> </h3> <p> You often need to make decisions based on the current time or date. In .NET the return value from <a href="https://docs.microsoft.com/dotnet/api/system.datetime.now">DateTime.Now</a> is a <code>DateTime</code> value. Typical variable names are <code>dateTime</code>, <code>date</code>, <code>time</code>, or <code>dt</code>, but why not call it <code>now</code>? </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;TryCreate(<span style="color:#2b91af;">Restaurant</span>&nbsp;restaurant,&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;scope&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">TransactionScope</span>(<span style="color:#2b91af;">TransactionScopeAsyncFlowOption</span>.Enabled); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.ReadReservations(restaurant.Id,&nbsp;reservation.At); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;Clock.GetCurrentDateTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!restaurant.MaitreD.WillAccept(now,&nbsp;reservations,&nbsp;reservation)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(restaurant.Id,&nbsp;reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;scope.Complete(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(restaurant.Id,&nbsp;reservation); }</pre> </p> <p> This is the <code>TryCreate</code> method called by the above <code>Post</code> method. Here, <code>DateTime.Now</code> is hidden behind <code>Clock.GetCurrentDateTime()</code> in order to make <a href="/2020/03/23/repeatable-execution">execution repeatable</a>, but the idea remains: the variable represents the current time or date, or, with a bit of good will, <code>now</code>. </p> <p> Notice that the <code>WillAccept</code> method (shown above) also uses <code>now</code> as a parameter name. That value's role is to represent <code>now</code> as a concept. </p> <p> When working with time, I also sometimes use the variable names <code>before</code> and <code>after</code>. This is mostly useful in integration tests: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;GetCurrentYear() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LegacyApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;before&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;response&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.GetCurrentYear(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;after&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;response.EnsureSuccessStatusCode(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;response.ParseJsonContent&lt;<span style="color:#2b91af;">CalendarDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;AssertOneOf(before.Year,&nbsp;after.Year,&nbsp;actual.Year); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Null(actual.Month); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Null(actual.Day); &nbsp;&nbsp;&nbsp;&nbsp;AssertLinks(actual); }</pre> </p> <p> While you can inject something like a <code>Clock</code> dependency in order to make your SUT deterministic, in integration tests you might want to see behaviour when using the system clock. You can often verify such behaviour by surrounding the test's <em>Act</em> phase with two calls to <code>DateTime.Now</code>. This gives you the time <code>before</code> and <code>after</code> the test exercised the SUT. </p> <p> When you do that, however, be careful with the assertions. If such a test runs at midnight, <code>before</code> and <code>after</code> might be two different dates. If it runs on midnight December 31, it might actually be two different years! That's the reason that the test passes as long as the <code>actual.Year</code> is either of <code>before.Year</code> and <code>after.Year</code>. </p> <h3 id="8183f4e53fdd4ceca701ef17a6509614"> Invalid values <a href="#8183f4e53fdd4ceca701ef17a6509614" title="permalink">#</a> </h3> <p> While integration tests often test happy paths, unit tests should also exercise error paths. What happens when you supply invalid input to a method? When you write such tests, you can identify the invalid values by naming the variables or parameters accordingly: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:blue;">null</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;bas&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;PutInvalidId(<span style="color:blue;">string</span>&nbsp;invalidId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;db&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FakeDatabase</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationsController</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SystemClock</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">InMemoryRestaurantDatabase</span>(<span style="color:#2b91af;">Some</span>.Restaurant), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;db); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dummyDto&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ReservationDto</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;At&nbsp;=&nbsp;<span style="color:#a31515;">&quot;2024-06-25&nbsp;18:19&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Email&nbsp;=&nbsp;<span style="color:#a31515;">&quot;colera@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Cole&nbsp;Aera&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Quantity&nbsp;=&nbsp;2 &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Put(invalidId,&nbsp;dummyDto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.IsAssignableFrom&lt;<span style="color:#2b91af;">NotFoundResult</span>&gt;(actual); }</pre> </p> <p> Here, the invalid input represent an ID. To indicate that, I called the parameter <code>invalidId</code>. </p> <p> The system under test is the <code>Put</code> method, which takes two arguments: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Put(<span style="color:blue;">string</span>&nbsp;id,&nbsp;<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto)</pre> </p> <p> When testing an error path, it's important to keep other arguments well-behaved. In this example, I want to make sure that it's the <code>invalidId</code> that causes the <code>NotFoundResult</code> result. Thus, the <code>dto</code> argument should be as well-behaved as possible, so that it isn't going to be the source of divergence. </p> <p> Apart from being well-behaved, that object plays no role in the test. It just needs to be there to make the code compile. <em>xUnit Test Patterns</em> calls such an object a <em>Dummy Object</em>, so I named the variable <code>dummyDto</code> as information to any reader familiar with that pattern language. </p> <h3 id="441f47fab8b74787840c0237b3958316"> Derived class names <a href="#441f47fab8b74787840c0237b3958316" title="permalink">#</a> </h3> <p> The thrust of all of these examples is that you don't <em>have</em> to name variables after their types. You can extend this line of reasoning to class inheritance. Just because a base class is called <code>Foo</code> it doesn't mean that you <em>have</em> to call a derived class <code>SomethingFoo</code>. </p> <p> This is something of which I have to remind myself. For example, to support integration testing with ASP.NET you'll need a <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.testing.webapplicationfactory-1">WebApplicationFactory&lt;TEntryPoint&gt;</a>. To override the default DI Container configuration, you'll have to derive from this class and override its <code>ConfigureWebHost</code> method. In <a href="/2020/04/20/unit-bias-against-collections">an example I've previously published</a> I didn't spend much time thinking about the class name, so <code>RestaurantApiFactory</code> it was. </p> <p> At first, I named the variables of this type <code>factory</code>, or something equally devoid of information. That bothered me, so instead tried <code>service</code>, which I felt was an improvement, but still too vapid. I then adopted <code>api</code> as a variable name, but then realised that that also suggested a better class name. So currently, this defines my self-hosting API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>&nbsp;:&nbsp;<span style="color:#2b91af;">WebApplicationFactory</span>&lt;<span style="color:#2b91af;">Startup</span>&gt;</pre> </p> <p> Here's how I use it: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;ReserveTableAtNono() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SelfHostedApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;client&nbsp;=&nbsp;api.CreateClient(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;dto&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation.ToDto(); &nbsp;&nbsp;&nbsp;&nbsp;dto.Quantity&nbsp;=&nbsp;6; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;response&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;client.PostReservation(<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;dto); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;at&nbsp;=&nbsp;<span style="color:#2b91af;">Some</span>.Reservation.At; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;AssertRemainingCapacity(client,&nbsp;at,&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;4); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;AssertRemainingCapacity(client,&nbsp;at,&nbsp;<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>,&nbsp;10); }</pre> </p> <p> The variable is just called <code>api</code>, but the reader can tell from the initialisation that this is an instance of the <code>SelfHostedApi</code> class. I like how that communicates that this is an integration test that uses a self-hosted API. It literally says that. </p> <p> This test also uses the <code>dto</code> naming convention. Additionally, you may take note of the variable and property called <code>at</code>. That's another name for a date and time. I struggled with naming this value, until <a href="http://blog.strobaek.org">Karsten Strøbæk</a> suggested that I used the simple word <em>at:</em> <code>reservation.At</code> indicates the date and time of the reservation without being encumbered by awkward details about date and time. Should we call it <code>date</code>? <code>time</code>? <code>dateTime</code>? No, just call it <code>at</code>. I find it elegant. </p> <h3 id="6692436d37dd491fa9920a5f4ac63118"> Conclusion <a href="#6692436d37dd491fa9920a5f4ac63118" title="permalink">#</a> </h3> <p> Sometimes, a <code>Reservation</code> object is just a <code>reservation</code>, and that's okay. At other times, it's the <code>actual</code> value, or the <code>expected</code> value. If it represents an invalid reservation in a test case, it makes sense to call the variable <code>invalidResevation</code>. </p> <p> Giving variables descriptive names improves <a href="/2019/03/04/code-quality-is-not-software-quality">code quality</a>. You don't have to write <a href="http://butunclebob.com/ArticleS.TimOttinger.ApologizeIncode">comments as apologies for poor readability</a> if a better name communicates what the comment would have said. </p> <p> Consider naming variables (and classes) for the <em>roles</em> they play, rather than their types. </p> <p> On the other hand, <a href="/2016/10/25/when-variable-names-are-in-the-way">when variable names are in the way</a>, consider <a href="https://en.wikipedia.org/wiki/Tacit_programming">point-free code</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="8d7dc043829244509b1a13c184f3cbbf"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Excellent name suggestions. Thanks for sharing them :) </p> <blockquote> I usually call the downcast variable <code>other</code> because, from the perspective of the instance, it's the other object. I usually use that convention whenever an instance interacts with another object of the same type. </blockquote> <p> The name <code>other</code> is good, but I prefer <code>that</code> because I think it is a better antonym of <code>this</code> (the keyword for the current instance) and because it has the same number of letters as <code>this</code>. </p> <blockquote> <p> It makes sense to me to call predicate input <em>candidates</em>. Typically, you have some input that you want to evaluate as either true or false. I think it makes sense to think of such a parameter as a 'truth candidate'. You can see one example of that in the above <code>SignatureIsValid</code> method. </p> <p>...</p> <p> Here, the reservation in question is actually not yet a reservation. It might be rejected, so it's a <code>candidate</code> reservation. </p> </blockquote> <p> I typically try to avoid turning some input into either true or false. In particular, I find it confusing for the syntax to say that some instance is a <code>Reservation</code> while the semantics says that it "is actually not yet a reservation". I think of this as an example of <a href="https://blog.ploeh.dk/2015/01/19/from-primitive-obsession-to-domain-modelling/">primitive obsession</a>. Strictly speaking, I think <a href="https://medium.com/the-sixt-india-blog/primitive-obsession-code-smell-that-hurt-people-the-most-5cbdd70496e9#5009:~:text=Primitive%20Obsession%20is%20when%20the%20code%20relies%20too%20much%20on%20primitives.">"Primitive Obsession is when the code relies too much on primitives."</a> (aka, on primitive types). In my mind though, I have generalized this to cover any code that relies too much on weaker types. Separate types <code>Reservation</code> and <code>bool</code> are weaker than separate types <code>Reservation</code> and <code>CandidateReservation</code>. I think Alexis King summarized this well with a blog post titled <a href="https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/">Parse, don’t validate</a>. </p> <p> And yet, my coworkers and I have engaged in friendly but serious debates for years about which of those two approaches is better. My argument, essentially as given above, is for separate types <code>Reservation</code> and <code>CandidateReservation</code>. The main counterargument is that these types are the same except for a database-generated ID, so just represent both using one type with an optional ID. </p> <p> Have you thought about this before? </p> <blockquote> <p> By naming the input parameter <code>dto</code>, I keep the name <code>reservation</code> free for the domain object, which ought to be the more important object of the two. </p> <p>...</p> <p> I could have named the input parameter <code>reservationDto</code> instead of <code>dto</code>, but that would diminish the 'mental distance' between <code>reservationDto</code> and <code>reservation</code>. I like to keep that distance, so that the roles are more explicit. </p> </blockquote> <p> I prefer to emphasize the roles even more by using the names <code>dto</code> and <code>model</code>. We are in the implementation of the (Post) route for <code>"restaurants/{restaurantId}/reservations"</code>, so I think it is clear from context that the <code>dto</code> and <code>model</code> are really a reservation DTO and a reservation model. </p> </div> <div class="comment-date">2020-11-30 20:46 UTC</div> </div> <div class="comment" id="931a54d51d3d445c9eaf6a41d8923406"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. Certainly, I didn't intent my article to dictate names. As you imply, there's room for both creativity and subjectivity, and that's fine. My suggestions were meant only for inspiration. <blockquote> <p> The main counterargument is that these types are the same except for a database-generated ID, so just represent both using one type with an optional ID. </p> <p> Have you thought about this before? </p> </blockquote> Yes; I would <a href="/2014/08/11/cqs-versus-server-generated-ids">think twice before deciding to model a domain type with a database-generated ID</a>. A server-generated ID is an implementation detail that shouldn't escape the data access layer. If it does, you have a leaky abstraction at hand. Sooner or later, it's going to bite you. </p> <p> The <code>Reservation</code> class in the above examples has this sole constructor: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Reservation</span>(<span style="color:#2b91af;">Guid</span>&nbsp;id,&nbsp;<span style="color:#2b91af;">DateTime</span>&nbsp;at,&nbsp;<span style="color:#2b91af;">Email</span>&nbsp;email,&nbsp;<span style="color:#2b91af;">Name</span>&nbsp;name,&nbsp;<span style="color:blue;">int</span>&nbsp;quantity)</pre> </p> <p> You can't create an instance without supplying an ID. On the other hand, any code can conjure up a GUID, so no server is required. At the type-level, there's no compelling reason to distinguish between a reservation and a candidate reservation. </p> <p> Granted, you <em>could</em> define two types, <code>Reservation</code> and <code>CandidateReservation</code>, but they'd be isomorphic. In Haskell, you'd probably use a <code>newtype</code> for one of these types, and then you're <a href="https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-type-safety">back at Alexis King's blog</a>. </p> </div> <div class="comment-date">2020-12-02 7:43 UTC</div> </div> <div class="comment" id="fe3ac3a60c754340801b877666a07b65"> <div class="comment-author"><a href="https://ttulka.com">Tomas Tulka</a></div> <div class="comment-content"> <blockquote>...naming is one of the hardest problems in software development. Perhaps it's hard because you have to do it so frequently.</blockquote> <p>Usually, doing things frequently means mastering them pretty quickly. Not so for naming. I guess, there are multiple issues:</p> <ol> <li>Words are ambiguous. The key is, not to do naming in isolation, the context matters. For example, it's difficult to come up with a good name for a method when we don't have a good name for its class, the whole component, etc. Similar with Clean Code's N5: the meaning of a short variable is clear in a small scope, closed context.</li> <li>Good naming requires deep understanding of the domain. Developers are usualy not good at the business they model. Sadly, it often means "necessary evil" for them.</li> </ol> <p>Naming variables by their roles is a great idea!</p> <p>Many thanks for another awesome post, I enjoyed reading it.</p> </div> <div class="comment-date">2020-12-11 09:11 UTC</div> </div> <div class="comment" id="e9299b251c8e4b7f9a168ff571f70950"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tomas, thank you for writing. <blockquote> <p> doing things frequently means mastering them pretty quickly. Not so for naming. I guess </p> </blockquote> Good point; I hadn't thought about that. I think that the reasons you list are valid. </p> <p> As an additional observation, it may be that there's a connection to the notion of <em>deliberate practice</em>. As the catch-phrase about professional experience puts it, there's a difference between 20 years of experience and one year of experience repeated 20 times. </p> <p> Doing a thing again and again generates little improvement if one does it by rote. One has to deliberately practice. In this case, it implies that a programmer should explicitly reflect on variable names, and consider more than one option. </p> <p> I haven't met many software developers who do that. </p> </div> <div class="comment-date">2020-12-15 9:19 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Good names are skin-deep https://blog.ploeh.dk/2020/11/23/good-names-are-skin-deep 2020-11-23T06:33:00+00:00 Mark Seemann <div id="post"> <p> <em>Good names are important, but insufficient, for code maintainability.</em> </p> <p> You should give the building blocks of your code bases descriptive names. It's easier to understand the purpose of a library, module, class, method, function, etcetera if the name contains a clue about the artefact's purpose. This is hardly controversial, and while naming is hard, most teams I visit agree that names are important. </p> <p> Still, despite good intentions and efforts to name things well, code bases deteriorate into unmaintainable clutter. </p> <p> Clearly, good names aren't enough. </p> <h3 id="5a20f600298844e98c89c423f5c66c5f"> Tenuousness of names <a href="#5a20f600298844e98c89c423f5c66c5f" title="permalink">#</a> </h3> <p> A good name is tenuous. First, naming is hard, so while you may have spent some effort coming up with a good name, other people may misinterpret it. Because they originate from natural language, names are as ambiguous as language. (<a href="/2018/07/02/terse-operators-make-business-code-more-readable">Terse operators, on the other hand...</a>) </p> <p> Another maintainability problem with names is that implementation may change over time, but the names remain constant. Granted, modern IDEs make it easy to rename methods, but developers rarely adjust names when they adjust behaviour. Even the best names may become misleading over time. </p> <p> These weakness aren't the worst, though. In my experience, a more fundamental problem is that all it takes is one badly named 'wrapper object' before the information in a good name is lost. </p> <p> <img src="/content/binary/vague-names-hiding-clear-names.png" alt="Object with clear names enclosed in object with vague names."> </p> <p> In the figure, the inner object is well-named. It has a clear name and descriptive method names. All it takes before this information is lost, however, is another object with vague names to 'encapsulate' it. </p> <h3 id="c0cfcf2d16a94e4c96c33c9d0359846f"> An attempt at a descriptive method name <a href="#c0cfcf2d16a94e4c96c33c9d0359846f" title="permalink">#</a> </h3> <p> Here's an example. Imagine an online restaurant reservation system. One of the features of this system is to take reservations and save them in the database. </p> <p> A restaurant, however, is a finite resource. It can only accommodate a certain number of guests at the same time. Whenever the system receives a reservation request, it'll have to retrieve the existing reservations for that time and make a decision. <a href="/2020/01/27/the-maitre-d-kata">Can it accept the reservation?</a> Only if it can should it save the reservation. </p> <p> How do you model such an interaction? How about a descriptive name? How about <code>TrySave</code>? Here's a possible implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">bool</span>&gt;&nbsp;TrySave(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(reservation)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservation.At, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;reservation.At&nbsp;+&nbsp;SeatingDuration) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;availableTables&nbsp;=&nbsp;Allocate(reservations); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!availableTables.Any(t&nbsp;=&gt;&nbsp;reservation.Quantity&nbsp;&lt;=&nbsp;t.Seats)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(reservation).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">true</span>; }</pre> </p> <p> There's an implicit naming convention in .NET that methods with the <code>Try</code> prefix indicate an operation that may or may not succeed. The return value of such methods is either <code>true</code> or <code>false</code>, and they may also have <code>out</code> parameters if they optionally produce a value. That's not the case here, but I think one could make the case that <code>TrySave</code> succinctly describes what's going on. </p> <p> All is good, then? </p> <h3 id="bbf9fd5509ba4a2c823483acd40fbe22"> A vague wrapper <a href="#bbf9fd5509ba4a2c823483acd40fbe22" title="permalink">#</a> </h3> <p> After our conscientious programmer meticulously designed and named the above <code>TrySave</code> method, it turns out that it doesn't meet all requirements. Users of the system file a bug: the system accepts reservations outside the restaurant's opening hours. </p> <p> The original programmer has moved on to greener pastures, so fixing the bug falls on a poor maintenance developer with too much to do. Having recently learned about the <a href="https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle">open-closed principle</a>, our new protagonist decides to wrap the existing <code>TrySave</code> in a new method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">bool</span>&gt;&nbsp;Check(<span style="color:#2b91af;">Reservation</span>&nbsp;reservation) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(reservation)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation.At&nbsp;&lt;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(reservation.At.TimeOfDay&nbsp;&lt;&nbsp;OpensAt) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(LastSeating&nbsp;&lt;&nbsp;reservation.At.TimeOfDay) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">false</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">await</span>&nbsp;Manager.TrySave(reservation).ConfigureAwait(<span style="color:blue;">false</span>); }</pre> </p> <p> This new method first checks whether the <code>reservation</code> is within opening hours and in the future. If that's not the case, it returns <code>false</code>. Only if these preconditions are fulfilled does it delegate the decision to that <code>TrySave</code> method. </p> <p> Notice, however, the name. The bug was urgent, and our poor programmer didn't have time to think of a good name, so <code>Check</code> it is. </p> <h3 id="6056d14ac9ab4046b4aa417d3902fbf1"> Caller's perspective <a href="#6056d14ac9ab4046b4aa417d3902fbf1" title="permalink">#</a> </h3> <p> How does this look from the perspective of calling code? Here's the Controller action that handles the pertinent HTTP request: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;r&nbsp;=&nbsp;dto.Validate(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(r&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;isOk&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Manager.Check(r).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!isOk) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">StatusCodeResult</span>(<span style="color:#2b91af;">StatusCodes</span>.Status500InternalServerError); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NoContentResult</span>(); }</pre> </p> <p> Try to forget the code you've just seen and imagine that you're looking at this code first. You'd be excused if you miss what's going on. It looks as though the method just does a bit of validation and then <em>checks</em> 'something' concerning the reservation. </p> <p> There's no hint that the <code>Check</code> method might perform the significant side effect of saving the reservation in the database. </p> <p> You'll only learn that if you <em>read</em> the implementation details of <code>Check</code>. As I argue in my <a href="https://cleancoders.com/episode/humane-code-real-episode-1">Humane Code video</a>, <em>if you have to read the source code of an object, encapsulation is broken.</em> </p> <p> Such code doesn't fit in your brain. You'll struggle as you try keep track of all the things that are going on in the code, all the way from the outer boundary of the application to implementation details that relate to databases, third-party services, etcetera. </p> <h3 id="59249ae122b540ca907549ade3eca649"> Straw man? <a href="#59249ae122b540ca907549ade3eca649" title="permalink">#</a> </h3> <p> You may think that this is a straw man argument. After all, wouldn't it be better to edit the original <code>TrySave</code> method? </p> <p> Perhaps, but it would make that class more complex. The <code>TrySave</code> method has a <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">cyclomatic complexity</a> of only <em>3</em>, while the <code>Check</code> method has a complexity of <em>5</em>. Combining them might easily take them over some <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a>. </p> <p> Additionally, each of these two classes have different dependencies. As the <code>TrySave</code> method implies, it relies on both <code>Repository</code> and <code>SeatingDuration</code>, and the <code>Allocate</code> helper method (not shown) uses a third dependency: the restaurant's table configuration. </p> <p> Likewise, the <code>Check</code> method relies on <code>OpensAt</code> and <code>LastSeating</code>. If you find it better to edit the original <code>TrySave</code> method, you'd have to combine these dependencies as well. Each time you do that, the class grows until it becomes a <a href="https://en.wikipedia.org/wiki/God_object">God object</a>. </p> <p> It's rational to attempt to separate things in multiple classes. It also, on the surface, seems to make unit testing easier. For example, here's a test that verifies that the <code>Check</code> method rejects reservations before the restaurant's opening time: </p> <p> <pre>[<span style="color:#2b91af;">Fact</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;RejectReservationBeforeOpeningTime() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;r&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Reservation</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.AddDays(10).Date.AddHours(17), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;colaera@example.com&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Cole&nbsp;Aera&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;mgrTD&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Mock</span>&lt;<span style="color:#2b91af;">IReservationsManager</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;mgrTD.Setup(mgr&nbsp;=&gt;&nbsp;mgr.TrySave(r)).ReturnsAsync(<span style="color:blue;">true</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">RestaurantManager</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(18), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>.FromHours(21), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mgrTD.Object); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;sut.Check(r); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.False(actual); }</pre> </p> <p> By replacing the <code>TrySave</code> method by a test double, you've ostensibly decoupled the <code>Check</code> method from all the complexity of the <code>TrySave</code> method. </p> <p> To be clear, this style of programming, with lots of nested interfaces and tests with <a href="/2013/10/23/mocks-for-commands-stubs-for-queries">mocks and stubs</a> is far from ideal, but I still find it better than a <a href="https://en.wikipedia.org/wiki/Big_ball_of_mud">big ball of mud</a>. </p> <h3 id="33cb10390fa4417f96b24e4b9102d4ed"> Alternative <a href="#33cb10390fa4417f96b24e4b9102d4ed" title="permalink">#</a> </h3> <p> A better alternative is <a href="https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell">Functional Core, Imperative Shell</a>, AKA <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a>. Move all impure actions to the edge of the system, leaving only <a href="https://en.wikipedia.org/wiki/Referential_transparency">referentially transparent</a> functions as the main implementers of logic. It could look like this: </p> <p> <pre>[<span style="color:#2b91af;">HttpPost</span>] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Post(<span style="color:#2b91af;">ReservationDto</span>&nbsp;dto) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(dto)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;id&nbsp;=&nbsp;dto.ParseId()&nbsp;??&nbsp;<span style="color:#2b91af;">Guid</span>.NewGuid(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>?&nbsp;r&nbsp;=&nbsp;dto.Validate(id); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(r&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">BadRequestResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.ReadReservations(r.At).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!MaitreD.WillAccept(<span style="color:#2b91af;">DateTime</span>.Now,&nbsp;reservations,&nbsp;r)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;NoTables500InternalServerError(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;Repository.Create(r).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Reservation201Created(r); }</pre> </p> <p> Nothing is swept under the rug here. <code>WillAccept</code> is a <a href="https://en.wikipedia.org/wiki/Pure_function">pure function</a>, and while it encapsulates significant complexity, the only thing you need to understand when you're trying to understand the above <code>Post</code> code is that it returns either <code>true</code> or <code>false</code>. </p> <p> Another advantage of pure functions is that they are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. That makes unit testing and test-driven development easier. </p> <p> Even with a functional core, you'll also have an imperative shell. You can still test that, too, such as the <code>Post</code> method. It isn't referentially transparent, so you might be inclined to use mocks and stubs, but I instead recommend <a href="/2019/02/18/from-interaction-based-to-state-based-testing">state-based testing with a Fake database</a>. </p> <h3 id="6758361ac712400aae148a7dcc2a4a70"> Conclusion <a href="#6758361ac712400aae148a7dcc2a4a70" title="permalink">#</a> </h3> <p> Good names are important, but don't let good names, alone, lull you into a false sense of security. All it takes is one vaguely named wrapper object, and all the information in your meticulously named methods is lost. </p> <p> This is one of many reasons I try to design with static types instead of names. Not that I dismiss the value of good names. After all, you'll have to give your types good names as well. </p> <p> Types are more robust in the face of inadvertent changes; or, rather, they tend to resist when we try to do something stupid. I suppose that's what lovers of dynamically typed languages feel as 'friction'. In my mind, it's entirely opposite. Types keep me honest. </p> <p> Unfortunately, most type systems don't offer an adequate degree of safety. Even in <a href="https://fsharp.org">F#</a>, which has a great type system, you can introduce impure actions into what you thought was a pure function, and <a href="/2020/02/24/discerning-and-maintaining-purity">you'd be none the wiser</a>. That's one of the reasons I find <a href="https://www.haskell.org">Haskell</a> so interesting. Because of <a href="/2020/06/08/the-io-container">the way IO works</a>, you can't inadvertently sweep surprises under the rug. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="9ecb24dc5f78413687547c7f74f2d8b9"> <div class="comment-author">Johannes Schmitt</div> <div class="comment-content"> <p> I find the idea of the impure/pure/impure sandwich rather interesting and I agree with the benefits that it yields. However, I was wondering about where to move synchronization logic, i.e. the reservation system should avoid double bookings. With the initial TrySave approach it would be clear for me where to put this logic: the synchonrization mechanism should be part of the TrySave method. With the impure/pure/impure sandwich, it will move out to the most outer layern (HTTP Controller) - at least this is how I'd see it. My feelings tells me that this is a bit smelly, but I can't really pin point why I think so. Can you give some advice on this? How would you solve that? </p> </div> <div class="comment-date">2020-12-12 19:08 UTC</div> </div> <div class="comment" id="ebce3b718bc84329b6979bcacf6c2573"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Johannes, thank you for writing. There are several ways to address that issue, depending on what sort of trade-off you're looking for. There's always a trade-off. </p> <p> You can address the issue with a lock-free architecture. This typically involves expressing the desired action as a Command and putting it on a durable queue. If you combine that with a single-threaded, single-instance Actor that pulls Commands off the queue, you need no further transaction processing, because the architecture itself serialises writes. You can find plenty of examples of such an architecture on the internet, including (IIRC) my Pluralsight course <a href="/functional-architecture-with-fsharp">Functional Architecture with F#</a>. </p> <p> Another option is to simply surround the <a href="/2020/03/02/impureim-sandwich">impureim sandwich</a> with a <a href="https://docs.microsoft.com/dotnet/api/system.transactions.transactionscope">TransactionScope</a> (if you're on .NET, that is). </p> </div> <div class="comment-date">2020-12-16 16:59 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Redirect legacy URLs https://blog.ploeh.dk/2020/11/16/redirect-legacy-urls 2020-11-16T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Evolving REST API URLs when cool URIs don't change.</em> </p> <p> More than one reader reacted to my article on <a href="/2020/10/26/fit-urls">fit URLs</a> by asking about bookmarks and original URLs. <a href="/2020/10/26/fit-urls#b0fb43ff9aba4c14a075e7effc9fae25">Daniel Sklenitzka's question</a> is a good example: <blockquote> <p> "I see how signing the URLs prevents clients from retro-engineering the URL templates, but how does it help preventing breaking changes? If the client stores the whole URL instead of just the ID and later the URL changes because a restaurant ID is added, the original URL is still broken, isn't it?" </p> </blockquote> While I answered the question on the same page, I think that it's worthwhile to expand it. </p> <h3 id="3e1e7cd6b79e4e7b964a96a527312ab0"> The rules of HTTP <a href="#3e1e7cd6b79e4e7b964a96a527312ab0" title="permalink">#</a> </h3> <p> I agree with the implicit assumption that clients are allowed to bookmark links. It seems, then, like a breaking change if you later change your internal URL scheme. That seems to imply that the bookmarked URL is gone, breaking a tenet of the HTTP protocol: <a href="https://www.w3.org/Provider/Style/URI">Cool URIs don't change</a>. </p> <p> REST APIs are supposed to play by the rules of HTTP, so it'd seem that once you've published a URL, you can never retire it. You can, on the other hand, change its behaviour. </p> <p> Let's call such URLs <em>legacy URLs</em>. Keep them around, but change them to return <code>301 Moved Permanently</code> responses. </p> <p> The rules of REST go both ways. The API is expected to play by the rules of HTTP, and so are the clients. Clients are not only expected to follow links, but also redirects. If a legacy URL starts returning a <code>301 Moved Permanently</code> response, a well-behaved client doesn't break. </p> <h3 id="96913fb84fb344ccae75e1e2542ed51c"> Reverse proxy <a href="#96913fb84fb344ccae75e1e2542ed51c" title="permalink">#</a> </h3> <p> As I've <a href="/2020/06/01/retiring-old-service-versions#bc73620aa71141b6a74f4a4aaf395d75">previously described</a>, one of the many benefits of HTTP-based services is that you can put a <a href="https://en.wikipedia.org/wiki/Reverse_proxy">reverse proxy</a> in front of your application servers. I've no idea how to configure or operate <a href="https://en.wikipedia.org/wiki/Nginx">NGINX</a> or <a href="https://en.wikipedia.org/wiki/Varnish_(software)">Varnish</a>, but from talking to people who do know, I get the impression that they're quite scriptable. </p> <p> Since the above ideas are independent of actual service implementation or behaviour, it's a generic problem that you should seek to address with general-purpose software. </p> <p> <img src="/content/binary/reverse-proxy-based-redirect.png" alt="Sequence diagram showing a reverse proxy returning a redirect response to a request for a legacy URL."> </p> <p> Imagine that a reverse proxy is configured with a set of rules that detects legacy URLs and knows how to forward them. Clearly, the reverse proxy must know of the REST API's current URL scheme to be able to do that. You might think that this would entail leaking an implementation detail, but just as I consider any database used by the API as part of the overall system, I'd consider the reverse proxy as just another part. </p> <h3 id="410c6db0f3a54e969c1f54aa9f1ce59e"> Redirecting with ASP.NET <a href="#410c6db0f3a54e969c1f54aa9f1ce59e" title="permalink">#</a> </h3> <p> If you don't have a reverse proxy, you can also implement redirects in code. It'd be better to use something like a reverse proxy, because that would mean that you get to delete code from your code base, but sometimes that's not possible. </p> <p> In ASP.NET, you can return <code>301 Moved Permanently</code> responses just like any other kind of HTTP response: </p> <p> <pre>[<span style="color:#2b91af;">Obsolete</span>(<span style="color:#a31515;">&quot;Use&nbsp;Get&nbsp;method&nbsp;with&nbsp;restaurant&nbsp;ID.&quot;</span>)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;calendar/{year}/{month}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;LegacyGet(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">RedirectToActionResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(Get), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;restaurantId&nbsp;=&nbsp;<span style="color:#2b91af;">Grandfather</span>.Id,&nbsp;year,&nbsp;month&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;permanent:&nbsp;<span style="color:blue;">true</span>); }</pre> </p> <p> This <code>LegacyGet</code> method redirects to the current Controller action called <code>Get</code> by supplying the arguments that the new method requires. The <code>Get</code> method has this signature: </p> <p> <pre>[<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/calendar/{year}/{month}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month)</pre> </p> <p> When I expanded the API from a single restaurant to a multi-tenant system, I had to <a href="https://en.wikipedia.org/wiki/Grandfather_clause">grandfather in</a> the original restaurant. I gave it a <code>restaurantId</code>, but in order to not put <a href="https://en.wikipedia.org/wiki/Magic_number_(programming)">magic constants</a> in the code, I defined it as the named constant <code>Grandfather.Id</code>. </p> <p> Notice that I also adorned the <code>LegacyGet</code> method with an <code>[Obsolete]</code> attribute to make it clear to maintenance programmers that this is legacy code. You might argue that the <em>Legacy</em> prefix already does that, but the <code>[Obsolete]</code> attribute will make the compiler emit a warning, which is <a href="/2011/04/29/Feedbackmechanismsandtradeoffs">even better feedback</a>. </p> <h3 id="69aad25084a5452b96e14bc81a28a53c"> Regression test <a href="#69aad25084a5452b96e14bc81a28a53c" title="permalink">#</a> </h3> <p> While legacy URLs may be just that: legacy, that doesn't mean that it doesn't matter whether or not they work. You may want to add regression tests. </p> <p> If you implement redirects in code (as opposed to a reverse proxy), you should also add automated tests that verify that the redirects work: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;http://localhost/calendar/2020?sig=ePBoUg5gDw2RKMVWz8KIVzF%2Fgq74RL6ynECiPpDwVks%3D&quot;</span>)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;http://localhost/calendar/2020/9?sig=ZgxaZqg5ubDp0Z7IUx4dkqTzS%2Fyjv6veDUc2swdysDU%3D&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;BookmarksStillWork(<span style="color:blue;">string</span>&nbsp;bookmarkedAddress) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;api&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LegacyApi</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateDefaultClient().GetAsync(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(bookmarkedAddress)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(<span style="color:#2b91af;">HttpStatusCode</span>.MovedPermanently,&nbsp;actual.StatusCode); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;follow&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;api.CreateClient().GetAsync(actual.Headers.Location); &nbsp;&nbsp;&nbsp;&nbsp;follow.EnsureSuccessStatusCode(); }</pre> </p> <p> This test interacts with a self-hosted service at the HTTP level. <code>LegacyApi</code> is a test-specific helper class that derives from <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.testing.webapplicationfactory-1">WebApplicationFactory&lt;Startup&gt;</a>. </p> <p> The test uses URLs that I 'bookmarked' before I evolved the URLs to a multi-tenant system. As you can tell from the host name (<code>localhost</code>), these are bookmarks against the self-hosted service. The test first verifies that the response is <code>301 Moved Permanently</code>. It then requests the new address and <a href="/2020/09/28/ensuresuccessstatuscode-as-an-assertion">uses EnsureSuccessStatusCode as an assertion</a>. </p> <h3 id="f16f3cd819804e54add83ad32aaff188"> Conclusion <a href="#f16f3cd819804e54add83ad32aaff188" title="permalink">#</a> </h3> <p> When you evolve fit URLs, it could break clients that may have bookmarked legacy URLs. Consider leaving <code>301 Moved Permanently</code> responses at those addresses. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Checking signed URLs with ASP.NET https://blog.ploeh.dk/2020/11/09/checking-signed-urls-with-aspnet 2020-11-09T12:19:00+00:00 Mark Seemann <div id="post"> <p> <em>Use a filter to check all requested URL signatures.</em> </p> <p> This article is part of <a href="/2020/10/26/fit-urls">a short series on fit URLs</a>. In the overview article, I argued that you should be signing URLs in order to prevent your REST APIs from becoming victims of <a href="https://www.hyrumslaw.com">Hyrum's law</a>. <a href="/2020/11/02/signing-urls-with-aspnet">In the previous article</a> you saw how to sign URLs with ASP.NET. </p> <p> In this article you'll see how to check the URLs of all HTTP requests to the API and reject those that aren't up to snuff. </p> <h3 id="beb942e872e6470e94777e228967b58a"> Filter <a href="#beb942e872e6470e94777e228967b58a" title="permalink">#</a> </h3> <p> If you want to intercept all incoming HTTP requests in ASP.NET Core, an <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.filters.iasyncactionfilter">IAsyncActionFilter</a> is a good option. This one should look at the URL of all incoming HTTP requests and detect if the client tried to tamper with it. </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IAsyncActionFilter</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>(<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;comes&nbsp;here...</span></pre> </p> <p> The interface only defines a single method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;OnActionExecutionAsync( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutionDelegate</span>&nbsp;next) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsGetHomeRequest(context)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;strippedUrl&nbsp;=&nbsp;GetUrlWithoutSignature(context); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(SignatureIsValid(strippedUrl,&nbsp;context)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;context.Result&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); }</pre> </p> <p> While the rule is to reject requests with invalid signatures, there's one exception. The 'home' resource requires no signature, as this is the only publicly documented URL for the API. Thus, if <code>IsGetHomeRequest</code> returns <code>true</code>, the filter invokes the <code>next</code> delegate and returns. </p> <p> Otherwise, it strips the signature off the URL and checks if the signature is valid. If it is, it again invokes the <code>next</code> delegate and returns. </p> <p> If the signature is invalid, on the other hand, the filter stops further execution by <em>not</em> invoking <code>next</code>. Instead, it sets the response to a <code>404 Not Found</code> result. </p> <p> It may seem odd to return <code>404 Not Found</code> if the signature is invalid. Wouldn't <code>401 Unauthorized</code> or <code>403 Forbidden</code> be more appropriate? </p> <p> Not really. Keep in mind that while this behaviour may use encryption technology, it's not a security feature. The purpose is to make it impossible for clients to 'retro-engineer' an implied interface. This protects them from breaking changes in the future. Clients are supposed to follow links, and the URLs given by the API itself are proper, existing URLs. If you try to edit a URL, then that URL doesn't work. It represents a resource that doesn't exist. While it may seem surprising at first, I find that a <code>404 Not Found</code> result is the most appropriate status code to return. </p> <h3 id="3584198282b64c85ab5872493da8149d"> Detecting a home request <a href="#3584198282b64c85ab5872493da8149d" title="permalink">#</a> </h3> <p> The <code>IsGetHomeRequest</code> helper method is straightforward: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsGetHomeRequest(<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;context.HttpContext.Request.Path&nbsp;==&nbsp;<span style="color:#a31515;">&quot;/&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp;&nbsp;context.HttpContext.Request.Method&nbsp;==&nbsp;<span style="color:#a31515;">&quot;GET&quot;</span>; }</pre> </p> <p> This predicate only looks at the <code>Path</code> and <code>Method</code> of the incoming request. Perhaps it also ought to check that the URL has no query string parameters, but I'm not sure if that actually matters. </p> <h3 id="c5f674b301404a2f95177d4c1dc73cc0"> Stripping off the signature <a href="#c5f674b301404a2f95177d4c1dc73cc0" title="permalink">#</a> </h3> <p> The <code>GetUrlWithoutSignature</code> method strips off the signature query string variable from the URL, but leaves everything else intact: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;GetUrlWithoutSignature(<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restOfQuery&nbsp;=&nbsp;<span style="color:#2b91af;">QueryString</span>.Create( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;context.HttpContext.Request.Query.Where(x&nbsp;=&gt;&nbsp;x.Key&nbsp;!=&nbsp;<span style="color:#a31515;">&quot;sig&quot;</span>)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;context.HttpContext.Request.GetEncodedUrl(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ub&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>(url); &nbsp;&nbsp;&nbsp;&nbsp;ub.Query&nbsp;=&nbsp;restOfQuery.ToString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ub.Uri.AbsoluteUri; }</pre> </p> <p> The purpose of removing only the <code>sig</code> query string parameter is that it restores the rest of the URL to the value that it had when it was signed. This enables the <code>SignatureIsValid</code> method to recalculate the <a href="https://en.wikipedia.org/wiki/HMAC">HMAC</a>. </p> <h3 id="4fa6c27d6b504f2baf4b27b2f431851a"> Validating the signature <a href="#4fa6c27d6b504f2baf4b27b2f431851a" title="permalink">#</a> </h3> <p> The <code>SignatureIsValid</code> method validates the signature: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;SignatureIsValid(<span style="color:blue;">string</span>&nbsp;candidate,&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;context.HttpContext.Request.Query[<span style="color:#a31515;">&quot;sig&quot;</span>]; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;receivedSignature&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.FromBase64String(sig.ToString()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;computedSignature&nbsp;=&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(candidate)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;signaturesMatch&nbsp;=&nbsp;computedSignature.SequenceEqual(receivedSignature); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;signaturesMatch; }</pre> </p> <p> If the <code>receivedSignature</code> equals the <code>computedSignature</code> the signature is valid. </p> <p> This prevents clients from creating URLs based on implied templates. Since clients don't have the signing key, they can't compute a valid HMAC, and therefore the URLs they'll produce will fail the integrity test. </p> <h3 id="8c8d2409a9a647b6a4d9c2970a02a9c7"> Configuration <a href="#8c8d2409a9a647b6a4d9c2970a02a9c7" title="permalink">#</a> </h3> <p> As is the case for the URL-signing feature, you'll first need to read the signing key from the configuration system. This is the same key used to sign URLs: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;urlSigningKey&nbsp;=&nbsp;<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes( &nbsp;&nbsp;&nbsp;&nbsp;Configuration.GetValue&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">&quot;UrlSigningKey&quot;</span>));</pre> </p> <p> Next, you'll need to register the filter with the ASP.NET framework: </p> <p> <pre>services.AddControllers(opts&nbsp;=&gt;&nbsp;opts.Filters.Add(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlIntegrityFilter</span>(urlSigningKey)));</pre> </p> <p> This is typically done in the <code>ConfigureServices</code> method of the <code>Startup</code> class. </p> <h3 id="e032ab8e735b4e54a733ded5d086bee8"> Conclusion <a href="#e032ab8e735b4e54a733ded5d086bee8" title="permalink">#</a> </h3> <p> With a filter like <code>UrlIntegrityFilter</code> you can check the integrity of URLs on all incoming requests to your REST API. This prevents clients from making up URLs based on an implied interface. This may seem restrictive, but is actually for their own benefit. When they can't assemble URLs from scratch, the only remaining option is to follow the links that the API provides. </p> <p> This enables you to evolve the API without breaking existing clients. While client developers may not initially appreciate having to follow links instead of building URLs out of templates, they may value that their clients don't break as you evolve the API. </p> <p> <strong>Next:</strong> <a href="/2020/11/16/redirect-legacy-urls">Redirect legacy URLs</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Signing URLs with ASP.NET https://blog.ploeh.dk/2020/11/02/signing-urls-with-aspnet 2020-11-02T08:20:00+00:00 Mark Seemann <div id="post"> <p> <em>A few Decorators is all it takes.</em> </p> <p> This article is part of <a href="/2020/10/26/fit-urls">a short series on fit URLs</a>. In the overview article, I argued that you should be signing URLs in order to prevent your REST APIs from becoming victims of <a href="https://www.hyrumslaw.com">Hyrum's law</a>. </p> <p> In this article, you'll see how to do this with ASP.NET Core 3.1, and in the next article you'll see how to check URL integrity. </p> <h3 id="8d83a9fd4dec48efa79455d6f3356726"> SigningUrlHelper <a href="#8d83a9fd4dec48efa79455d6f3356726" title="permalink">#</a> </h3> <p> I wanted the URL-signing functionality to slot into the ASP.NET framework, which supplies the <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.iurlhelper">IUrlHelper</a> interface for the purpose of creating URLs. (For example, the <a href="/2020/08/10/an-aspnet-core-url-builder">UrlBuilder I recently described</a> relies on that interface.) </p> <p> Since it's an interface, you can define a <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorator</a> around it: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IUrlHelper</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>(<span style="color:#2b91af;">IUrlHelper</span>&nbsp;inner,&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.inner&nbsp;=&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;comes&nbsp;here...</span></pre> </p> <p> As you can tell, this Decorator requires an <code>inner</code> <code>IUrlHelper</code> and a <code>urlSigningKey</code>. Most of the members just delegate to the <code>inner</code> implementation: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;IsLocalUrl(<span style="color:blue;">string</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;inner.IsLocalUrl(url); }</pre> </p> <p> The <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.iurlhelper.action">Action</a> method creates URLs, so this is the method to modify: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Action(<span style="color:#2b91af;">UrlActionContext</span>&nbsp;actionContext) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;inner.Action(actionContext); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsLocalUrl(url)) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;b&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ActionContext.HttpContext.Request.Host.ToUriComponent()); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(b.Uri,&nbsp;url).AbsoluteUri; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ub&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UriBuilder</span>(url); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">using</span>&nbsp;<span style="color:blue;">var</span>&nbsp;hmac&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HMACSHA256</span>(urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sig&nbsp;=&nbsp;<span style="color:#2b91af;">Convert</span>.ToBase64String( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;hmac.ComputeHash(<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes(url))); &nbsp;&nbsp;&nbsp;&nbsp;ub.Query&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">QueryString</span>(ub.Query).Add(<span style="color:#a31515;">&quot;sig&quot;</span>,&nbsp;sig).ToString(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;ub.ToString(); }</pre> </p> <p> The <code>actionContext</code> may sometimes indicate a local (relative) URL, in which case I wanted to convert it to an absolute URL. Once that's taken care of, the method calculates an <a href="https://en.wikipedia.org/wiki/HMAC">HMAC</a> and adds it as a query string variable. </p> <h3 id="37abf2a288444c9cab0541f6acafda8b"> SigningUrlHelperFactory <a href="#37abf2a288444c9cab0541f6acafda8b" title="permalink">#</a> </h3> <p> While it's possible take ASP.NET's default <code>IUrlHelper</code> instance (e.g. from <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.controllerbase.url">ControllerBase.Url</a>) and manually decorate it with <code>SigningUrlHelper</code>, that doesn't slot seamlessly into the framework. </p> <p> For example, to add the <code>Location</code> header that you saw in <a href="/2020/10/26/fit-urls">the previous article</a>, the code is this: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;Reservation201Created( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Reservation</span>&nbsp;r) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CreatedAtActionResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(Get), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">null</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;restaurantId,&nbsp;id&nbsp;=&nbsp;r.Id.ToString(<span style="color:#a31515;">&quot;N&quot;</span>)&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;r.ToDto()); }</pre> </p> <p> The method just returns a new <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.createdatactionresult">CreatedAtActionResult</a> object, and the framework takes care of the rest. No explicit <code>IUrlHelper</code> object is used, so there's nothing to manually decorate. By default, then, the URLs created from such <code>CreatedAtActionResult</code> objects aren't signed. </p> <p> It turns out that the ASP.NET framework uses an interface called <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.routing.iurlhelperfactory">IUrlHelperFactory</a> to create <code>IUrlHelper</code> objects. Decorate that as well: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>(<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;inner,&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.inner&nbsp;=&nbsp;inner; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.urlSigningKey&nbsp;=&nbsp;urlSigningKey; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;GetUrlHelper(<span style="color:#2b91af;">ActionContext</span>&nbsp;context) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;inner.GetUrlHelper(context); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelper</span>(url,&nbsp;urlSigningKey); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> Straightforward: use the <code>inner</code> object to get an <code>IUrlHelper</code> object, and decorate it with <code>SigningUrlHelper</code>. </p> <h3 id="92c13952cd48418c81c6925d9af15635"> Configuration <a href="#92c13952cd48418c81c6925d9af15635" title="permalink">#</a> </h3> <p> The final piece of the puzzle is to tell the framework about the <code>SigningUrlHelperFactory</code>. You can do this in the <code>Startup</code> class' <code>ConfigureServices</code> method. </p> <p> First, read the signing key from the configuration system (e.g. a configuration file): </p> <p> <pre><span style="color:blue;">var</span>&nbsp;urlSigningKey&nbsp;=&nbsp;<span style="color:#2b91af;">Encoding</span>.ASCII.GetBytes( &nbsp;&nbsp;&nbsp;&nbsp;Configuration.GetValue&lt;<span style="color:blue;">string</span>&gt;(<span style="color:#a31515;">&quot;UrlSigningKey&quot;</span>));</pre> </p> <p> Then use the signing key to configure the <code>SigningUrlHelperFactory</code> service. Here, I wrapped that in a little helper method: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ConfigureUrSigning( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IServiceCollection</span>&nbsp;services, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">byte</span>[]&nbsp;urlSigningKey) { &nbsp;&nbsp;&nbsp;&nbsp;services.RemoveAll&lt;<span style="color:#2b91af;">IUrlHelperFactory</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;services.AddSingleton&lt;<span style="color:#2b91af;">IUrlHelperFactory</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">SigningUrlHelperFactory</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlHelperFactory</span>(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;urlSigningKey)); }</pre> </p> <p> This method first removes the default <code>IUrlHelperFactory</code> service and then adds the <code>SigningUrlHelperFactory</code> instead. It decorates <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.routing.urlhelperfactory">UrlHelperFactory</a>, which is the default built-in implementation of the interface. </p> <h3 id="2efe02caa55742679a953a398c235bc5"> Conclusion <a href="#2efe02caa55742679a953a398c235bc5" title="permalink">#</a> </h3> <p> You can extend the ASP.NET framework to add a signature to all the URLs it generates. All it takes is two simple Decorators. </p> <p> <strong>Next:</strong> <a href="/2020/11/09/checking-signed-urls-with-aspnet">Checking signed URLs with ASP.NET</a>. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Fit URLs https://blog.ploeh.dk/2020/10/26/fit-urls 2020-10-26T06:19:00+00:00 Mark Seemann <div id="post"> <p> <em>Keep REST API URLs evolvable. A way to address Hyrum's law.</em> </p> <p> Publishing and maintaining a true (<a href="https://martinfowler.com/articles/richardsonMaturityModel.html">level 3</a>) RESTful API is difficult. This is the style of REST design where clients are expected to <em>follow links</em> to perform the work they want to accomplish; not assemble URLs from templates. </p> <p> Have you ever designed a URL scheme and published it, only to later discover that you wished you'd come up with a different structure? If you've published a set of URL templates, changing your mind constitutes a breaking change. If clients follow links, however, URLs are opaque and you can redesign the URLs without breaking existing clients. </p> <p> You can try to document this design principle all you want, to no avail. You can <em>tell</em> client developers that they're supposed to follow links, not try to <a href="/2013/05/01/rest-lesson-learned-avoid-hackable-urls">retro-engineer the URLs</a>, and still they'll do it. </p> <p> I know; I've experienced it. When we later changed the URL structure, it didn't take long for the client developers to complain that we broke their code. </p> <h3 id="f9591f3f57534243afc883e166727635"> Hyrum's law <a href="#f9591f3f57534243afc883e166727635" title="permalink">#</a> </h3> <p> This is an example of <a href="https://www.hyrumslaw.com">Hyrum's law</a> in action, albeit on the scale of web service interactions, rather than low-level APIs. The presence of a discernible system to URLs suggests an <em>implicit</em> interface. </p> <p> Consider this 'home' resource for an online restaurant reservation system: </p> <p> <pre>GET / HTTP/1.1 HTTP/1.1 200 OK Content-Type: application/json; charset=utf-8 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/reservations&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/10&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/10/23&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> It doesn't take much interaction with the API before you realise that there's a system to the URLs provided in the links. If you want to see the calendar for a specific date, you can easily retro-engineer the URL template <code>/calendar/{yyyy}/{MM}/{dd}</code>, and <code>/calendar/{yyyy}/{MM}</code> for a month, and so on. </p> <p> The same is likely to happen with the <em>reservations</em> link. You can <code>POST</code> to this link to make a new reservation: </p> <p> <pre>POST /reservations HTTP/1.1 Content-Type: application/json { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-12-09 19:15&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;rainboughs@example.com&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Raine&nbsp;Burroughs&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;5 } HTTP/1.1 201 Created Content-Type: application/json Location: http://localhost:53568/reservations/fabc5bf63a1a4db38b95deaa89c01178 { &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;id&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;fabc5bf63a1a4db38b95deaa89c01178&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;at&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-12-09T19:15:00.0000000&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;email&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;rainboughs@example.com&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;name&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;Raine&nbsp;Burroughs&quot;</span>, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;quantity&quot;</span>:&nbsp;5 }</pre> </p> <p> Notice that when the API responds, its <code>Location</code> header gives you the URL for that particular reservation. It doesn't take long to figure out that there's a template there, as well: <code>/reservations/{id}</code>. </p> <p> So client developers may just store the ID (<code>fabc5bf63a1a4db38b95deaa89c01178</code>) and use the implied template to construct URLs on the fly. And who can blame them? </p> <p> That, however, misses the point of REST. The ID of that reservation isn't <code>fabc5bf63a1a4db38b95deaa89c01178</code>, but rather <code>http://localhost:53568/reservations/fabc5bf63a1a4db38b95deaa89c01178</code>. Yes, the URL, all of it, is the ID. </p> <h3 id="fff55f373ff24a4da6cd4e50dd8b97bd"> Evolving URLs <a href="#fff55f373ff24a4da6cd4e50dd8b97bd" title="permalink">#</a> </h3> <p> Why does that matter? </p> <p> It matters because you're human, and you make mistakes. Or, rather, it's intrinsic to software development that you learn as you work. You'll make decisions at the beginning that you'll want to change as you gain more insight. </p> <p> Also, requirements change. Consider the URL template scheme implied by the above examples. Can you spot any problems? Would you want to change anything? </p> <p> Imagine, for example, that you've already deployed the first version of the API. It's a big success. Now the product owner wants to expand the market to more restaurants. She wants to make the service a multi-tenant API. How does that affect URLs? </p> <p> In that new context, perhaps URLs like <code>/restaurants/1/reservations</code> or <code>/restaurants/90125/calendar/2020/10</code> would be better. </p> <p> That, however, would be a breaking change if clients construct URLs based on implied templates. </p> <p> Couldn't you just pass the restaurant ID as an HTTP header instead of in the URL? Yes, technically you could do that, but that doesn't work well with HTTP caching. It's not a RESTful thing to do, for that, and other, reasons. </p> <h3 id="41cdd42bac1b4bd5a573070ee27e902d"> Fitness <a href="#41cdd42bac1b4bd5a573070ee27e902d" title="permalink">#</a> </h3> <p> Do we just give up in the face of Hyrum's law? Or can we keep URLs evolvable? In evolution, organisms evolve according to a 'fitness function', so to name such URLs, we could call them <em>fit URL</em>. </p> <p> To keep URLs fit, we must prevent client developers from retro-engineering the implied interface. My original thought was to give each URL an opaque ID, such as a GUID, but <a href="/2013/05/01/rest-lesson-learned-avoid-hackable-urls#424bd3f6199e422c8294e300d312ffb4">in 2015 Dan Kubb instead suggested to sign the URLs</a>. What a great idea! </p> <p> If you do that, then the above home resource might look like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/reservations?sig=1WiLlS5705bfsffPzaFYLwntrS4FCjE5CLdaeYTHxxg%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020?sig=eIFuUkb6WprPrp%2B4HPSPaavcUdwVjeG%2BKVrIRqDs9OI%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020/10?sig=mGqkAjY7vMbC5Fr7UiRXWWnjn3pFn21MYrMagpdWaU0%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/restaurants/1/calendar/2020/10/23?sig=Ua5F%2FucP6zmAy219cHa4WG7zIcCa0hgVD5ModXcNQuo%3D&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> Even if you can still figure out what the URL templates are, it doesn't avail you. Creating a new reservation may return a URL like <code>https://localhost:53568/restaurants/1/reservations/fabc5bf63a1a4db38b95deaa89c01178?sig=8e80PmVi8aSS1UH6iSJ73nHmOsCLrUMs7yggEOkvEqo%3D</code>, but you can't just replace the ID with another ID and expect it to work: </p> <p> <pre>GET /restaurants/1/reservations/79520877ef4f4acdb69838e22ad04510?sig=8e80PmVi8aSS1UH6iSJ73nHmOsCLrUMs7yggEOkvEqo%3D HTTP/1.1 HTTP/1.1 404 Not Found</pre> </p> <p> You're requesting a URL that doesn't exist, so the result is <code>404 Not Found</code>. To be clear: yes, there <em>is</em> a reservation with the ID <code>79520877ef4f4acdb69838e22ad04510</code>, but its URL isn't the above URL. </p> <h3 id="104b052da10e463297020c109dec705d"> ASP.NET implementation <a href="#104b052da10e463297020c109dec705d" title="permalink">#</a> </h3> <p> In two articles, I'll show you how to implement <em>fit URLs</em> in ASP.NET Core. <ul> <li><a href="/2020/11/02/signing-urls-with-aspnet">Signing URLs with ASP.NET</a></li> <li><a href="/2020/11/09/checking-signed-urls-with-aspnet">Checking signed URLs with ASP.NET</a></li> <li><a href="/2020/11/16/redirect-legacy-urls">Redirect legacy URLs</a></li> </ul> The ASP.NET framework comes with enough extensibility points to make this a non-intrusive operation. I implemented this using a filter and a few <a href="https://en.wikipedia.org/wiki/Decorator_pattern">Decorators</a> in a way so that you can easily turn the feature on or off. </p> <h3 id="ff0ced83b9fb49e8835fc92e08dd2859"> Conclusion <a href="#ff0ced83b9fb49e8835fc92e08dd2859" title="permalink">#</a> </h3> <p> One of the major benefits of true RESTful API design is that it's evolvable. It enables you to learn and improve as you go, without breaking existing clients. </p> <p> You have to take care, however, that clients don't retro-engineer the URL templates that you may be using for implementation purposes. You want to be able to change URLs in the future. </p> <p> Hyrum's law suggests that clients will rely on undocumented features if they can. By signing the URLs you keep them fit to evolve. </p> <p> <strong>Next:</strong> <a href="/2020/11/02/signing-urls-with-aspnet">Signing URLs with ASP.NET</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="b0fb43ff9aba4c14a075e7effc9fae25"> <div class="comment-author"><a href="https://github.com/Skleni">Daniel Sklenitzka</a></div> <div class="comment-content"> <p> I see how signing the URLs prevents clients from retro-engineering the URL templates, but how does it help preventing breaking changes? If the client stores the whole URL instead of just the ID and later the URL changes because a restaurant ID is added, the original URL is still broken, isn't it? </p> </div> <div class="comment-date">2020-10-27 07:20 UTC</div> </div> <div class="comment" id="848275a366db40e2a35810554633447f"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Daniel, thank you for writing. You're correct when assuming that clients are allowed to 'bookmark' URLs. An implicit assumption that I didn't explicitly state is that clients are supposed to follow not only links, but also redirects. Thus, to avoid breaking changes, it's the API's responsibility to leave a <code>301 Moved Permanently</code> response behind at the old address. </p> <p> As a service owner, though, you have some flexibility in how to achieve this. You can code this into the service code itself, but another option might be to use a reverse proxy for such purposes. One of the many advantages of REST is that you can offload a lot HTTP-level behaviour on standard networking software; <a href="/2020/06/01/retiring-old-service-versions#bc73620aa71141b6a74f4a4aaf395d75">here's another example</a>. </p> </div> <div class="comment-date">2020-10-28 7:15 UTC</div> </div> <div class="comment" id="880c4cda8ab14b1d8de05438becac2f1"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> I think I have the same confusion as Daniel. </p> <blockquote> <p> If you've published a set of URL templates, changing your mind constitutes a breaking change. If clients follow links, however, URLs are opaque and you can redesign the URLs without breaking existing clients. </p> <p> ... </p> <p> You're correct when assuming that clients are allowed to 'bookmark' URLs. An implicit assumption that I didn't explicitly state is that clients are supposed to follow not only links, but also redirects. Thus, to avoid breaking changes, it's the API's responsibility to leave a <code>301 Moved Permanently</code> response behind at the old address. </p> </blockquote> <p> Is there a certain kind of breaking change that exists for a level 2 API that doesn't exist for a level 3 API? </p> </div> <div class="comment-date">2020-10-28 22:55 UTC</div> </div> <div class="comment" id="aae1427ebab14c428291984efb06fd94"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I haven't thought much about whether there are categories of errors that differ between the levels, but I think that in practice, there's a difference in mindset. </p> <p> With a level 2 API, I think most people are still caught in a mindset that's largely a projection of RPC. URL templates map easily to programming procedures (e.g. object-oriented methods). Thus, you have a mental model that includes <code>PostReservation</code>, <code>GetCalendar</code>, etc. People think of <em>that</em> as being the API. With that mindset, I don't think that many client developers configure their HTTP clients to follow redirects. Thus, one could argue that changing URLs are breaking changes for a level 2 API, even if you leave <code>301 Moved Permanently</code> responses behind. </p> <p> With level 3 APIs, you encourage client developers to think in terms of 'the web'. That includes following links and redirects. I believe that there's a difference in perception, even if there may not be any technical difference. </p> <p> I do believe, however, that the real advantage is that you impose a smaller maintenance burden on client developers with a level 3 API. Granted, a client developer may have to spend a little more effort up front to follow links, but once a compliant client is in place, it <em>needs no more maintenance</em>. It'll just keep working. </p> <p> Not so with published URL templates. Here you have to entice client developers to <em>actively</em> update their code. This may be impossible if you don't know who the clients are, or if the client software is in maintenance mode. This may make it harder to <a href="/2020/06/01/retiring-old-service-versions">retire old versions</a>. You may be stuck with these old versions forever. </p> </div> <div class="comment-date">2020-10-30 17:32 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Monomorphic functors https://blog.ploeh.dk/2020/10/19/monomorphic-functors 2020-10-19T07:36:00+00:00 Mark Seemann <div id="post"> <p> <em>Non-generic mappable containers, with a realistic example.</em> </p> <p> This article is an instalment in <a href="/2018/03/22/functors">an article series about functors</a>. Previous articles have covered <a href="/2018/03/26/the-maybe-functor">Maybe</a>, <a href="/2018/09/10/the-lazy-functor">Lazy</a>, and other functors. This article looks at what happens when you weaken one of the conditions that the article series so far has implied. </p> <p> In the <a href="/2018/03/22/functors">introductory article</a>, I wrote: <blockquote> As a rule of thumb, if you have a type with a generic type argument, it's a candidate to be a functor. </blockquote> That still holds, but then <a href="https://about.me/tysonwilliams">Tyson Williams</a> <a href="/2018/03/22/functors#dee7ec216a464248b6103ea0979948ab">asks if that's a required condition</a>. It turns out that it isn't. In this article, you'll learn about the implications of weakening this condition. </p> <p> As is my habit with many of the articles in this article series, I'll start by uncovering the structure of the concept, and only later show a more realistic example. </p> <h3 id="bea56fadde1e4e9da8389a6c31937216"> Mapping strings <a href="#bea56fadde1e4e9da8389a6c31937216" title="permalink">#</a> </h3> <p> So far in this article series, you've seen examples of <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">containers</a> with a generic type argument: <code>Tree&lt;T&gt;</code>, <code>Task&lt;T&gt;</code>, and so on. Furthermore, you've seen how a functor is a container with a <em>structure-preserving map</em>. This function has various names in different languages: <code>Select</code>, <code>fmap</code>, <code>map</code>, etcetera. </p> <p> Until now, you've only seen examples where this mapping enables you to translate from one type argument to another. You can, for example, translate the characters in a string to Boolean values, like this: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()) Enumerable.WhereSelectEnumerableIterator&lt;char, bool&gt; &nbsp;&nbsp;{ false, true, false, true, false, false, false, true, false, false, false, true, false, false }</pre> </p> <p> This works because in C#, <a href="https://docs.microsoft.com/dotnet/api/system.string">the String class</a> implements various interfaces, among these <code>IEnumerable&lt;char&gt;</code>. By treating a <code>string</code> as an <code>IEnumerable&lt;char&gt;</code>, you can map each element. That's the standard IEnumerable functor (AKA <em>the list functor</em>). </p> <p> What if you'd like to map the characters in a string to other characters? Perhaps you'd like to map vowels to upper case, and all other characters to lower case. You could try this: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)) Enumerable.WhereSelectEnumerableIterator&lt;char, char&gt; &nbsp;&nbsp;{ 's', 'A', 'f', 'E', ' ', 'f', 'r', 'O', 'm', ' ', 'h', 'A', 'r', 'm' }</pre> </p> <p> That sort of works, but as you can tell, the result isn't a <code>string</code>, it's an <code>IEnumerable&lt;char&gt;</code>. </p> <p> This isn't a big problem, because one of the <code>string</code> constructor overloads take a <code>char</code> array as input, so you can do this: </p> <p> <pre>&gt; <span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>&nbsp;(<span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.<span style="color:#2b91af;">Select</span>(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)).<span style="color:#2b91af;">ToArray</span>()) "sAfE frOm hArm"</pre> </p> <p> It isn't the prettiest, but it gets the job done. </p> <h3 id="fd3da7dcd560415d81ab82bfb772d94c"> Monomorphic functor in C# <a href="#fd3da7dcd560415d81ab82bfb772d94c" title="permalink">#</a> </h3> <p> If you contemplate the last example, you may arrive at the conclusion that you could package some of that boilerplate code in a reusable function. Since we're already talking about functors, why not call it <code>Select</code>? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Select(<span style="color:blue;">this</span>&nbsp;<span style="color:blue;">string</span>&nbsp;source,&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:blue;">char</span>,&nbsp;<span style="color:blue;">char</span>&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">string</span>(source.AsEnumerable().Select(selector).ToArray()); }</pre> </p> <p> It somewhat simplifies things: </p> <p> <pre>&gt; <span style="color:#a31515;">&quot;Safe&nbsp;From&nbsp;Harm&quot;</span>.Select(c&nbsp;=&gt;&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c)) "sAfE frOm hArm"</pre> </p> <p> Since I deliberately wrote the <code>Select</code> method in the style of other <code>Select</code> methods (apart from the generics), you may wonder if C# query syntax also works? </p> <p> <pre>&gt; <span style="color:blue;">from</span>&nbsp;c&nbsp;<span style="color:blue;">in</span>&nbsp;<span style="color:#a31515;">&quot;Army&nbsp;of&nbsp;Me&quot;</span> . <span style="color:blue;">select</span>&nbsp;c.IsVowel()&nbsp;?&nbsp;<span style="color:blue;">char</span>.ToUpper(c)&nbsp;:&nbsp;<span style="color:blue;">char</span>.ToLower(c) "ArmY Of mE"</pre> </p> <p> It compiles and works! The C# compiler understands monomorphic containers! </p> <p> I admit that I was quite surprised when I first tried this out. </p> <h3 id="8ef3808c11a34d9aaa1c25b94539f843"> Monomorphic functor in Haskell <a href="#8ef3808c11a34d9aaa1c25b94539f843" title="permalink">#</a> </h3> <p> Surprisingly, in this particular instance, C# comes out looking more flexible than <a href="https://www.haskell.org">Haskell</a>. This is mainly because in C#, functors are implemented as a special compiler feature, whereas in Haskell, <code>Functor</code> is defined using the general-purpose <em>type class</em> language feature. </p> <p> There's <a href="https://hackage.haskell.org/package/mono-traversable">a package</a> that defines a <code>MonoFunctor</code> type class, as well as some instances. With it, you can write code like this: </p> <p> <pre><span style="color:#2b91af;">ftg</span>&nbsp;::&nbsp;<span style="color:blue;">Text</span> ftg&nbsp;=&nbsp;omap&nbsp;(\c&nbsp;-&gt;&nbsp;<span style="color:blue;">if</span>&nbsp;isVowel&nbsp;c&nbsp;<span style="color:blue;">then</span>&nbsp;toUpper&nbsp;c&nbsp;<span style="color:blue;">else</span>&nbsp;toLower&nbsp;c)&nbsp;<span style="color:#a31515;">&quot;Fade&nbsp;to&nbsp;Grey&quot;</span></pre> </p> <p> Even though <code>Text</code> isn't a <code>Functor</code> instance, it <em>is</em> a <code>MonoFunctor</code> instance. The value of <code>ftg</code> is <code>"fAdE tO grEY"</code>. </p> <p> All the normal functors (<code>[]</code>, <code>Maybe</code>, etc.) are also <code>MonoFunctor</code> instances, since the normal <code>Functor</code> instance is more capable than a <code>MonoFunctor</code>. </p> <h3 id="30dd605a3bb34d1985e1d277e1c9b0a5"> Restaurant example <a href="#30dd605a3bb34d1985e1d277e1c9b0a5" title="permalink">#</a> </h3> <p> While I've introduced the concept of a monomorphic functor with a trivial string example, I actually discovered the C# feature when I was working on some more realistic code. As I often do, I was working on a variation of an online restaurant reservation system. </p> <p> The code base contained a rather complex variation of an implementation of the <a href="/2020/01/27/the-maitre-d-kata">Maître d' kata</a>. The <code>MaitreD</code> constructor looked like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;opensAt, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeOfDay</span>&nbsp;lastSeating, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">TimeSpan</span>&nbsp;seatingDuration, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:#2b91af;">Table</span>&gt;&nbsp;tables)</pre> </p> <p> This had worked well when the system only dealt with a single restaurant, but I was now expanding it to a multi-tenant system and I needed to keep track of some more information about each restaurant, such as its name and ID. While I could have added such information to the <code>MaitreD</code> class, I didn't want to pollute that class with data it didn't need. While the restaurant's opening time and seating duration are necessary for the decision algorithm, the name isn't. </p> <p> So I introduced a wrapper class: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Restaurant</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(<span style="color:blue;">int</span>&nbsp;id,&nbsp;<span style="color:blue;">string</span>&nbsp;name,&nbsp;<span style="color:#2b91af;">MaitreD</span>&nbsp;maitreD) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Id&nbsp;=&nbsp;id; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;name; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MaitreD&nbsp;=&nbsp;maitreD; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">int</span>&nbsp;Id&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Name&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">MaitreD</span>&nbsp;MaitreD&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;More&nbsp;code&nbsp;follows...</span></pre> </p> <p> I also added copy-and-update methods (AKA 'withers'): </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithId(<span style="color:blue;">int</span>&nbsp;newId) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(newId,&nbsp;Name,&nbsp;MaitreD); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithName(<span style="color:blue;">string</span>&nbsp;newName) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(Id,&nbsp;newName,&nbsp;MaitreD); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;WithMaitreD(<span style="color:#2b91af;">MaitreD</span>&nbsp;newMaitreD) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>(Id,&nbsp;Name,&nbsp;newMaitreD); }</pre> </p> <p> Still, if you need to modify the <code>MaitreD</code> within a given <code>Restaurant</code> object, you first have to have a reference to the <code>Restaurant</code> so that you can read its <code>MaitreD</code> property, then you can edit the <code>MaitreD</code>, and finally call <code>WithMaitreD</code>. Doable, but awkward: </p> <p> <pre>restaurant.WithMaitreD(restaurant.MaitreD.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5)))</pre> </p> <p> So I got the idea that I might try to add a structure-preserving map to <code>Restaurant</code>, which I did: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Restaurant</span>&nbsp;Select(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">MaitreD</span>,&nbsp;<span style="color:#2b91af;">MaitreD</span>&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(selector&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(selector)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;WithMaitreD(selector(MaitreD)); }</pre> </p> <p> The first time around, it enabled me to rewrite the above expression as: </p> <p> <pre>restaurant.Select(m&nbsp;=&gt;&nbsp;m.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5)))</pre> </p> <p> That's already a little nicer. It also handles the situation where you may not have a named <code>Restaurant</code> variable you can query; e.g. if you have a method that returns a <code>Restaurant</code> object, and you just want to continue calling into a <a href="https://martinfowler.com/bliki/FluentInterface.html">Fluent API</a>. </p> <p> Then I thought, <em>I wonder if query syntax works, too...</em> </p> <p> <pre><span style="color:blue;">from</span>&nbsp;m&nbsp;<span style="color:blue;">in</span>&nbsp;restaurant <span style="color:blue;">select</span>&nbsp;m.WithSeatingDuration(<span style="color:#2b91af;">TimeSpan</span>.FromHours(.5))</pre> </p> <p> And it <em>does</em> work! </p> <p> I know that a lot of people don't like query syntax, but I think it has certain advantages. In this case, it actually isn't shorter, but it's a nice alternative. I particularly find that if I try to <a href="/2019/11/04/the-80-24-rule">fit my code into a tight box</a>, query syntax sometimes gives me an opportunity to format code over multiple lines in a way that's more natural than with method-call syntax. </p> <h3 id="c386660f74424f33921f35265e61ecf0"> Conclusion <a href="#c386660f74424f33921f35265e61ecf0" title="permalink">#</a> </h3> <p> A monomorphic functor is still a functor, only it constrains the type of mapping you can perform. It can be useful to map monomorphic containers like strings, or immutable nested data structures like the above <code>Restaurant</code> class. </p> <p> <strong>Next:</strong> <a href="/2018/12/03/set-is-not-a-functor">Set is not a functor</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bb0bb9d1bf094c1897f9dd42fb49712c"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> Excellent article! I was planning to eventually write a blog post on this topic, but now there is no need. </p> <blockquote> <p> I know that a lot of people don't like query syntax, but I think it has certain advantages. In this case, it actually isn't shorter, but it's a nice alternative. I particularly find that if I try to <a href="/2019/11/04/the-80-24-rule">fit my code into a tight box</a>, query syntax sometimes gives me an opportunity to format code over multiple lines in a way that's more natural than with method-call syntax. </p> </blockquote> <p> Said another way, one objective measure that is better in this case when using query syntax is that the maximum line width is smaller. It is objective primarily in the sense that maximum line width is objective and secondarily in the sense that we generally agree that code with a lower maximum line width is typically easier to understand. </p> <p> Another such objective measure of quality that I value is the maximum number of matching pairs of parentheses that are nested. This measure improves to two in your query syntax code from three in your previous code. The reason it was three before is because there are four levels of detail and your code decreases the level of detail three times: <code>Restaurant</code> to <code>MaitreD</code>, <code>MaitreD</code> to <code>TimeSpan</code>, and <code>TimeSpan</code> to <code>double</code>. Query syntax is one way to decrease the level of detail without having to introduce a pair of matching parentheses. </p> <p> I find more success minimizing this measure of quality when taking a bottom-up approach. Using the <a href="https://github.com/louthy/language-ext/blob/3df8c837961a502d7a03268b6cac636a57c49056/LanguageExt.Core/DataTypes/Cond/Cond.cs#L56-L70"><code>Apply</code> extension method from <code>language-ext</code></a>, which is like the <a href="https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/symbol-and-operator-reference/#table-of-symbols-and-operators:~:text=Passes%20the%20result%20of%20the%20left%20side%20to%20the%20function%20on%20the%20right%20side%20(forward%20pipe%20operator).">forward pipe operator in F#</a>, we can rewrite this code without any nested pairs of matching parentheses as </p> <p> <pre>.5.Apply(<span style="color:#2b91af;">TimeSpan</span>.FromHours).Apply(m&nbsp;=&gt;&nbsp;m.WithSeatingDuration).Apply(restaurant.Select)</pre> </p> <p> (I did not check if this code compiles. If it does not, it would be because the C# compiler is unsure how to implicitly convert some method group to its intended <code>Func&lt;,&gt;</code> delegate.) This code also has natural line-breaking points before each dot operator, which leads to a comfortable value for the maximum line width. </p> <p> Another advantage of this code that I value is that the execution happens in the same order in which I (as an English speaker) read it: left to right. Your code before using the query syntax executed from right to left as dictated by the matching parentheses. In fact, since the dot operator is left associative, the degree to which execution occurs from left to right is inversely correlated to the number of nested pairs of parentheses. (One confusing thing here is the myriad of different semantic meanings in C# to the syntactic use of a pair of parentheses.) </p> </div> <div class="comment-date">2020-10-19 14:05 UTC</div> </div> <div class="comment" id="8e6b870ce4aa44aa98a16a7969def6e9"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. I've never thought about measuring complexity by nested parentheses, but I like it! It makes intuitive sense. </p> <p> I'm not sure it applies to LISP-like languages, but perhaps some reader comes by some day who can enlighten us. </p> </div> <div class="comment-date">2020-10-21 13:55 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Subjectivity https://blog.ploeh.dk/2020/10/12/subjectivity 2020-10-12T07:40:00+00:00 Mark Seemann <div id="post"> <p> <em>A subjective experience can be sublime, but it isn't an argument.</em> </p> <p> I once wrote <a href="https://amzn.to/36xLycs">a book</a>. One of the most common adjectives people used about it was <em>opinionated</em>. That's okay. It is, I think, a ramification of <a href="/2020/04/13/curb-code-rot-with-thresholds#6c5e4b0817f742bf9ecc0a9050ecb7b5">the way I write</a>. I prioritise writing so that you understand what I mean, and why I mean it. You can call that opinionated if you like. </p> <p> This means that I frequently run into online arguments, mostly on Twitter. This particularly happens when I write something that readers don't like. </p> <h3 id="6a04cf26fdfc483db096de13451a7273"> The sublime <a href="#6a04cf26fdfc483db096de13451a7273" title="permalink">#</a> </h3> <p> Whether or not you <em>like</em> something is a subjective experience. I have nothing against subjective experiences. </p> <p> Subjective experiences fuel art. Before I became a programmer, I wanted to be an artist. I've loved <a href="https://en.wikipedia.org/wiki/Bande_dessin%C3%A9e">French-Belgian</a> comics for as long as I remember (my favourites have always been <a href="https://en.wikipedia.org/wiki/Hermann_Huppen">Hermann</a> and <a href="https://en.wikipedia.org/wiki/Jean_Giraud">Jean Giraud</a>), and until I gave up in my mid-twenties, I wanted to become a graphic novelist. </p> <p> <img src="/content/binary/graphic-novel-pages.jpg" alt="Spread of original pages of my graphic novel."> </p> <p> This interest in drawing also fuelled a similar interest in paintings and art in general. I've had many sublime experiences in front of a work of art in various museums around the world. These were all subjective experiences that were precious to me. </p> <p> For as long as I've been interested in graphic arts, I've also loved music. I've worn <a href="https://en.wikipedia.org/wiki/LP_record">LPs</a> thin in the 1970s and '80s. I've played guitar and wanted to be a rock star. I regularly get goosebumps from listing to outstanding music. These are all subjective experiences. </p> <p> Having an experience is part of the human condition. </p> <p> How you <em>feel</em> about something, however, isn't an argument. </p> <h3 id="c60fc277c32d4e67be30f81d7434da51"> Arguing about subjective experience <a href="#c60fc277c32d4e67be30f81d7434da51" title="permalink">#</a> </h3> <p> I rarely read music reviews. How you experience music is such a personal experience that I find it futile to argue about it. Maybe you share my feelings that <a href="https://en.wikipedia.org/wiki/Army_of_Me">Army of Me</a> is a sublime track, or maybe it leaves you cold. If you don't like that kind of music, I can't argue that you should. </p> <p> Such a discussion generally takes place at the kindergarten level: <em>"Army of Me is a great song!</em>, <em>"No it isn't,"</em> <em>"Yes it is,"</em> and so on. </p> <p> This doesn't mean that we can't talk about music (or, by extension, other subjective experiences). Talking about music can be exhilarating, particularly when you meet someone who shares your taste. Finding common ground, exploring and inspiring each other, learning about artists you didn't know about, is wonderful. It forges bonds between people. I suppose it's the same mechanism in which sports fans find common ground. </p> <p> Sharing subjective experience may not be entirely futile. You can't use your <em>feelings</em> as an argument, but explaining <em>why</em> you feel a certain way can be illuminating. This is the realm of much art criticism. A good, popular and easily digestible example is the <a href="https://strongsongspodcast.com">Strong Songs podcast</a>. The host, Kirk Hamilton, likes plenty of music that I don't. I must admit, though, that his <a href="https://strongsongspodcast.com/satisfied-from-hamilton">exigesis of <em>Satisfied</em> from Hamilton</a> left me with more appreciation for that song than I normally have for that genre. </p> <p> This is why, in a technical argument, when people say, <em>"I don't like it"</em>, I ask <em>why?</em> </p> <p> Your <em>preference</em> doesn't convince me to change my mind, but if you can explain your perspective, it may make me consider a point of which I wasn't aware. </p> <h3 id="8759a0fb5e2f496ebc7d264072906648"> It Depends™ <a href="#8759a0fb5e2f496ebc7d264072906648" title="permalink">#</a> </h3> <p> Among other roles, I consult. Consultants are infamous for saying <em>it depends</em>, and obviously, if that's all we say, it'd be useless. </p> <p> Most advice is given in a context. When a consultant starts a sentence with <em>it depends</em>, he or she must continue by explaining on what it depends. Done right, this can be tremendously valuable, because it gives the customer a framework they can use to make decisions. </p> <p> In the same family as <em>it depends</em>, I often run into arguments like <em>nothing is black or white</em>, <em>it's a trade-off</em>, or other platitudes. Okay, it probably <em>is</em> a trade-off, but between <em>what?</em> </p> <p> If it's a trade-off between my reasoned argument on the one hand, and someone's preference on the other hand, my reasoned argument wins. </p> <p> Why? Because I can always trump with <em>my</em> preference as well. </p> <h3 id="6dd1b6841a5d42c8b295432859c5187d"> Conclusion <a href="#6dd1b6841a5d42c8b295432859c5187d" title="permalink">#</a> </h3> <p> Our subjective experiences are important pieces of our lives. I hope that my detour around my love of art highlighted that I consider subjective experiences vital. </p> <p> When we discuss how to do things, however, a <em>preference</em> isn't an argument. You may prefer to do things in one way, and I another. Such assertions, alone, don't move the discussion anywhere. </p> <p> What we <em>can</em> do is to attempt to uncover the underlying causes for our preferences. There's no guarantee that we'll reach enlightenment, but it's certain that we'll get nowhere if we don't. </p> <p> That's the reason that, when someone states <em>"I prefer"</em>, I often ask <em>why?</em> </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="f826ff3d591f4d6ebb1e7d93c00cea3a"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> <p> Consultants are infamous for saying <em>it depends</em>, and obviously, if that's all we say, it'd be useless. </p> <p> Most advice is given in a context. When a consultant starts a sentence with <em>it depends</em>, he or she must continue by explaining on what it depends. Done right, this can be tremendously valuable, because it gives the customer a framework they can use to make decisions. </p> </blockquote> <p> I have a vague memory of learning this through some smoothly worded saying. As I recall, it went something like this. </p> <blockquote> Anyone can say "It depends", but an expert can say on what it depends. </blockquote> <p> I don't think that is the exact phrase, because I don't think it sounds very smooth. Does anyone know of a succinct and smooth mnemonic phrase that captures this idea? </p> </div> <div class="comment-date">2020-10-12 13:54 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Fortunately, I don't squash my commits https://blog.ploeh.dk/2020/10/05/fortunately-i-dont-squash-my-commits 2020-10-05T06:00:00+00:00 Mark Seemann <div id="post"> <p> <em>The story of a bug, and how I addressed it.</em> </p> <p> Okay, I admit it: I could have given this article all sorts of alternative titles, each of which would have made as much sense as the one I chose. I didn't want to go with some of the other titles I had in mind, because they would give it all away up front. I didn't want to spoil the surprise. </p> <p> I recently ran into this bug, it took me hours to troubleshoot it, and I was appalled when I realised what the problem was. </p> <p> This is the story of that bug. </p> <p> There are several insights from this story, and I admit that I picked the most click-baity one for the title. </p> <h3 id="94bc204499df483897121bb0343f73f7"> Yak shaving <a href="#94bc204499df483897121bb0343f73f7" title="permalink">#</a> </h3> <p> I was working on the umpteenth variation of an online restaurant reservations system, and one of the features I'd added was a schedule only visible to the <a href="https://en.wikipedia.org/wiki/Ma%C3%AEtre_d%27h%C3%B4tel">maître d'</a>. The schedule includes a list of all reservations for a day, including guests' email addresses and so on. For that reason, I'd protected that resource by requiring a valid <a href="https://en.wikipedia.org/wiki/JSON_Web_Token">JSON Web Token</a> (JWT) with an appropriate role. </p> <p> I'd deployed a new version of the API and went for an ad-hoc test. To my surprise, that particular resource didn't work. When I tried to request it, I got a <code>403 Forbidden</code> response. </p> <p> "That's odd," I though, "it worked the last time I tried this." </p> <p> The system is set up with continuous deployment. I push <em>master</em> to a remote repository, and a build pipeline takes over from there. It only deploys the new version if all tests pass, so my first reaction was that I might have made a mistake with the JWT. </p> <p> I wasted significant time decoding the JWT and comparing its contents to what it was supposed to be. I couldn't find any problems. </p> <p> I also meticulously compared the encryption key I'd used to sign the JWT with the key on the server. They were identical. </p> <p> Incredulous, and running out of ideas, I tried running all tests on my development machine. Indeed, all 170 tests passed. </p> <p> Finally, I gave up and ran the API on my development machine. It takes all of a 30 seconds to configure the code to run in that environment, so you're excused if you wonder why I didn't already do that. What can I say? I've almost two decades of experience with automated test suites. Usually, if all tests pass, the problem is environmental: a network topology issue, a bad or missing connection string, a misconfigured encryption key, an invalid JWT, things like that. </p> <p> To my surprise, the problem also manifested on my machine. </p> <h3 id="b726532ae06844eb8aeaa2c27ca0d913"> Not my code <a href="#b726532ae06844eb8aeaa2c27ca0d913" title="permalink">#</a> </h3> <p> Okay, even with hundreds of tests, perhaps some edge case went unnoticed. The only problem with that hypothesis was that this was hardly an edge case. I was only making a <code>GET</code> request with a <code>Bearer</code> token. I wasn't going through some convoluted sequence of steps. </p> <p> <pre>GET /restaurants/1/schedule/2020/9/30 HTTP/1.1 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5c[...]</pre> </p> <p> I expected a successful response containing some JSON, but the result was <code>403 Forbidden</code>. That was the same behaviour I saw in the production environment. </p> <p> Now, to be clear, this is indeed a protected resource. If you present an invalid JWT, <code>403 Forbidden</code> is the expected response. That's why I wasted a few hours looking for problems with the the JWT. </p> <p> I finally, hesitatingly, concluded that the problem might be somewhere else. The JWT looked okay. So, hours into my troubleshooting I reluctantly set a breakpoint in my code and started the debugger. It isn't rational, but I tend to see it as a small personal defeat if I have to use the debugger. Even so, if used judiciously, it can be an effective way to identify problems. </p> <p> The debugger never hit my breakpoint. </p> <p> To be clear, the beginning of my Controller method looked like this: </p> <p> <pre>[<span style="color:#2b91af;">Authorize</span>(Roles&nbsp;=&nbsp;<span style="color:#a31515;">&quot;MaitreD&quot;</span>)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/schedule/{year}/{month}/{day}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;restaurantId, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;day) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!AccessControlList.Authorize(restaurantId)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ForbidResult</span>();</pre> </p> <p> My breakpoint was on the first line (the <code>if</code> conditional), but the debugger didn't break into it. When I issued my <code>GET</code> request, I immediately got the <code>403 Forbidden</code> response. The breakpoint just sat there in the debugger, mocking me. </p> <p> When that happens, it's natural to conclude that the problem occurs somewhere in the framework; in this case, ASP.NET. To test that hypothesis, I commented out the <code>[Authorize]</code> attribute and reissued the <code>GET</code> request. My hypothesis was that I'd get a <code>200 OK</code> response, since the attribute is what tells ASP.NET to check authorisation. </p> <p> The hypothesis held. The response was <code>200 OK</code>. </p> <h3 id="fdff404f12644a898b6a3e695fb6533f"> Test interdependency <a href="#fdff404f12644a898b6a3e695fb6533f" title="permalink">#</a> </h3> <p> I hate when that happens. It's up there with fixing other people's printers. The problem is in the framework, not in my code. I didn't have any authorisation callbacks registered, so I was fairly certain that the problem wasn't in my code. </p> <p> I rarely jump to the conclusion that there's a bug in the framework. In my experience, <a href="https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault">select is rarely broken</a>. My new hypothesis had to be that I'd somehow managed to misconfigure the framework. </p> <p> But where? There were automated tests that verified that a client could request that resource with a valid JWT. There were other automated tests that verified what happened if you presented an invalid JWT, or none at all. And all tests were passing. </p> <p> While I was fiddling with the tests, I eventually ran a parametrised test by itself, instead of the entire test suite: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>] [<span style="color:#2b91af;">InlineData</span>(&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Hipgnosta&quot;</span>,&nbsp;2024,&nbsp;11,&nbsp;&nbsp;2)] [<span style="color:#2b91af;">InlineData</span>(&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Nono&quot;</span>,&nbsp;2018,&nbsp;&nbsp;9,&nbsp;&nbsp;9)] [<span style="color:#2b91af;">InlineData</span>(<span style="color:#a31515;">&quot;The&nbsp;Vatican&nbsp;Cellar&quot;</span>,&nbsp;2021,&nbsp;10,&nbsp;10)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;GetRestaurantScheduleWhileAuthorized( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">string</span>&nbsp;name, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">int</span>&nbsp;day)</pre> </p> <p> This parametrised test has three test cases. When I ran just that test method, two of the test cases passed, but one failed: the <em>Nono</em> case, for some reason I haven't yet figured out. </p> <p> I didn't understand why that test case ought to fail while the others succeeded, but I had an inkling. I commented out the <em>Nono</em> test case and ran the test method again. </p> <p> One passed and one failing test. </p> <p> Now <em>The Vatican Cellar</em> test case was failing. I commented that out and ran the test method again. The remaining test case failed. </p> <p> This reeks of some sort of test interdependency. Apparently, <em>something</em> happens during the first test run that makes succeeding tests pass, but it happens too late for the first one. But what? </p> <h3 id="c95c3e707f46467abbf8ff02a9594c4c"> Bisect <a href="#c95c3e707f46467abbf8ff02a9594c4c" title="permalink">#</a> </h3> <p> Then something occurred to me that I should have thought of sooner. This feature used to work. Not only had the tests been passing, but I'd actually interacted with the deployed service, presenting a valid JWT and received a <code>200 OK</code> response. </p> <p> Once than dawned on me, I realised that it was just a manner of performing a binary search. Since, of course, I use version control, I had a version I knew worked, and a version that didn't work. The task, then, was to find the commit that introduced the defect. </p> <p> As I've already implied, the system in question is an example code base. While I have a cloud-based production environment, none but I use it. It had been four or five days since I'd actually interacted with the real service, and I'd been busy making changes, trusting exclusively in my test suite. I tend to make frequent, small commits instead of big, infrequent commits, so I had accumulated about a hundred and fifty commits since the 'last known good' deployment. </p> <p> Searching through hundreds of commits sounds overwhelming, but using binary search, it's actually not that bad. Pick the commit halfway between the 'last known good' commit and the most recent commit, and check it out. See if the defect is present there. If it is, you know that it was introduced somewhere between the commit you're looking at, and the 'last known good' commit. If it isn't present, it was introduced later. Regardless of the outcome, you know in which half to look. You now pick a new commit in the middle of that set and repeat the exercise. Even with, say, a hundred commits, the first bisection reduces the candidate set to 50, the next bisection to 25, then 13, then 7, 4, 2, and then you have it. If you do this systematically, you should find the exact commit in less than eight iterations. </p> <p> This is, as far as I understand it, the algorithm used by <em>Git bisect</em>. You don't have to use the <code>bisect</code> command - the algorithm is easy enough to do by hand - but let's see how it works. </p> <p> You start a <code>bisect</code> session with: </p> <p> <pre><span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...))</span> $ git bisect start <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...)|BISECTING)</span></pre> </p> <p> This starts an interactive session, which you can tell from the Git integration in Git Bash (it says <code>BISECTING</code>). You now mark a commit as being bad: </p> <p> <pre>$ git bisect bad <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((93c6c35...)|BISECTING)</span></pre> </p> <p> If you don't provide a commit ID at that point, Git is going to assume that you meant that the current commit (in this case <code>93c6c35</code>) is bad. That's what I had in mind, so that's fine. </p> <p> You now tell it about a commit ID that you know is good: </p> <p> <pre>$ git bisect good 7dfdab2 Bisecting: 75 revisions left to test after this (roughly 6 steps) [1f78c9a90c2088423ab4fc145b7b2ec3859d6a9a] Use InMemoryRestaurantDatabase in a test <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((1f78c9a...)|BISECTING)</span></pre> </p> <p> Notice that Git is already telling us how many iterations we should expect. You can also see that it checked out a new commit (<code>1f78c9a</code>) for you. That's the half-way commit. </p> <p> At this point, I manually ran the test method with the three test cases. All three passed, so I marked that commit as good: </p> <p> <pre>$ git bisect good Bisecting: 37 revisions left to test after this (roughly 5 steps) [5abf65a72628efabbf05fccd1b79340bac4490bc] Delete Either API <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((5abf65a...)|BISECTING)</span></pre> </p> <p> Again, Git estimates how many more steps are left and checks out a new commit (<code>5abf65a</code>). </p> <p> I repeated the process for each step, marking the commit as either good or bad, depending on whether or not the test passed: </p> <p> <pre>$ git bisect bad Bisecting: 18 revisions left to test after this (roughly 4 steps) [fc48292b0d654f4f20522710c14d7726e6eefa70] Delete redundant Test Data Builders <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((fc48292...)|BISECTING)</span> $ git bisect good Bisecting: 9 revisions left to test after this (roughly 3 steps) [b0cb1f5c1e9e40b1dabe035c41bfb4babfbe4585] Extract WillAcceptUpdate helper method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((b0cb1f5...)|BISECTING)</span> $ git bisect good Bisecting: 4 revisions left to test after this (roughly 2 steps) [d160c57288455377f8b0ad05985b029146228445] Extract ConfigureClock helper method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((d160c57...)|BISECTING)</span> $ git bisect good Bisecting: 2 revisions left to test after this (roughly 1 step) [4cb73c219565d8377aa67d79024d6836f9000935] Compact code <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((4cb73c2...)|BISECTING)</span> $ git bisect good Bisecting: 0 revisions left to test after this (roughly 1 step) [34238c7d2606e9007b96b54b43e678589723520c] Extract CreateTokenValidationParameters method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((34238c7...)|BISECTING)</span> $ git bisect bad Bisecting: 0 revisions left to test after this (roughly 0 steps) [7d6583a97ff45fbd85878cecb5af11d93213a25d] Move Configure method up <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((7d6583a...)|BISECTING)</span> $ git bisect good 34238c7d2606e9007b96b54b43e678589723520c is the first bad commit <span style="color: orange">commit 34238c7d2606e9007b96b54b43e678589723520c</span> Author: Mark Seemann &lt;mark@example.com&gt; Date: Wed Sep 16 07:15:12 2020 +0200 Extract CreateTokenValidationParameters method Restaurant.RestApi/Startup.cs | 32 <span style="color: green">+++++++++++++++++++</span><span style="color: red">-------------</span> 1 file changed, 19 insertions(+), 13 deletions(-) <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((7d6583a...)|BISECTING)</span></pre> </p> <p> Notice that the last step finds the culprit. It tells you which commit is the bad one, but surprisingly doesn't check it out for you. You can do that using a label Git has created for you: </p> <p> <pre>$ git checkout refs/bisect/bad Previous HEAD position was 7d6583a Move Configure method up HEAD is now at 34238c7 Extract CreateTokenValidationParameters method <span style="color: green">mark@Vindemiatrix</span> <span style="color: purple">MINGW64</span> <span style="color: orange">~/Documents/Redacted/Restaurant</span> <span style="color: darkturquoise">((34238c7...)|BISECTING)</span></pre> </p> <p> You've now found and checked out the offending commit. Hopefully, the changes in that commit should give you a clue about the problem. </p> <h3 id="2f3c813d0cbf47eabb52467cff0c7692"> The culprit <a href="#2f3c813d0cbf47eabb52467cff0c7692" title="permalink">#</a> </h3> <p> What's in that commit? Take a gander: </p> <p> <pre>$ git show <span style="color: orange">commit 34238c7d2606e9007b96b54b43e678589723520c (</span><span style="color: mediumturquoise">HEAD</span><span style="color: orange">,</span> refs/bisect/bad<span style="color: orange">)</span> Author: Mark Seemann &lt;mark@example.com&gt; Date: Wed Sep 16 07:15:12 2020 +0200 Extract CreateTokenValidationParameters method <strong>diff --git a/Restaurant.RestApi/Startup.cs b/Restaurant.RestApi/Startup.cs index 6c161b5..bcde861 100644 --- a/Restaurant.RestApi/Startup.cs +++ b/Restaurant.RestApi/Startup.cs</strong> <span style="color: darkturquoise">@@ -79,10 +79,6 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private void ConfigureAuthorization(IServiceCollection services) { <span style="color: red">- JwtSecurityTokenHandler.DefaultMapInboundClaims = false; - - var secret = Configuration["JwtIssuerSigningKey"]; -</span> services.AddAuthentication(opts =&gt; { opts.DefaultAuthenticateScheme = <span style="color: darkturquoise">@@ -91,15 +87,8 @@</span> namespace Ploeh.Samples.Restaurants.RestApi JwtBearerDefaults.AuthenticationScheme; }).AddJwtBearer(opts =&gt; { <span style="color: red">- opts.TokenValidationParameters = new TokenValidationParameters - { - ValidateIssuerSigningKey = true, - IssuerSigningKey = new SymmetricSecurityKey( - Encoding.ASCII.GetBytes(secret)), - ValidateIssuer = false, - ValidateAudience = false, - RoleClaimType = "role" - };</span> <span style="color: green">+ opts.TokenValidationParameters = + CreateTokenValidationParameters();</span> opts.RequireHttpsMetadata = false; }); <span style="color: darkturquoise">@@ -108,6 +97,23 @@</span> namespace Ploeh.Samples.Restaurants.RestApi sp.GetService&lt;IHttpContextAccessor&gt;().HttpContext.User)); } <span style="color: green">+ private TokenValidationParameters CreateTokenValidationParameters() + { + JwtSecurityTokenHandler.DefaultMapInboundClaims = false; + + var secret = Configuration["JwtIssuerSigningKey"]; + + return new TokenValidationParameters + { + ValidateIssuerSigningKey = true, + IssuerSigningKey = new SymmetricSecurityKey( + Encoding.ASCII.GetBytes(secret)), + ValidateIssuer = false, + ValidateAudience = false, + RoleClaimType = "role" + }; + } +</span> private void ConfigureRepository(IServiceCollection services) {</pre> </p> <p> Can you spot the problem? </p> <p> As soon as I saw that diff, the problem almost jumped out at me. I'm so happy that I make small commits. What you see here is, I promise, the entire diff of that commit. </p> <p> Clearly, you're not familiar with this code base, but even so, you might be able to intuit the problem from the above diff. You don't need domain-specific knowledge for it, or knowledge of the rest of the code base. </p> <p> I'll give you a hint: I moved a line of code into a lambda expression. </p> <h3 id="1cfcd1adb0b24b6098aa250c73f5cba2"> Deferred execution <a href="#1cfcd1adb0b24b6098aa250c73f5cba2" title="permalink">#</a> </h3> <p> In the above commit, I extracted a helper method that I called <code>CreateTokenValidationParameters</code>. As the name communicates, it creates JWT validation parameters. In other words, it supplies the configuration values that determine how ASP.NET figures out how to authorise an HTTP request. </p> <p> Among other things, it sets the <code>RoleClaimType</code> property to <code>"role"</code>. By default, the <code>[Authorize]</code> attribute is looking for a role claim with a rather complicated identifier. Setting the <code>RoleClaimType</code> enables you to change the identifier. By setting it to <code>"role"</code>, I'm telling the ASP.NET framework that it should look for claims named <code>role</code> in the JWT, and these should be translated to role claims. </p> <p> Except that it doesn't work. </p> <p> Ostensibly, this is for backwards compatibility reasons. You have to set <a href="https://docs.microsoft.com/dotnet/api/system.identitymodel.tokens.jwt.jwtsecuritytokenhandler.defaultmapinboundclaims">JwtSecurityTokenHandler.DefaultMapInboundClaims</a> to <code>false</code>. Notice that this is a global variable. </p> <p> I thought that the line of code that does that conceptually belongs together with the other code that configures the JWT validation parameters, so I moved it in there. I didn't think much about it, and all my tests were still passing. </p> <p> What happened, though, was that I moved mutation of a global variable into a helper method that was called from a lambda expression. Keep in mind that lambda expressions represent code that may run later; that execution is deferred. </p> <p> By moving that mutation statement into the helper method, I inadvertently deferred its execution. When the application's <code>Startup</code> code runs, it configures the service. All the code that runs inside of <code>AddJwtBearer</code>, however, doesn't run immediately; it runs when needed. </p> <p> This explains why all tests were passing. My test suite has plenty of self-hosted integration tests in the style of <a href="/outside-in-tdd">outside-in test-driven development</a>. When I ran all the tests, the deferred code block would run in some other test context, flipping that global bit as a side effect. When the test suite reaches the test that fails when run in isolation, the bit is already flipped, and then it works. </p> <p> It took me hours to figure out what the problem was, and it turned out that the root cause was a global variable. </p> <p> Global variables are evil. Who knew? </p> <h3 id="cbf70c019a264e8fa861e13536db688f"> Can't reproduce <a href="#cbf70c019a264e8fa861e13536db688f" title="permalink">#</a> </h3> <p> Once you figure out what the problem is, you should reproduce it with an automated test. </p> <p> Yes, and how do I do that here? I already had tests that verify that you can <code>GET</code> the desired resource if you present a valid JWT. And <em>that test passes!</em> </p> <p> The only way I can think of to reproduce the issue with an automated test is to create a completely new, independent test library with only one test. I considered doing that, weighed the advantages against the disadvantages and decided, given the context, that it wasn't worth the effort. </p> <p> That means, though, that I had to accept that within the confines of my existing testing strategy, I can't reproduce the defect. This doesn't happen to me often. In fact, I can't recall that it's ever happened to me before, like this. </p> <h3 id="2ae70aa4524e48bf9b0190aeaa158192"> Resolution <a href="#2ae70aa4524e48bf9b0190aeaa158192" title="permalink">#</a> </h3> <p> It took me hours to find the bug, and ten seconds to fix it. I just moved the mutation of <code>JwtSecurityTokenHandler.DefaultMapInboundClaims</code> back to the <code>ConfigureAuthorization</code> method: </p> <p> <pre><strong>diff --git a/Restaurant.RestApi/Startup.cs b/Restaurant.RestApi/Startup.cs index d4917f5..e40032f 100644 --- a/Restaurant.RestApi/Startup.cs +++ b/Restaurant.RestApi/Startup.cs</strong> <span style="color: darkturquoise">@@ -79,6 +79,7 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private void ConfigureAuthorization(IServiceCollection services) { <span style="color: green">+ JwtSecurityTokenHandler.DefaultMapInboundClaims = false;</span> services.AddAuthentication(opts =&gt; { opts.DefaultAuthenticateScheme = <span style="color: darkturquoise">@@ -99,8 +100,6 @@</span> namespace Ploeh.Samples.Restaurants.RestApi private TokenValidationParameters CreateTokenValidationParameters() { <span style="color: red">- JwtSecurityTokenHandler.DefaultMapInboundClaims = false; -</span> var secret = Configuration["JwtIssuerSigningKey"]; return new TokenValidationParameters</pre> </p> <p> An ad-hoc smoke test verified that this solved the problem. </p> <h3 id="55a0017f1aab47b6b9049e63d51d79ff"> Conclusion <a href="#55a0017f1aab47b6b9049e63d51d79ff" title="permalink">#</a> </h3> <p> There are multiple insights to be had from this experience: <ul> <li>Global variables are evil</li> <li>Binary search, or bisection, is useful when troubleshooting</li> <li>Not all bugs can be reproduced by an automated test</li> <li>Small commits make it easy to detect problems</li> </ul> I've always disliked Git's <em>squash</em> feature, and here's one of the many reasons to dislike it. Had this happened in a code base with a 'nice history' (as the <em>squash</em> proponents like to present it), that small commit would have been bundled with various other commits. The problem wouldn't have jumped at me if buried in dozens of other changes. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. EnsureSuccessStatusCode as an assertion https://blog.ploeh.dk/2020/09/28/ensuresuccessstatuscode-as-an-assertion 2020-09-28T05:50:00+00:00 Mark Seemann <div id="post"> <p> <em>It finally dawned on me that EnsureSuccessStatusCode is a fine test assertion.</em> </p> <p> Okay, I admit that sometimes I can be too literal and rigid in my thinking. At least as far back as 2012 I've been doing <a href="/outside-in-tdd">outside-in TDD</a> against self-hosted HTTP APIs. When you do that, you'll regularly need to verify that an HTTP response is successful. </p> <h3 id="77cb250db1ab4278bad68cf7736ae62e"> Assert equality <a href="#77cb250db1ab4278bad68cf7736ae62e" title="permalink">#</a> </h3> <p> You can, of course, write an assertion like this: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.Equal(<span style="color:#2b91af;">HttpStatusCode</span>.OK,&nbsp;response.StatusCode);</pre> </p> <p> If the assertion fails, it'll produce a comprehensible message like this: </p> <p> <pre>Assert.Equal() Failure Expected: OK Actual: BadRequest</pre> </p> <p> What's not to like? </p> <h3 id="2ba5bb17d1f8423184a50b2fa1fcb667"> Assert success <a href="#2ba5bb17d1f8423184a50b2fa1fcb667" title="permalink">#</a> </h3> <p> The problem with testing strict equality in this context is that it's often too narrow a definition of success. </p> <p> You may start by returning <code>200 OK</code> as response to a <code>POST</code> request, but then later evolve the API to return <code>201 Created</code> or <code>202 Accepted</code>. Those status codes still indicate success, and are typically accompanied by more information (e.g. a <code>Location</code> header). Thus, evolving a REST API from returning <code>200 OK</code> to <code>201 Created</code> or <code>202 Accepted</code> isn't a breaking change. </p> <p> It does, however, break the above assertion, which may now fail like this: </p> <p> <pre>Assert.Equal() Failure Expected: OK Actual: Created</pre> </p> <p> You could attempt to fix this issue by instead verifying <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage.issuccessstatuscode">IsSuccessStatusCode</a>: </p> <p> <pre><span style="color:#2b91af;">Assert</span>.True(response.IsSuccessStatusCode);</pre> </p> <p> That permits the API to evolve, but introduces another problem. </p> <h3 id="0389a3d633c64aec8068be619ddce54c"> Legibility <a href="#0389a3d633c64aec8068be619ddce54c" title="permalink">#</a> </h3> <p> What happens when an assertion against <code>IsSuccessStatusCode</code> fails? </p> <p> <pre>Assert.True() Failure Expected: True Actual: False</pre> </p> <p> That's not helpful. At least, you'd like to know which other status code was returned instead. You can address that concern by using an overload to <code>Assert.True</code> (most unit testing frameworks come with such an overload): </p> <p> <pre><span style="color:#2b91af;">Assert</span>.True( &nbsp;&nbsp;&nbsp;&nbsp;response.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{response.StatusCode}<span style="color:#a31515;">.&quot;</span>);</pre> </p> <p> This produces more helpful error messages: </p> <p> <pre>Actual status code: BadRequest. Expected: True Actual: False</pre> </p> <p> Granted, the <em>Expected: True, Actual: False</em> lines are redundant, but at least the information you care about is there; in this example, the status code was <code>400 Bad Request</code>. </p> <h3 id="9697ab68967948f89792b82d00028ccf"> Conciseness <a href="#9697ab68967948f89792b82d00028ccf" title="permalink">#</a> </h3> <p> That's not bad. You can use that <code>Assert.True</code> overload everywhere you want to assert success. You can even copy and paste the code, <a href="/2014/08/07/why-dry">because it's an idiom that doesn't change</a>. Still, since <a href="/2019/11/04/the-80-24-rule">I like to stay within 80 characters line width</a>, this little phrase takes up three lines of code. </p> <p> Surely, a helper method can address that problem: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AssertSuccess(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">HttpResponseMessage</span>&nbsp;response) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.True( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;response.IsSuccessStatusCode, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">$&quot;Actual&nbsp;status&nbsp;code:&nbsp;</span>{response.StatusCode}<span style="color:#a31515;">.&quot;</span>); }</pre> </p> <p> I can now write a one-liner as an assertion: </p> <p> <pre>response.AssertSuccess();</pre> </p> <p> What's not to like? </p> <h3 id="6990f2fbc33f4beca15c122b5b4ccc22"> Carrying coals to Newcastle <a href="#6990f2fbc33f4beca15c122b5b4ccc22" title="permalink">#</a> </h3> <p> If you're already familiar with the <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage">HttpResponseMessage</a> class, you may have been reading this far with some frustration. It already comes with a method called <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpresponsemessage.ensuresuccessstatuscode">EnsureSuccessStatusCode</a>. Why not just use that instead? </p> <p> <pre>response.EnsureSuccessStatusCode();</pre> </p> <p> As long as the <code>response</code> status code is in the 200 range, this method call succeeds, but if not, it throws an exception: </p> <p> <pre>Response status code does not indicate success: 400 (Bad Request).</pre> </p> <p> That's exactly the information you need. </p> <p> I admit that I can sometimes be too rigid in my thinking. I've known about this method for years, but I never thought of it as a <em>unit testing assertion</em>. It's as if there's a category in my programming brain that's labelled <em>unit testing assertions</em>, and it only contains methods that originate from unit testing. </p> <p> I even knew about <a href="https://docs.microsoft.com/dotnet/api/system.diagnostics.debug.assert">Debug.Assert</a>, so I knew that assertions also exist outside of unit testing. That assertion isn't suitable for unit testing, however, so I never included it in my category of unit testing assertions. </p> <p> The <code>EnsureSuccessStatusCode</code> method doesn't have <em>assert</em> in its name, but it behaves just like a unit testing assertion: it throws an exception with a useful error message if a criterion is not fulfilled. I just never thought of it as useful for unit testing purposes because it's part of 'production code'. </p> <h3 id="5ff49213dd304440887719d6879ff7fb"> Conclusion <a href="#5ff49213dd304440887719d6879ff7fb" title="permalink">#</a> </h3> <p> The built-in <code>EnsureSuccessStatusCode</code> method is fine for unit testing purposes. Who knows, perhaps other class libraries contain similar assertion methods, although they may be hiding behind names that don't include <em>assert</em>. Perhaps your own production code contains such methods. If so, they can double as unit testing assertions. </p> <p> I've been doing test-driven development for close to two decades now, and one of many consistent experiences is this: <ul> <li>In the beginning of a project, the unit tests I write are verbose. This is natural, because I'm still getting to know the domain.</li> <li>After some time, I begin to notice patterns in my tests, so I refactor them. I introduce helper APIs in the test code.</li> <li>Finally, I realise that some of those unit testing APIs might actually be useful in the production code too, so I move them over there.</li> </ul> Test-driven development is process that gives you feedback about your production code's APIs. Listen to your tests. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="20484094920349b7a91eebcb9b15c628"> <div class="comment-author"><a href="https://github.com/YevgeniyShunevych">Yevgeniy Shunevych</a></div> <div class="comment-content"> <p> Unfortunately, I can't agree that it's fine to use <code>EnsureSuccessStatusCode</code> or other similar methods for assertions. An exception of a kind different from assertion (<code>XunitException</code> for XUnit, <code>AssertionException</code> for NUnit) is handled in a bit different way by the testing framework. <code>EnsureSuccessStatusCode</code> method throws <code>HttpRequestException</code>. Basically, a test that fails with such exception is considered as "Failed by error". </p> <p> For example, the test result message in case of 404 status produced by the <code>EnsureSuccessStatusCode</code> method is: </p> <p> <pre>System.Net.Http.HttpRequestException : Response status code does not indicate success: 404 (Not Found).</pre> </p> <p> Where "System.Net.Http.HttpRequestException :" text looks redundant, as we do assertion. This also may make other people think that the reason for this failure is not an assertion, but an unexpected exception. In case of regular assertion exception type, that text is missing. </p> <p> Other points are related to NUnit: <ul> <li> There are 2 different failure test outcome kinds in NUnit: <ul> <li>Failure: a test assertion failed. (Status=Failed, Label=empty)</li> <li>Error: an unexpected exception occurred. (Status=Failed, Label=Error)</li> </ul> See NUnit <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html#common-outcomes">Common Outcomes</a>. Thus `EnsureSuccessStatusCode` produces "Error" status instead of desirable "Failure". </li> <li>You can't use methods like <code>EnsureSuccessStatusCode</code> as assertion inside multiple asserts.</li> <li>NUnit tracks the count of assertions for each test. <code>"assertions"</code> property gets into the test results XML file and might be useful. This property increments on assertion methods, <code>EnsureSuccessStatusCode</code> - obviously doesn't increment it.</li> </ul> </p> </div> <div class="comment-date">2020-09-30 15:25 UTC</div> </div> <div class="comment" id="16adadb969f1406c9c18605c8c85ef83"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Yevgeniy, thank you for writing. This shows, I think, that all advice is contextual. It's been more than a decade since I regularly used NUnit, but I vaguely recall that it gives special treatment to its own assertion exceptions. </p> <p> I also grant that the "System.Net.Http.HttpRequestException" part of the message is redundant. To make matters worse, even without that 'prologue' the information that we care about (the actual status code) is all the way to the right. It'll often require horizontal scrolling to see it. </p> <p> That doesn't much bother me, though. I think that one should optimise for the most frequent scenario. For example, I favour taking a bit of extra time to write readable code, because <a href="/2018/09/17/typing-is-not-a-programming-bottleneck">the bottleneck in programming isn't typing</a>. In the same vein, I find it a good trade-off being able to avoid adding more code against an occasional need to scroll horizontally. The cost of code isn't the time it takes to write it, but the maintenance burden it imposes, as well as the cognitive load it adds. </p> <p> The point here is that while horizontal scrolling is inconvenient, the <em>most frequent scenario</em> is that tests don't fail. The inconvenience only manifests in the rare circumstance that a test fails. </p> <p> I want to thank you, however, for pointing out the issues with NUnit. It reinforces the impression I already had of it, as a framework to stay away from. That design seems to me to impose a lock-in that prevents you from mixing NUnit with improved APIs. </p> <p> In contrast, the <a href="https://www.nuget.org/packages/xunit.core">xunit.core</a> library doesn't include <a href="https://www.nuget.org/packages/xunit.assert">xunit.assert</a>. I often take advantage of that when I write unit tests in F#. While the xUnit.net assertion library is adequate, <a href="https://github.com/SwensenSoftware/unquote">Unquote</a> is much better for F#. I appreciate that xUnit.net enables me to combine the tools that work best in a given situation. </p> </div> <div class="comment-date">2020-10-02 07:35 UTC</div> </div> <div class="comment" id="3dbfa056360b4df6a052be2776a6f930"> <div class="comment-author"><a href="https://github.com/YevgeniyShunevych">Yevgeniy Shunevych</a></div> <div class="comment-content"> <p> I see your point, Mark. Thanks for your reply. </p> <p> I would like to add a few words in defense of the NUnit. </p> <p> First of all, there is no problem to use an extra library for NUnit assertions too. For example, I like <a href="https://fluentassertions.com/">FluentAssertions</a> that works great for both XUnit and NUnit. </p> <p> NUnit 3 also has a set of useful features that are missing in XUnit like: <ul> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html">Multiple asserts</a> - quite useful for UI testing, for example. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/Warnings.html">Warnings</a>. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html">TestContext</a>. </li> <li> <a href="https://docs.nunit.org/articles/nunit/writing-tests/TestContext.html#addtestattachment-37">File attachments</a> - useful for UI tests or tests that verify files. For example, when the UI test fails, you can take a screenshot, save it and add to NUnit as an attachment. Or when you test PDF file, you can attach the file that doesn't pass the test. </li> </ul> </p> <p> I can recommend to reconsider the view of NUnit, as the library progressed well during its 3rd version. XUnit and NUnit seem interchangeable for unit testing. But NUnit, from my point of view, provides the features that are helpful in integration and UI testing. </p> </div> <div class="comment-date">2020-10-05 13:24 UTC</div> </div> <div class="comment" id="fded96f5f68a4d8ba1e782678dd854be"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Yevgeniy, I haven't done UI testing since sometime in 2005, I think, so I wasn't aware of that. It's good to know that NUnit may excel at that, and the other scenarios you point out. </p> <p> To be clear, while I still think I may prefer xUnit.net, I don't consider it perfect. </p> </div> <div class="comment-date">2020-10-05 14:10 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An XPath query for long methods https://blog.ploeh.dk/2020/09/21/an-xpath-query-for-long-methods 2020-09-21T05:28:00+00:00 Mark Seemann <div id="post"> <p> <em>A filter for Visual Studio code metrics.</em> </p> <p> I consider it a good idea to <a href="/2020/04/13/curb-code-rot-with-thresholds">limit</a> the size of code blocks. Small methods are <a href="/2019/11/18/small-methods-are-easy-to-troubleshoot">easier to troubleshoot</a> and in general fit better in your head. When it comes to vertical size, however, the editors I use don't come with visual indicators like <a href="https://marketplace.visualstudio.com/items?itemName=PaulHarrington.EditorGuidelines">they do for horizontal size</a>. </p> <p> When you're in the midst of developing a feature, you don't want to be <em>prevented</em> from doing so by a tool that refuses to compile your code if methods get too long. On the other hand, it might be a good idea to regularly run a tool over your code base to identify which methods are getting too long. </p> <p> With Visual Studio you can calculate code metrics, which include a measure of the number of lines of code for each method. One option produces an XML file, but if you have a large code base, those XML files are big. </p> <p> The XML files typically look like this: </p> <p> <pre><span style="color:blue;">&lt;?</span><span style="color:#a31515;">xml</span><span style="color:blue;">&nbsp;</span><span style="color:red;">version</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1.0</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">encoding</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">utf-8</span>&quot;<span style="color:blue;">?&gt;</span> <span style="color:blue;">&lt;</span><span style="color:#a31515;">CodeMetricsReport</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Version</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1.0</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Targets</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Target</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Restaurant.RestApi.csproj</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Assembly</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Ploeh.Samples.Restaurants.RestApi,&nbsp;...</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">87</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">537</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">208</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">3188</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">711</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Namespaces</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Namespace</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Ploeh.Samples.Restaurants.RestApi</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">87</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">499</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">204</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">3100</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">701</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Types</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">NamedType</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CalendarController</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">73</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">14</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">34</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">DepthOfInheritance</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">1</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">190</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">42</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Members</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Method</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">Task</span><span style="color:red;">&amp;lt;</span><span style="color:blue;">ActionResult</span><span style="color:red;">&amp;gt;</span><span style="color:blue;">&nbsp;CalendarController.Get(...</span>&quot;<span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">MaintainabilityIndex</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">64</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">CyclomaticComplexity</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">2</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ClassCoupling</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">12</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">SourceLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">28</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;</span><span style="color:#a31515;">Metric</span><span style="color:blue;">&nbsp;</span><span style="color:red;">Name</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">ExecutableLines</span>&quot;<span style="color:blue;">&nbsp;</span><span style="color:red;">Value</span><span style="color:blue;">=</span>&quot;<span style="color:blue;">7</span>&quot;<span style="color:blue;">&nbsp;/&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Metrics</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Method</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;!--</span><span style="color:green;">Much&nbsp;more&nbsp;data&nbsp;goes&nbsp;here</span><span style="color:blue;">--&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Members</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">NamedType</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Types</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Namespace</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Namespaces</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Assembly</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Target</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&nbsp;&nbsp;&lt;/</span><span style="color:#a31515;">Targets</span><span style="color:blue;">&gt;</span> <span style="color:blue;">&lt;/</span><span style="color:#a31515;">CodeMetricsReport</span><span style="color:blue;">&gt;</span></pre> </p> <p> How can you filter such a file to find only those methods that are too long? </p> <p> That sounds like a job for <a href="https://en.wikipedia.org/wiki/XPath">XPath</a>. I admit, though, that I use XPath only rarely. While the general idea of the syntax is easy to grasp, it has enough subtle pitfalls that it's not that easy to use, either. </p> <p> Partly for my own benefit, and partly for anyone else who might need it, here's an XPath query that looks for long methods: </p> <p> <pre>//Members/child::*[Metrics/Metric[@Value &gt; 24 and @Name = "SourceLines"]]</pre> </p> <p> This query looks for <a href="/2019/11/04/the-80-24-rule">methods longer that 24 lines of code</a>. If you don't agree with that <a href="/2020/04/13/curb-code-rot-with-thresholds">threshold</a> you can always change it to another value. You can also change <code>@Name</code> to look for <code>CyclomaticComplexity</code> or one of the other metrics. </p> <p> Given the above XML metrics report, the XPath filter would select (among other members) the <code>CalendarController.Get</code> method, because it has 28 lines of source code. It turns out, though, that the filter produces some false positives. The method in question is actually fine: </p> <p> <pre><span style="color:green;">/*&nbsp;This&nbsp;method&nbsp;loads&nbsp;a&nbsp;year&#39;s&nbsp;worth&nbsp;of&nbsp;reservations&nbsp;in&nbsp;order&nbsp;to&nbsp;segment &nbsp;*&nbsp;them&nbsp;all.&nbsp;In&nbsp;a&nbsp;realistic&nbsp;system,&nbsp;this&nbsp;could&nbsp;be&nbsp;quite&nbsp;stressful&nbsp;for &nbsp;*&nbsp;both&nbsp;the&nbsp;database&nbsp;and&nbsp;the&nbsp;web&nbsp;server.&nbsp;Some&nbsp;of&nbsp;that&nbsp;concern&nbsp;can&nbsp;be &nbsp;*&nbsp;addressed&nbsp;with&nbsp;an&nbsp;appropriate&nbsp;HTTP&nbsp;cache&nbsp;header&nbsp;and&nbsp;a&nbsp;reverse&nbsp;proxy, &nbsp;*&nbsp;but&nbsp;a&nbsp;better&nbsp;solution&nbsp;would&nbsp;be&nbsp;a&nbsp;CQRS-style&nbsp;architecture&nbsp;where&nbsp;the &nbsp;*&nbsp;calendars&nbsp;get&nbsp;re-rendered&nbsp;as&nbsp;materialised&nbsp;views&nbsp;in&nbsp;a&nbsp;background &nbsp;*&nbsp;process.&nbsp;That&#39;s&nbsp;beyond&nbsp;the&nbsp;scope&nbsp;of&nbsp;this&nbsp;example&nbsp;code&nbsp;base,&nbsp;though. &nbsp;*/</span> [<span style="color:#2b91af;">ResponseCache</span>(Duration&nbsp;=&nbsp;60)] [<span style="color:#2b91af;">HttpGet</span>(<span style="color:#a31515;">&quot;restaurants/{restaurantId}/calendar/{year}&quot;</span>)] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;restaurantId,&nbsp;<span style="color:blue;">int</span>&nbsp;year) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;restaurant&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;RestaurantDatabase &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.GetRestaurant(restaurantId).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(restaurant&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NotFoundResult</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;<span style="color:#2b91af;">Period</span>.Year(year); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;MakeDays(restaurant,&nbsp;period) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Name&nbsp;=&nbsp;restaurant.Name, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Year&nbsp;=&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Days&nbsp;=&nbsp;days &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> That method only has 18 lines of actual source code, from the beginning of the method declaration to the closing bracket. Visual Studio's metrics calculator, however, also counts the attributes and the comments. </p> <p> In general, I only add comments when I want to communicate something that I can't express as a type or a method name, so in this particular code base, it's not much of an issue. If you consistently adorn every method with doc comments, on the other hand, you may need to perform some pre-processing on the source code before you calculate the metrics. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. We need young programmers; we need old programmers https://blog.ploeh.dk/2020/09/14/we-need-young-programmers-we-need-old-programmers 2020-09-14T05:46:00+00:00 Mark Seemann <div id="post"> <p> <em>The software industry loves young people, but old-timers serve an important purpose, too.</em> </p> <p> Our culture idolises youth. There's several reasons for this, I believe. Youth seems synonymous with vigour, strength, beauty, and many other desirable qualities. The cynical perspective is that young people, while rebellious, also tend to be easy to manipulate, if you know which buttons to push. A middle-aged man like me isn't susceptible to the argument that I should buy a particular pair of Nike shoes because they're named after Michael Jordan, but for a while, one pair wasn't enough for my teenage daughter. </p> <p> In intellectual pursuits (like software development), youth is often extolled as the source of innovation. You're often confronted with examples like that of <a href="https://en.wikipedia.org/wiki/%C3%89variste_Galois">Évariste Galois</a>, who made all his discoveries before turning 21. <a href="https://en.wikipedia.org/wiki/Ada_Lovelace">Ada Lovelace</a> was around 28 years when she produced what is considered the 'first computer program'. <a href="https://en.wikipedia.org/wiki/Alan_Turing">Alan Turing</a> was 24 when he wrote <a href="https://en.wikipedia.org/wiki/Turing%27s_proof">On Computable Numbers, with an Application to the Entscheidungsproblem</a>. </p> <p> Clearly, young age is no detriment to making ground-breaking contributions. It has even become folklore that everyone past the age of 35 is a has-been whose only chance at academic influence is to write a textbook. </p> <h3 id="800321a74c054ea0b75815c86f4ce18d"> The story of the five monkeys <a href="#800321a74c054ea0b75815c86f4ce18d" title="permalink">#</a> </h3> <p> You may have seen a story called <em>the five monkeys experiment</em>. It's most likely a fabrication, but it goes like this: </p> <p> A group of scientists placed five monkeys in a cage, and in the middle, a ladder with bananas on the top. Every time a monkey went up the ladder, the scientists soaked the rest of the monkeys with cold water. After a while, every time a monkey went up the ladder, the others would beat it up. </p> <p> After some time, none of the monkeys dared go up the ladder regardless of the temptation. The scientists then substituted one of the monkeys with a new one, who'd immediately go for the bananas, only to be beaten up by the others. After several beatings, the new member learned not to climb the ladder even though it never knew why. </p> <p> A second monkey was substituted and the same occurred. The first monkey participated in beating the second. A third monkey was exchanged and the story repeated. The fourth was substituted and the beating was repeated. Finally the fifth monkey was replaced. </p> <p> Left was a group of five monkeys who, even though they never received a cold shower, continued to beat up any monkey who attempted to climb the ladder. If it was possible to ask the monkeys why they would beat up all who attempted to go up the ladder, the answer would probably be: </p> <p> "That's how we do things here." </p> <p> While the story is probably just that: a story, it tells us something about the drag induced by age and experience. If you've been in the business for decades, you've seen numerous failed attempts at something you yourself tried when you were young. You know that it can't be done. </p> <p> Young people don't know that a thing can't be done. If they can avoid the monkey-beating, they'll attempt the impossible. </p> <h3 id="4add8a9af0424d7e889d3125837ed611"> Changing circumstances <a href="#4add8a9af0424d7e889d3125837ed611" title="permalink">#</a> </h3> <p> Is attempting the impossible a good idea? </p> <p> In general, no, because it's... impossible. There's a reason older people tell young people that a thing can't be done. It's not just because they're stodgy conservatives who abhor change. It's because they see the effort as wasteful. Perhaps they're even trying to be kind, guiding young people off a path where only toil and disappointment is to be found. </p> <p> What old people don't realise is that sometimes, circumstances change. </p> <p> What was impossible twenty years ago may not be impossible today. We see this happening in many fields. Producing a commercially viable electric car was impossible for decades, until, with the advances made in battery technology, it became possible. </p> <p> Technology changes rapidly in software development. People trying something previously impossible may find that it's possible today. Once, if you had lots of data, you had to store it in fully normalised form, because storage was expensive. For a decade, relational databases were the only game in town. Then circumstances changed. Storage became cheaper, and a new movement of NoSQL storage emerged. What was before impossible became possible. </p> <p> Older people often don't see the new opportunities, because they 'know' that some things are impossible. Young people push the envelope driven by a combination of zest and ignorance. Most fail, but a few succeed. </p> <h3 id="4272a069588e47f796646bd282b9de02"> Lottery of the impossible <a href="#4272a069588e47f796646bd282b9de02" title="permalink">#</a> </h3> <p> I think of this process as a lottery. Imagine that every impossible thing is a red ball in an urn. Every young person who tries the impossible draws a random ball from the urn. </p> <p> The urn contains millions of red balls, but every now and then, one of them turns green. You don't know which one, but if you draw it, it represents something that was previously impossible which has now become possible. </p> <p> This process produces growth, because once discovered, the new and better way of doing things can improve society in general. Occasionally, the young discoverer may even gain some fame and fortune. </p> <p> It seems wasteful, though. Most people who attempt the impossible will reach the predictable conclusion. What was deemed impossible was, indeed, impossible. </p> <p> When I'm in a cynical mood, I don't think that it's youth in itself that is the source of progress. It's just the <a href="https://en.wikipedia.org/wiki/Law_of_large_numbers">law of large numbers</a> applied. If there's a one in million chance that something will succeed, but ten million people attempt it, it's only a matter of time before one succeeds. </p> <p> Society at large can benefit from the success of the few, but ten million people still wasted their efforts. </p> <h3 id="016744f0ea77495c958a7914f08187db"> We need the old, too <a href="#016744f0ea77495c958a7914f08187db" title="permalink">#</a> </h3> <p> If you accept the argument that young people are more likely to try the impossible, we need the young people. Do we need the old people? </p> <p> I'm turning fifty in 2020. You may consider that old, but I expect to work for many more years. I don't know if the software industry needs fifty-year-olds, but that's not the kind of old I have in mind. I'm thinking of people who have retired, or are close to retirement. </p> <p> In our youth-glorifying culture, we tend to dismiss the opinion and experiences of old people. <em>Oh, well, it's just a codgy old man</em> (or woman), we'll say. </p> <p> We ignore the experience of the old, because we believe that they haven't been keeping up with times. Their experiences don't apply to us, because we live under new circumstance. Well, see above. </p> <p> I'm not advocating that we turn into a gerontocracy that venerates our elders solely because of their age. Again, according to the law of large numbers, some people live to old age. There need not be any correlation between survivors and wisdom. </p> <p> We need the old to tell us the truth, because they have little to lose. </p> <h3 id="8b5c613ba6c44bb4b4e6dbba7ae7d19a"> Nothing to lose <a href="#8b5c613ba6c44bb4b4e6dbba7ae7d19a" title="permalink">#</a> </h3> <p> In the last couple of years, I've noticed a trend. A book comes out, exposing the sad state of affairs in some organisation. This has happened regularly in Denmark, where I live. One book may expose the deplorable conditions of the Danish tax authorities, one may describe the situation in the ministry of defence, one criticises the groupthink associated with the climate crisis, and so on. </p> <p> Invariably, it turns out that the book is written by a professor emeritus or a retired department head. </p> <p> I don't think that these people, all of a sudden, had an epiphany after they retired. They knew all about the rot in the system they were part of, while they were part of it, but they've had too much to lose. You could argue that they should have said something before they retired, but that requires a moral backbone we can't expect most people to have. </p> <p> When people retire, the threat of getting fired disappears. Old people can speak freely to a degree most other people can't. </p> <p> Granted, many may simply use that freedom to spew bile or shout <em>Get off my lawn!</em>, but many are in the unique position to reveal truths no-one else dare speak. Many are, perhaps, just bitter, but some may possess knowledge that they are in a unique position to reveal. </p> <p> When that grumpy old guy on Twitter writes something that makes you uncomfortable, consider this: he may still be right. </p> <h3 id="2d64bd2c7ccb4b7ca2418802ed82689e"> Being unreasonable <a href="#2d64bd2c7ccb4b7ca2418802ed82689e" title="permalink">#</a> </h3> <p> In a way, you could say that we need young and old people for the same fundamental reason. Not all of them, but enough of them, are in a position to be unreasonable. <blockquote> <p> "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." </p> <footer><cite>George Bernard Shaw</cite></footer> </blockquote> Young people and old people are unreasonable in each their own way, and we need both. </p> <h3 id="df88f595ec814e2bafcbd018ff5f5ad2"> Conclusion <a href="#df88f595ec814e2bafcbd018ff5f5ad2" title="permalink">#</a> </h3> <p> We need young people in the software development industry. Because of their vigour and inexperience, they'll push the envelope. Most will fail to do the impossible, but a few succeed. </p> <p> This may seem like a cynical view, but we've all been young, and most of us have been through such a phase. It's like a rite of passage, and even if you fail to make your mark on the world, you're still likely to have learned a lot. </p> <p> We need old people because they're in a position to speak truth to the world. Notice that I didn't make my argument about the <em>experience</em> of old-timers. Actually, I find that valuable as well, but that's the ordinary argument: <em>Listen to old people, because they have experience and wisdom.</em> </p> <p> Some of them do, at least. </p> <p> I didn't make much out of that argument, because you already know it. There'd be no reason to write this essay if that was all I had to say. Old people have less on the line, so they can speak more freely. If someone you used to admire retires and all of a sudden starts saying or writing unpleasant and surprising things, there might be a good explanation, and it might be a good idea to pay attention. </p> <p> Or maybe he or she is just bitter or going senile... </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Add null check without brackets in Visual Studio https://blog.ploeh.dk/2020/09/07/add-null-check-without-brackets-in-visual-studio 2020-09-07T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>A Visual Studio tweak.</em> </p> <p> The most recent versions of Visual Studio have included many new <em>Quick Actions</em>, accessible with <kbd>Ctrl</kbd> + <kbd>.</kbd> or <kbd>Alt</kbd> + <kbd>Enter</kbd>. The overall feature has been around for some years, but the product group has been adding features at a good pace recently, it seems to me. </p> <p> One feature has been around for at least a year: <em>Add null check</em>. In a default installation of Visual Studio, it looks like this: </p> <p> <img src="/content/binary/add-null-check-with-brackets.png" alt="Screen shot of the 'Add null check' Quick Action. By default it'll add brackets around the throw statement."> </p> <p> As the screen shot shows, it'll auto-generate a Guard Clause like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Reservation</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;otherSeating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;other.Date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Overlaps(otherSeating); }</pre> </p> <p> Part of my personal coding style is that I don't use brackets for one-liners. This is partially motivated by the desire to save vertical space, since I try to <a href="/2019/11/04/the-80-24-rule">keep methods as small as possible</a>. Some people worry that not using brackets for one-liners makes the code more vulnerable to defects, but I typically have automated regressions tests to keep an eye on correctness. </p> <p> The above default behaviour of the <em>Add null check</em> Quick Action annoyed me because I had to manually remove the brackets. It's one of those things that annoy you a little, but not enough that you throw aside what you're doing to figure out if you can change the situation. </p> <p> Until it annoyed me enough to investigate whether there was something I <em>could</em> do about it. It turns out to be easy to tweak the behaviour. </p> <p> In Visual Studio 2019, go to <em>Tools</em>, <em>Options</em>, <em>Text Editor</em>, <em>C#</em>, <em>Code Style</em>, <em>General</em> and change the <em>Prefer braces</em> option to <em>No</em>: </p> <p> <img src="/content/binary/visual-studio-prefer-braces-option.png" alt="Screen shot of Visual Studio Options dialog box."> </p> <p> This changes the behaviour of the <em>Add null check</em> Quick Action: </p> <p> <img src="/content/binary/add-null-check-without-brackets.png" alt="Screen shot of the 'Add null check' Quick Action after the behaviour change. It no longer adds brackets around the throw statement."> </p> <p> After applying the Quick Action, my code now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;Overlaps(<span style="color:#2b91af;">Reservation</span>&nbsp;other) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(other&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(other)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;otherSeating&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Seating</span>(SeatingDuration,&nbsp;other.Date); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Overlaps(otherSeating); }</pre> </p> <p> This better fits my preference. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="bbe35bc429dc4d759fab30cfc90c9bdd"> <div class="comment-author">Daniel</div> <div class="comment-content"> <p> The same should go for if's, switch's, try/catch and perhaps a few other things. While we are at it, spacing is also a good source for changing layout in code, e.g: decimal.Round( amount * 1.0000m, provider.Getdecimals( currency ) ) ). </p> </div> <div class="comment-date">2020-09-07 7:14 UTC</div> </div> <div class="comment" id="ec5cf23c648e4277b851685bc55e258a"> <div class="comment-author">James World</div> <div class="comment-content"> <p> When assigning parameters to fields, I like this one-line option for a null check: </p> <p> <pre>_thing = thing ?? throw new ArgumentNullException(nameof(thing));</pre>. </p> <p> How do you feel about it? You could even use a disard in your example: <pre>_ = thing ?? throw new ArgumentNullException(nameof(thing));</pre> </p> </div> <div class="comment-date">2020-09-07 13:14 UTC</div> </div> <div class="comment" id="e456bf65fe3e426f97639b8855622a7b"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> James, thank you for writing. I'm aware of that option, but rarely use it in my examples. I try to write my example code in a way that they may also be helpful to Java developers, or programmers that use other C-based languages. Obviously, that concern doesn't apply if I discuss something specific to C#, but when I try to explain design patterns or low-level architecture, I try to stay away from language features exclusive to C#. </p> <p> In other contexts, I might or might not use that language feature, but I admit that it jars me. I find that it's a 'clever' solution to something that's not much of a problem. The <code>??</code> operator is intended as a short-circuiting operator one can use to simplify variable assignment. Throwing on the right-hand side never assigns a value. </p> <p> It's as though you would use the <a href="/2018/04/09/coalescing-composite-as-a-monoid">coalescing monoid</a> to pick the left-hand side, but let the program fail on the right-hand side. In Haskell it would look like this: </p> <p> <pre>x = thing &lt;&gt; undefined</pre> </p> <p> The compiler infers that <code>x</code> has the same type as <code>thing</code>, e.g. <code>Semigroup a =&gt; Maybe a</code>. If <code>thing</code>, however, is <code>Nothing</code>, the expression crashes when evaluated. </p> <p> While this compiles in Haskell, I consider it unidiomatic. It too blatantly advertises the existence of the so-called <a href="https://en.wikipedia.org/wiki/Bottom_type">bottom value</a> (⊥). We know it's there, but we prefer to pretend that it's not. </p> <p> That line of reasoning is subjective, and if I find myself in a code base that uses <code>??</code> like you've shown, I'd try to fit in with that style. It doesn't annoy me that much, after all. </p> <p> It's also shorter, I must admit that. </p> </div> <div class="comment-date">2020-09-08 05:57 UTC</div> </div> <div class="comment" id="5c16a687f8b4421fa826ef7c66656767"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> <p> The above default behaviour of the <em>Add null check</em> Quick Action annoyed me because I had to manually remove the brackets. It's one of those things that annoy you a little, but not enough that you throw aside what you're doing to figure out if you can change the situation. </p> <p> Until it annoyed me enough to investigate whether there was something I <em>could</em> do about it. It turns out to be easy to tweak the behaviour. </p> <p> In Visual Studio 2019, go to <em>Tools</em>, <em>Options</em>, <em>Text Editor</em>, <em>C#</em>, <em>Code Style</em>, <em>General</em> and change the <em>Prefer braces</em> option to <em>No</em>: </p> </blockquote> <p> You can do more than that. As suggested by Visual Studio in your second screenshot, you can (after making the change to not prefer braces) generate an <code>.editorconfig</code> file containing your coding style settings. Visual Studio will prompt you to save the file alongside your solution. If you do so, then any developer that opens these files with Visual Studio with also have your setting to not prefer braces. I wrote a <a href="https://tysonwilliams.coding.blog/2020-09-12_editorconfig">short post about EditorConfig</a> that contains more information. </p> </div> <div class="comment-date">2020-09-13 01:53 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Properties for all https://blog.ploeh.dk/2020/08/31/properties-for-all 2020-08-31T08:39:00+00:00 Mark Seemann <div id="post"> <p> <em>Writing test cases for all possible input values.</em> </p> <p> I've noticed that programmers new to automated testing struggle with a fundamental task: how to come up with good test input? </p> <p> There's plenty of design patterns that address that issue, including <a href="http://www.natpryce.com/articles/000714.html">Test Data Builders</a>. Still, test-driven development, when done right, gives you good feedback on the design of your API. I don't consider having to resort to Test Data Builder as much of a compromise, but still, it's even better if you can model the API in question so that it requires no redundant data. </p> <p> Most business processes can be modelled as finite state machines. Once you've understood the problem well enough to enumerate the possible states, you can reverse-engineer an <a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data type</a> from that enumeration. </p> <p> While the data carried around in the state machine may be unconstrained, the number of state and state transitions is usually limited. Model the states as enumerable values, and you can cover all input values with simple combinatorics. </p> <h3 id="44553eea5abb4ab58c73ce173a12ddde"> Tennis states <a href="#44553eea5abb4ab58c73ce173a12ddde" title="permalink">#</a> </h3> <p> The <a href="https://codingdojo.org/kata/Tennis">Tennis kata</a> is one of my favourite exercises. Just like a business process, it turns out that you can model the rules as a finite state machine. There's a state where both players have <em>love</em>, 15, 30, or 40 points (although both can't have 40 points; that state is called <em>deuce</em>); there's a state where a player has the <em>advantage</em>; and so on. </p> <p> I've <a href="https://blog.ploeh.dk/2016/02/10/types-properties-software-designing-with-types">previously written about how to design the states of the Tennis kata with types</a>, so here, I'll just limit myself to the few types that turn out to matter as test input: players and points. </p> <p> Here are both types defined as <a href="https://www.haskell.org">Haskell</a> <a href="https://en.wikipedia.org/wiki/Tagged_union">sum types</a>: </p> <p> <pre><span style="color:blue;">data</span>&nbsp;Player&nbsp;=&nbsp;PlayerOne&nbsp;|&nbsp;PlayerTwo&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>,&nbsp;<span style="color:#2b91af;">Read</span>,&nbsp;<span style="color:#2b91af;">Enum</span>,&nbsp;<span style="color:#2b91af;">Bounded</span>) <span style="color:blue;">data</span>&nbsp;Point&nbsp;=&nbsp;Love&nbsp;|&nbsp;Fifteen&nbsp;|&nbsp;Thirty&nbsp;<span style="color:blue;">deriving</span>&nbsp;(<span style="color:#2b91af;">Eq</span>,&nbsp;<span style="color:#2b91af;">Show</span>,&nbsp;<span style="color:#2b91af;">Read</span>,&nbsp;<span style="color:#2b91af;">Enum</span>,&nbsp;<span style="color:#2b91af;">Bounded</span>)</pre> </p> <p> Notice that both are instances of the <code>Enum</code> and <code>Bounded</code> type classes. This means that it's possible to enumerate all values that inhabit each type. You can use that to your advantage when writing unit tests. </p> <h3 id="5a6d5d80394b4486bb17adc4cd8cf727"> Properties for every value <a href="#5a6d5d80394b4486bb17adc4cd8cf727" title="permalink">#</a> </h3> <p> Most people think of <a href="/property-based-testing-intro">property-based testing</a> as something that involves a special framework such as <a href="https://hackage.haskell.org/package/QuickCheck">QuickCheck</a>, <a href="https://github.com/hedgehogqa">Hedgehog</a>, or <a href="https://fscheck.github.io/FsCheck/index.html">FsCheck</a>. I often use such frameworks, but as I've <a href="/2015/02/23/property-based-testing-without-a-property-based-testing-framework">written about before</a>, sometimes enumerating all possible input values is simpler and faster than relying on randomly generated values. There's no reason to generate 100 random values of a type inhabited by only three values. </p> <p> Instead, write properties for the entire <a href="https://en.wikipedia.org/wiki/Domain_of_a_function">domain</a> of the function in question. </p> <p> In Haskell, you can define a generic enumeration like this: </p> <p> <pre><span style="color:#2b91af;">every</span>&nbsp;::&nbsp;(<span style="color:blue;">Enum</span>&nbsp;a,&nbsp;<span style="color:blue;">Bounded</span>&nbsp;a)&nbsp;<span style="color:blue;">=&gt;</span>&nbsp;[a] every&nbsp;=&nbsp;[<span style="color:blue;">minBound</span>&nbsp;..&nbsp;<span style="color:blue;">maxBound</span>] </pre> </p> <p> I don't know why this function doesn't already exist in the standard library, but I suppose that most people just rely on the list comprehension syntax <code>[minBound .. maxBound]</code>... </p> <p> With it you can write simple properties of the Tennis game. For example: </p> <p> <pre><span style="color:#a31515;">&quot;Given&nbsp;deuce,&nbsp;when&nbsp;player&nbsp;wins,&nbsp;then&nbsp;that&nbsp;player&nbsp;has&nbsp;advantage&quot;</span>&nbsp;~:&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;player&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;score&nbsp;Deuce&nbsp;player &nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;$&nbsp;Advantage&nbsp;player&nbsp;~=?&nbsp;actual </pre> </p> <p> Here I'm using <a href="/2018/05/07/inlined-hunit-test-lists">inlined HUnit test lists</a>. With the Tennis kata, it's easiest to start with the <em>deuce</em> case, because regardless of player, the new state is that the winning player has advantage. That's what that property asserts. Because of the <code>do</code> notation, the property produces a list of test cases, one for <code>every</code> player. There's only two <code>Player</code> values, so it's only two test cases. </p> <p> The beauty of <code>do</code> notation (or the list monad in general) is that you can combine enumerations, for example like this: </p> <p> <pre><span style="color:#a31515;">&quot;Given&nbsp;forty,&nbsp;when&nbsp;player&nbsp;wins&nbsp;ball,&nbsp;then&nbsp;that&nbsp;player&nbsp;wins&nbsp;the&nbsp;game&quot;</span>&nbsp;~:&nbsp;<span style="color:blue;">do</span> &nbsp;&nbsp;player&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;opp&nbsp;&lt;-&nbsp;every &nbsp;&nbsp;<span style="color:blue;">let</span>&nbsp;actual&nbsp;=&nbsp;score&nbsp;(Forty&nbsp;$&nbsp;FortyData&nbsp;player&nbsp;opp)&nbsp;player &nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;$&nbsp;Game&nbsp;player&nbsp;~=?&nbsp;actual </pre> </p> <p> While <code>player</code> is a <code>Player</code> value, <code>opp</code> (<em>Other Player's Point</em>) is a <code>Point</code> value. Since there's two possible <code>Player</code> values and three possible <code>Point</code> values, the combination yields six test cases. </p> <p> I've been doing the Tennis kata a couple of times using this approach. I've also done it in C# with <a href="https://xunit.net">xUnit.net</a>, where the first test of the <em>deuce</em> state looks like this: </p> <p> <pre>[<span style="color:#2b91af;">Theory</span>,&nbsp;<span style="color:#2b91af;">MemberData</span>(<span style="color:blue;">nameof</span>(Player))] <span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;TransitionFromDeuce(<span style="color:#2b91af;">IPlayer</span>&nbsp;player) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;sut&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Deuce</span>(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actual&nbsp;=&nbsp;sut.BallTo(player); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;expected&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Advantage</span>(player); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Assert</span>.Equal(expected,&nbsp;actual); }</pre> </p> <p> <a href="/2019/12/16/zone-of-ceremony">C# takes more ceremony than Haskell</a>, but the idea is the same. The <code>Player</code> data source for the <code>[Theory]</code> is defined as an enumeration of the possible values: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IEnumerable</span>&lt;<span style="color:blue;">object</span>[]&gt;&nbsp;Player { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">get</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">object</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PlayerOne</span>()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">yield</span>&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:blue;">object</span>[]&nbsp;{&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PlayerTwo</span>()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> All the times I've done the Tennis kata by fully exhausting the domain of the transition functions in question, I've arrived at 40 test cases. I wonder if that's the number of possible state transitions of the game, or if it's just an artefact of the way I've modelled it. I suspect the latter... </p> <h3 id="21ccf5bd3c3f48859bbb8c766b0a0610"> Conclusion <a href="#21ccf5bd3c3f48859bbb8c766b0a0610" title="permalink">#</a> </h3> <p> You can sometimes enumerate all possible inputs to an API. Even if a function takes a Boolean value and a byte as input, the enumeration of all possible combinations is only 2 * 256 = 512 values. You computer will tear through all of those combinations faster than you can say <em>random selection</em>. Consider writing APIs that take algebraic data types as input, and writing properties that exhaust the domain of the functions in question. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Adding REST links as a cross-cutting concern https://blog.ploeh.dk/2020/08/24/adding-rest-links-as-a-cross-cutting-concern 2020-08-24T06:47:00+00:00 Mark Seemann <div id="post"> <p> <em>Use a piece of middleware to enrich a Data Transfer Object. An ASP.NET Core example.</em> </p> <p> When developing true REST APIs, you should <a href="https://martinfowler.com/articles/richardsonMaturityModel.html">use hypermedia controls</a> (i.e. <em>links</em>) to guide clients to the resources they need. I've always felt that the code that generates these links tends to make otherwise readable Controller methods unreadable. </p> <p> I'm currently experimenting with generating links as a cross-cutting concern. So far, I like it very much. </p> <h3 id="57785b860f604552b099e42b296f6fb6"> Links from home <a href="#57785b860f604552b099e42b296f6fb6" title="permalink">#</a> </h3> <p> Consider an online restaurant reservation system. When you make a <code>GET</code> request against the home resource (which is the only published URL for the API), you should receive a representation like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:reservations&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/reservations&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/13&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> As you can tell, my example just runs on my local development machine, but I'm sure that you can see past that. There's three calendar links that clients can use to <code>GET</code> the restaurant's calendar for the current day, month, or year. Clients can use these resources to present a user with a date picker or a similar user interface so that it's possible to pick a date for a reservation. </p> <p> When a client wants to make a reservation, it can use the URL identified by the <code>rel</code> (<em>relationship type</em>) <code>"urn:reservations"</code> to make a <code>POST</code> request. </p> <h3 id="724d54b2bb4a4071b0f965105541c792"> Link generation as a Controller responsibility <a href="#724d54b2bb4a4071b0f965105541c792" title="permalink">#</a> </h3> <p> I first wrote the code that generates these links directly in the Controller class that serves the home resource. It looked like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IActionResult</span>&nbsp;Get() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;links&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">List</span>&lt;<span style="color:#2b91af;">LinkDto</span>&gt;(); &nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToReservations()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(enableCalendar) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToYear(now.Year)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToMonth(now.Year,&nbsp;now.Month)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;links.Add(Url.LinkToDay(now.Year,&nbsp;now.Month,&nbsp;now.Day)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;Ok(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>&nbsp;{&nbsp;Links&nbsp;=&nbsp;links.ToArray()&nbsp;}); }</pre> </p> <p> That doesn't look too bad, but 90% of the code is exclusively concerned with generating links. (<code>enableCalendar</code>, by the way, is a <a href="https://en.wikipedia.org/wiki/Feature_toggle">feature flag</a>.) That seems acceptable in this special case, because there's really nothing else the home resource has to do. For other resources, the Controller code might contain some composition code as well, and then all the link code starts to look like noise that makes it harder to understand the actual purpose of the Controller method. You'll see an example of a non-trivial Controller method later in this article. </p> <p> It seemed to me that enriching a Data Transfer Object (DTO) with links ought to be a cross-cutting concern. </p> <h3 id="1973741c11414414bcd0ca2748bf0f0d"> LinksFilter <a href="#1973741c11414414bcd0ca2748bf0f0d" title="permalink">#</a> </h3> <p> In ASP.NET Core, you can implement cross-cutting concerns with a type of middleware called <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.filters.iasyncactionfilter">IAsyncActionFilter</a>. I added one called <code>LinksFilter</code>: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LinksFilter</span>&nbsp;:&nbsp;<span style="color:#2b91af;">IAsyncActionFilter</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">bool</span>&nbsp;enableCalendar; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;UrlHelperFactory&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">LinksFilter</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IUrlHelperFactory</span>&nbsp;urlHelperFactory, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">CalendarFlag</span>&nbsp;calendarFlag) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;UrlHelperFactory&nbsp;=&nbsp;urlHelperFactory; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;enableCalendar&nbsp;=&nbsp;calendarFlag.Enabled; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;OnActionExecutionAsync( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutingContext</span>&nbsp;context, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">ActionExecutionDelegate</span>&nbsp;next) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;ctxAfter&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;next().ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(!(ctxAfter.Result&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>&nbsp;ok)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;url&nbsp;=&nbsp;UrlHelperFactory.GetUrlHelper(ctxAfter); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">switch</span>&nbsp;(ok.Value) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">case</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>&nbsp;homeDto: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(homeDto,&nbsp;url); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">case</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span>&nbsp;calendarDto: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(calendarDto,&nbsp;url); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">default</span>: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">break</span>; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> There's only one method to implement. If you want to run some code <em>after</em> the Controllers have had their chance, you invoke the <code>next</code> delegate to get the resulting context. It should contain the response to be returned. If <code>Result</code> isn't an <code>OkObjectResult</code> there's no content to enrich with links, so the method just returns. </p> <p> Otherwise, it switches on the type of the <code>ok.Value</code> and passes the DTO to an appropriate helper method. Here's the <code>AddLinks</code> overload for <code>HomeDto</code>: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddLinks(<span style="color:#2b91af;">HomeDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(enableCalendar) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToReservations(), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToYear(now.Year), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToMonth(now.Year,&nbsp;now.Month), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToDay(now.Year,&nbsp;now.Month,&nbsp;now.Day) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">else</span> &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[]&nbsp;{&nbsp;url.LinkToReservations()&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> You can probably recognise the implemented behaviour from before, where it was implemented in the <code>Get</code> method. That method now looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">ActionResult</span>&nbsp;Get() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">HomeDto</span>()); }</pre> </p> <p> That's clearly much simpler, but you probably think that little has been achieved. After all, doesn't this just move some code from one place to another? </p> <p> Yes, that's the case in this particular example, but I wanted to start with an example that was so simple that it highlights how to move the code to a filter. Consider, then, the following example. </p> <h3 id="9156b0b69f4d485fa373a8c4b06547d5"> A calendar resource <a href="#9156b0b69f4d485fa373a8c4b06547d5" title="permalink">#</a> </h3> <p> The online reservation system enables clients to navigate its calendar to look up dates and time slots. A representation might look like this: </p> <p> <pre>{ &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;previous&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/12&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;next&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/14&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;], &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;year&quot;</span>:&nbsp;2020, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;month&quot;</span>:&nbsp;8, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;day&quot;</span>:&nbsp;13, &nbsp;&nbsp;<span style="color:#2e75b6;">&quot;days&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;links&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:year&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:month&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;rel&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;urn:day&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;href&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;http://localhost:53568/calendar/2020/8/13&quot;</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;], &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;date&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;2020-08-13&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;entries&quot;</span>:&nbsp;[ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;18:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;19:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:15:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:30:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;20:45:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;time&quot;</span>:&nbsp;<span style="color:#a31515;">&quot;21:00:00&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2e75b6;">&quot;maximumPartySize&quot;</span>:&nbsp;10 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;] &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;] }</pre> </p> <p> This is a JSON representation of the calendar for August 13, 2020. The data it contains is the identification of the date, as well as a series of <code>entries</code> that lists the largest reservation the restaurant can accept for each time slot. </p> <p> Apart from the data, the representation also contains links. There's a general collection of links that currently holds only <code>next</code> and <code>previous</code>. In addition to that, each day has its own array of links. In the above example, only a single day is represented, so the <code>days</code> array contains only a single object. For a month calendar (navigatable via the <code>urn:month</code> link), there'd be between 28 and 31 <code>days</code>, each with its own <code>links</code> array. </p> <p> Generating all these links is a complex undertaking all by itself, so separation of concerns is a boon. </p> <h3 id="8f85a3a882164352aa57d600a21628be"> Calendar links <a href="#8f85a3a882164352aa57d600a21628be" title="permalink">#</a> </h3> <p> As you can see in the above <code>LinksFilter</code>, it branches on the type of value wrapped in an <code>OkObjectResult</code>. If the type is <code>CalendarDto</code>, it calls the appropriate <code>AddLinks</code> overload: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">void</span>&nbsp;AddLinks(<span style="color:#2b91af;">CalendarDto</span>&nbsp;dto,&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;dto.ToPeriod(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;previous&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">PreviousPeriodVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;next&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">NextPeriodVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;dto.Links&nbsp;=&nbsp;<span style="color:blue;">new</span>[] &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToPeriod(previous,&nbsp;<span style="color:#a31515;">&quot;previous&quot;</span>), &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.LinkToPeriod(next,&nbsp;<span style="color:#a31515;">&quot;next&quot;</span>) &nbsp;&nbsp;&nbsp;&nbsp;}; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(dto.Days&nbsp;<span style="color:blue;">is</span>&nbsp;{&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">foreach</span>&nbsp;(<span style="color:blue;">var</span>&nbsp;day&nbsp;<span style="color:blue;">in</span>&nbsp;dto.Days) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AddLinks(day,&nbsp;url); }</pre> </p> <p> It both generates the <code>previous</code> and <code>next</code> links on the <code>dto</code>, as well as the links for each day. While I'm not going to bore you with more of that code, you can tell, I hope, that the <code>AddLinks</code> method calls other helper methods and classes. The point is that link generation involves more than just a few lines of code. </p> <p> You already saw that in the first example (related to <code>HomeDto</code>). The question is whether there's still some significant code left in the Controller class? </p> <h3 id="ab37807dfce0431b8308f86ec18a44aa"> Calendar resource <a href="#ab37807dfce0431b8308f86ec18a44aa" title="permalink">#</a> </h3> <p> The <code>CalendarController</code> class defines three overloads of <code>Get</code> - one for a single day, one for a month, and one for an entire year. Each of them looks like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">ActionResult</span>&gt;&nbsp;Get(<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;period&nbsp;=&nbsp;<span style="color:#2b91af;">Period</span>.Month(year,&nbsp;month); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;MakeDays(period).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">OkObjectResult</span>( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">CalendarDto</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Year&nbsp;=&nbsp;year, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Month&nbsp;=&nbsp;month, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Days&nbsp;=&nbsp;days &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}); }</pre> </p> <p> It doesn't look as though much is going on, but at least you can see that it returns a <code>CalendarDto</code> object. </p> <p> While the method looks simple, it's not. Significant work happens in the <code>MakeDays</code> helper method: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">DayDto</span>[]&gt;&nbsp;MakeDays(<span style="color:#2b91af;">IPeriod</span>&nbsp;period) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;firstTick&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">FirstTickVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;lastTick&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LastTickVisitor</span>()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;reservations&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;Repository &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ReadReservations(firstTick,&nbsp;lastTick).ConfigureAwait(<span style="color:blue;">false</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;days&nbsp;=&nbsp;period.Accept(<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">DaysVisitor</span>()) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Select(d&nbsp;=&gt;&nbsp;MakeDay(d,&nbsp;reservations)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.ToArray(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;days; }</pre> </p> <p> After having read relevant <code>reservations</code> from the database, it applies complex business logic to allocate them and thereby being able to report on remaining capacity for each time slot. </p> <p> Not having to worry about link generation while doing all that work seems like a benefit. </p> <h3 id="87b164914be0450fbd523735946cf146"> Filter registration <a href="#87b164914be0450fbd523735946cf146" title="permalink">#</a> </h3> <p> You must tell the ASP.NET Core framework about any filters that you add. You can do that in the <code>Startup</code> class' <code>ConfigureServices</code> method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">void</span>&nbsp;ConfigureServices(<span style="color:#2b91af;">IServiceCollection</span>&nbsp;services) { &nbsp;&nbsp;&nbsp;&nbsp;services.AddControllers(opts&nbsp;=&gt;&nbsp;opts.Filters.Add&lt;<span style="color:#2b91af;">LinksFilter</span>&gt;()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> When registered, the filter executes for each HTTP request. When the object represents a <code>200 OK</code> result, the filter populates the DTOs with links. </p> <h3 id="ced9fd76d526420aa1a847da4398e904"> Conclusion <a href="#ced9fd76d526420aa1a847da4398e904" title="permalink">#</a> </h3> <p> By treating RESTful link generation as a cross-cutting concern, you can separate if from the logic of generating the data structure that represents the resource. That's not the only way to do it. You could also write a simple function that populates DTOs, and call it directly from each Controller action. </p> <p> What I like about using a filter is that I don't have to remember to do that. Once the filter is registered, it'll populate all the DTOs it knows about, regardless of which Controller generated them. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="fff391e80e27427e8fc93959deef3768"> <div class="comment-author"><a href="https://twitter.com/MehdiFalamarzi?s=08">Mehdi Falamarzi</a></div> <div class="comment-content"> <p> Thanks for your good and insightful posts. </p> <p> Separation of REST concerns from MVC controller's concerns is a great idea, But in my opinion this solution has two problems: </p> <h3 id="52608f57b49942aaaafdab7aae22c85"> Distance between related REST implementations <a href="#52608f57b49942aaaafdab7aae22c85" title="permalink">#</a> </h3> <p> When implementing REST by MVC pattern often REST archetypes are the reasons for a MVC Controller class to be created. As long as the MVC Controller class describes the archetype, And links of a resource is a part of the response when implementing hypermedia controls, having the archetype and its related links in the place where the resource described is a big advantage in easiness and readability of the design. pulling out link implementations and putting them in separate classes causes higher readability and uniformity of the code abstranction levels in the action method at the expense of making a distance between related REST implementation. </p> <h3 id="fa1203384d3746bfadd276aae6ac860f"> Implementation scalability <a href="#fa1203384d3746bfadd276aae6ac860f" title="permalink">#</a> </h3> <p> There is a switch statement on the ActionExecutingContext's result in the LinksFilter to decide what links must be represented to the client in the response.The DTOs thre are the results of the clients's requests for URIs. If this solution generalised for every resources the API must represent there will be several cases for the switch statement to handle. Beside that every resources may have different implemetations for generating their links. Putting all this in one place leads the LinksFilter to be coupled with too many helper classes and this coupling process never stops. </p> <h3 id="4dd0703da482480abffbaffabdb27741"> Solution <a href="#4dd0703da482480abffbaffabdb27741" title="permalink">#</a> </h3> <p> LinkDescriptor and LinkSubscriber for resources links definition </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">LinkDescriptor</span> { <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Rel { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Href { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #333399; font-weight: bold">string</span> Resource { <span style="color: #008800; font-weight: bold">get</span>; <span style="color: #008800; font-weight: bold">set</span>; } } </pre> </div> <p> Letting the MVC controller classes have their resource's links definitions but not in action methods. </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> ActionResult <span style="color: #0066BB; font-weight: bold">Get</span>() { <span style="color: #008800; font-weight: bold">return</span> <span style="color: #008800; font-weight: bold">new</span> <span style="color: #0066BB; font-weight: bold">OkObjectResult</span>(<span style="color: #008800; font-weight: bold">new</span> HomeDto()); } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">RegisterLinks</span>(LinkSubscriber subscriber) { subscriber .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:reservations&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/reservations&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:year&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:month&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020/8&quot;</span>) .Add( resource: <span style="background-color: #fff0f0">&quot;/Home&quot;</span>, rel: <span style="background-color: #fff0f0">&quot;urn:day&quot;</span>, href: <span style="background-color: #fff0f0">&quot;/calendar/2020/8/13&quot;</span>); } </pre> </div> <p> Registering resources links by convention </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">MvcBuilderExtensions</span> { <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">static</span> IMvcBuilder <span style="color: #0066BB; font-weight: bold">AddRestLinks</span>(<span style="color: #008800; font-weight: bold">this</span> IMvcBuilder mvcBuilder) { <span style="color: #333399; font-weight: bold">var</span> subscriber = <span style="color: #008800; font-weight: bold">new</span> LinkSubscriber(); <span style="color: #333399; font-weight: bold">var</span> linkRegistrationMethods = GetLinkRegistrationMethods(mvcBuilder.Services); PopulateLinks(subscriber, linkRegistrationMethods); mvcBuilder.Services.AddSingleton&lt;IEnumerable&lt;LinkDescriptor&gt;&gt;(subscriber.LinkDescriptors); <span style="color: #008800; font-weight: bold">return</span> mvcBuilder; } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">static</span> List&lt;MethodInfo&gt; GetLinkRegistrationMethods(IServiceCollection services) { <span style="color: #008800; font-weight: bold">return</span> <span style="color: #0066BB; font-weight: bold">typeof</span>(MvcBuilderExtensions).Assembly.ExportedTypes .Where(tp =&gt; <span style="color: #008800; font-weight: bold">typeof</span>(ControllerBase).IsAssignableFrom(tp)) .Select(tp =&gt; tp.GetMethod(<span style="background-color: #fff0f0">&quot;RegisterLinks&quot;</span>, <span style="color: #008800; font-weight: bold">new</span>[] { <span style="color: #008800; font-weight: bold">typeof</span>(LinkSubscriber) })) .Where(mi =&gt; mi != <span style="color: #008800; font-weight: bold">null</span>) .ToList(); } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">static</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">PopulateLinks</span>(LinkSubscriber subscriber, List&lt;MethodInfo&gt; linkRegistrationMethods) { <span style="color: #008800; font-weight: bold">foreach</span> (<span style="color: #333399; font-weight: bold">var</span> method <span style="color: #008800; font-weight: bold">in</span> linkRegistrationMethods) { method.Invoke(<span style="color: #008800; font-weight: bold">null</span>, <span style="color: #008800; font-weight: bold">new</span>[] { subscriber }); } } } </pre> </div> <p> Add dependencies and execute procedures for adding links to responses </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">ConfigureServices</span>(IServiceCollection services) { services.AddControllers(conf =&gt; conf.Filters.Add&lt;LinksFilter&gt;()) .AddRestLinks(); } </pre> </div> <p> And Last letting the LinksFilter to dynamicaly add resources links by utilizing ExpandoObject </p> <div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"> <pre style="margin: 0; line-height: 125%"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">class</span> <span style="color: #BB0066; font-weight: bold">LinksFilter</span> : IAsyncActionFilter { <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">readonly</span> IEnumerable&lt;LinkDescriptor&gt; links; <span style="color: #008800; font-weight: bold">public</span> <span style="color: #0066BB; font-weight: bold">LinksFilter</span>(IEnumerable&lt;LinkDescriptor&gt; links) { <span style="color: #008800; font-weight: bold">this</span>.links = links; } <span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">async</span> Task <span style="color: #0066BB; font-weight: bold">OnActionExecutionAsync</span>(ActionExecutingContext context, ActionExecutionDelegate next) { <span style="color: #333399; font-weight: bold">var</span> relatedLinks = links .Where(lk =&gt; context.HttpContext.Request.Path.Value.ToLower() == lk.Resource); <span style="color: #008800; font-weight: bold">if</span> (relatedLinks.Any()) <span style="color: #008800; font-weight: bold">await</span> <span style="color: #0066BB; font-weight: bold">ManipulateResponseAsync</span>(context, next, relatedLinks); } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">async</span> Task <span style="color: #0066BB; font-weight: bold">ManipulateResponseAsync</span>(ActionExecutingContext context, ActionExecutionDelegate next, IEnumerable&lt;LinkDescriptor&gt; relatedLinks) { <span style="color: #333399; font-weight: bold">var</span> ctxAfter = <span style="color: #008800; font-weight: bold">await</span> next().ConfigureAwait(<span style="color: #008800; font-weight: bold">false</span>); <span style="color: #008800; font-weight: bold">if</span> (!(ctxAfter.Result <span style="color: #008800; font-weight: bold">is</span> ObjectResult objRes)) <span style="color: #008800; font-weight: bold">return</span>; <span style="color: #333399; font-weight: bold">var</span> expandoResult = <span style="color: #008800; font-weight: bold">new</span> ExpandoObject(); FillExpandoWithResultProperties(expandoResult, objRes.Value); FillExpandoWithLinks(expandoResult, relatedLinks); objRes.Value = expandoResult; } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">FillExpandoWithResultProperties</span>(ExpandoObject resultExpando, <span style="color: #333399; font-weight: bold">object</span> <span style="color: #008800; font-weight: bold">value</span>) { <span style="color: #333399; font-weight: bold">var</span> properties = <span style="color: #008800; font-weight: bold">value</span>.GetType().GetProperties(); <span style="color: #008800; font-weight: bold">foreach</span> (<span style="color: #333399; font-weight: bold">var</span> property <span style="color: #008800; font-weight: bold">in</span> properties) { resultExpando.TryAdd(property.Name, property.GetValue(<span style="color: #008800; font-weight: bold">value</span>)); } } <span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">void</span> <span style="color: #0066BB; font-weight: bold">FillExpandoWithLinks</span>(ExpandoObject resultExpando, IEnumerable&lt;LinkDescriptor&gt; relatedLinks) { <span style="color: #333399; font-weight: bold">var</span> linksToAdd = relatedLinks.Select(lk =&gt; <span style="color: #008800; font-weight: bold">new</span> { Rel = lk.Rel, Href = lk.Href }); resultExpando.TryAdd(<span style="background-color: #fff0f0">&quot;Links&quot;</span>, linksToAdd); } } </pre> </div> <p> If absoulute URI in href field is prefered, IUriHelper can be injected in LinksFilter to create URI paths. </p> </div> <div class="comment-date">2020-08-24 23:39 UTC</div> </div> <div class="comment" id="127d2750e83811ea8dd900155d8065e1"> <div class="comment-author"><a href="https://github.com/JesHansen">Jes Hansen</a></div> <div class="comment-content"> <p> Mark, thanks for figuring out the tricky parts so we don't have to. :-) </p> <p> I did not see a link to a repo with the completed code from this article, and a cursory look around your Github profile didn't give away any obvious clues. Is the example code in the article part of a repo we can clone? If so, could you please provide a link? </p> </div> <div class="comment-date">2020-08-27 07:45 UTC</div> </div> <div class="comment" id="13197699aaff4a90901910a1975c4de0"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Jes, it's part of a larger project that I'm currently working on. Eventually, I hope to publish it, but it's not yet in a state where I wish to do that. </p> <p> Did I leave out essential details that makes it hard to reproduce the idea? </p> </div> <div class="comment-date">2020-08-27 08:07 UTC</div> </div> <div class="comment" id="0e3477f6e84311eaadc700155d8065e1"> <div class="comment-author"><a href="https://github.com/JesHansen">Jes Hansen</a></div> <div class="comment-content"> <p> No, your presentation was fine. Looking forward to see the completed project! </p> </div> <div class="comment-date">2020-08-27 08:57 UTC</div> </div> <div class="comment" id="a5f42d7c00e947a885506abdd2a16628"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Mehdi, thank you for writing. It's true that the <code>LinksFilter</code> implementation code contains a <code>switch</code> statement, and that this is one of multiple possible designs. I do believe that this is a trade-off rather than a problem per se. </p> <p> That <code>switch</code> statement is an implementation detail of the filter, and something I might decide to change in the future. I did choose that implementation, though, because I it was the simplest design that came to my mind. As presented, the <code>switch</code> statement just calls some private helper methods (all called <code>AddLinks</code>), but if one wanted that code close to the rest of the Controller code, one could just move those helper methods to the relevant Controller classes. </p> <p> While you wouldn't need to resort to Reflection to do that, it's true that this would leave that <code>switch</code> statement as a central place where developers would have to go if they add a new resource. It's true that your proposed solution addresses that problem, but doesn't it just shift the burden somewhere else? Now, developers will have to know that they ought to add a <code>RegisterLinks</code> method with a specific signature to their Controller classes. This replaces a design with compile-time checking with something that may fail at run time. How is that an improvement? </p> <p> I think that I understand your other point about the distance of code, but it assumes a particular notion of REST that I find parochial. Most (.NET) developers I've met design REST APIs in a code-centric (if not a database-centric) way. They tend to conflate <em>representations</em> with <em>resources</em> and translate both to Controller classes. </p> <p> The idea behind <em>Representational State Transfer</em>, however, is to decouple state from representation. Resources have state, but can have multiple representations. Vice versa, many resources may share the same representation. In the code base I used for this article, not only do I have three overloaded <code>Get</code> methods on <code>CalendarController</code> that produce <code>CalendarDto</code> representations, I also have a <code>ScheduleController</code> class that does the same. </p> <p> Granted, not all REST API code bases are designed like this. I admit that what actually motivated me to do things like this was to avoid having to inherit from <code>ControllerBase</code>. Moving all the code that relies heavily on the ASP.NET infrastructure keeps the Controller classes lighter, and thus easier to test. I should probably write an article about that... </p> </div> <div class="comment-date">2020-09-01 07:56 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Unit testing is fine https://blog.ploeh.dk/2020/08/17/unit-testing-is-fine 2020-08-17T05:29:00+00:00 Mark Seemann <div id="post"> <p> <em>Unit testing considered harmful? I think not.</em> </p> <p> Once in a while, some article, video, or podcast makes the rounds on social media, arguing that unit testing is bad, overrated, harmful, or the like. I'm not going to link to any specific resources, because this post isn't an attack on any particular piece of work. Many of these are sophisticated, thoughtful, and make good points, but still arrive at the wrong conclusion. </p> <p> The line of reasoning tends to be to show examples of bad unit tests and conclude that, based on the examples, unit tests are bad. </p> <p> The power of examples is great, but clearly, this is a logical fallacy. </p> <h3 id="5dbc2a18fc7f439b9db0282f0613b2a5"> Symbolisation <a href="#5dbc2a18fc7f439b9db0282f0613b2a5" title="permalink">#</a> </h3> <p> In case it isn't clear that the argument is invalid, I'll demonstrate it using the techniques I've learned from <a href="https://bit.ly/predicate-logic-intro">Howard Pospesel's introduction to predicate logic</a>. </p> <p> We can begin by symbolising the natural-language arguments into well-formed formulas. I'll keep this as simple as possible: </p> <p> <pre>∃xBx ⊢ ∀xBx (Domain: unit tests; Bx = x is bad)</pre> </p> <p> Basically, <code>Bx</code> states that <code>x</code> is bad, where <code>x</code> is a unit test. <code>∃xBx</code> is a statement that there exists a unit test <code>x</code> for which <code>x</code> is bad; i.e. <em>bad unit tests exist</em>. The statement <code>∀xBx</code> claims that for all unit tests <code>x</code>, <code>x</code> is bad; i.e. <em>all unit tests are bad</em>. The turnstile symbol <code>⊢</code> in the middle indicates that the antecedent on the left proves the consequent to the right. </p> <p> Translated back to natural language, the claim is this: <em>Because bad unit tests exist, all unit tests are bad.</em> </p> <p> You can trivially prove this <a href="https://en.wikipedia.org/wiki/Sequent">sequent</a> invalid. </p> <h3 id="a5f5e5c8e1f94696b8ffb86a119170b4"> Logical fallacy <a href="#a5f5e5c8e1f94696b8ffb86a119170b4" title="permalink">#</a> </h3> <p> One way to prove the sequent invalid is to use a truth tree: </p> <p> <img src="/content/binary/truth-tree-open-all-unit-tests-bad.png" alt="Truth tree that disproves the sequent about all unit tests being bad."> </p> <p> Briefly, the way this works is that the statements on the left-hand side represent truth, while the ones to the right are false. By placing the antecedent on the left, but the consequent on the right, you're basically assuming the sequent to be wrong. This is is also the way you <em>prove</em> correct sequents true; if the conclusion is assumed false, a logical truth should lead to a contradiction. If it doesn't, the sequent is invalid. That's what happens here. </p> <p> The tree remains <em>open</em>, which means that the original sequent is invalid. It's a logical fallacy. </p> <h3 id="c19e880f945d4a44a40ca1ccb9b96f74"> Counter examples <a href="#c19e880f945d4a44a40ca1ccb9b96f74" title="permalink">#</a> </h3> <p> You probably already knew that. All it takes to counter a universal assertion such as <em>all unit tests are bad</em> is to produce a counter example. One is sufficient, because if just a single good unit test exists, it can't be true that <em>all</em> are bad. </p> <p> Most of the think pieces that argue that unit testing is bad do so by showing examples of bad unit tests. These tests typically involve lots of mocks and stubs; they tend to test the interaction between internal components instead of the components themselves, or the system as a whole. I agree that this often leads to <a href="http://xunitpatterns.com/Fragile%20Test.html">fragile tests</a>. </p> <p> While I still spend my testing energy according to the <a href="https://martinfowler.com/bliki/TestPyramid.html">Test Pyramid</a>, I don't write unit tests like that. I rarely use dynamic mock libraries. Instead, I <a href="/2020/03/02/impureim-sandwich">push impure actions to the boundary of the system</a> and write most of the application code as <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>, which are <a href="/2015/05/07/functional-design-is-intrinsically-testable">intrinsically testable</a>. No test-induced damage there. </p> <p> I follow up with <a href="/outside-in-tdd">boundary tests</a> that demonstrate that the <a href="/2015/12/21/integration-testing-composed-functions">functions are integrated into a working system</a>. That's just another layer in the Test Pyramid, but smaller. You don't need that many integration tests when you have a foundation of good unit tests. </p> <p> While I'm currently working on a larger body of work that showcases this approach, this blog <a href="/2019/04/01/an-example-of-state-based-testing-in-c">already has examples of this</a>. </p> <h3 id="414e235a672042a6a07e572027250df0"> Conclusion <a href="#414e235a672042a6a07e572027250df0" title="permalink">#</a> </h3> <p> You often hear or see the claim that unit tests are bad. The supporting argument is that a particular (popular, I admit) style of unit testing is bad. </p> <p> If the person making this claim only knows of that single style of unit testing, it's natural to jump to the conclusion that all unit testing must be bad. </p> <p> That's not the case. I write most of my unit tests in a style dissimilar from the interaction-heavy, mocks-and-stubs-based style that most people use. These test have a low maintenance burden and don't cause test-induced damage. </p> <p> Unit testing is fine. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. An ASP.NET Core URL Builder https://blog.ploeh.dk/2020/08/10/an-aspnet-core-url-builder 2020-08-10T06:59:00+00:00 Mark Seemann <div id="post"> <p> <em>A use case for the Immutable Fluent Builder design pattern variant.</em> </p> <p> The <a href="/2020/02/10/builder-isomorphisms">Fluent Builder</a> design pattern is popular in object-oriented programming. Most programmers use the mutable variant, while I favour the immutable alternative. The advantages of Immutable Fluent Builders, however, may not be immediately clear. <blockquote> <p> "I never thought of someone reusing a configured builder (soulds like too big class/SRP violation)." </p> <footer><cite><a href="https://twitter.com/ELuciusFTW/status/1226800015047348224">Guy Buss</a></cite></footer> </blockquote> It inspires me when I encounter a differing perspective. Could I be wrong? Or did I fail to produce a compelling example? </p> <p> It's possible that I'm wrong, but in my <a href="/2020/02/10/builder-isomorphisms">my recent article on Builder isomorphisms</a> I focused on the pattern variations themselves, to the point where a convincing example wasn't my top priority. </p> <p> I recently encountered a good use case for an Immutable Fluent Builder. </p> <h3 id="e2761ee2b9654ad982a2ec1a6264c76f"> Build links <a href="#e2761ee2b9654ad982a2ec1a6264c76f" title="permalink">#</a> </h3> <p> I was developing a REST API and wanted to generate some links like these: </p> <p> <pre>{ "links": [ { "rel": "urn:reservations", "href": "http://localhost:53568/reservations" }, { "rel": "urn:year", "href": "http://localhost:53568/calendar/2020" }, { "rel": "urn:month", "href": "http://localhost:53568/calendar/2020/7" }, { "rel": "urn:day", "href": "http://localhost:53568/calendar/2020/7/7" } ] }</pre> </p> <p> As I recently described, <a href="/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper">the ASP.NET Core Action API is tricky</a>, and since there was some repetition, I was looking for a way to reduce the code duplication. At first I just thought I'd make a few private helper methods, but then it occurred to me that an Immutable Fluent Builder as an <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a> to the <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.urlhelperextensions.action">Action API</a> might offer a fertile alternative. </p> <h3 id="5adf09c648a54679b1b137776e4f1192"> UrlBuilder class <a href="#5adf09c648a54679b1b137776e4f1192" title="permalink">#</a> </h3> <p> The various <code>Action</code> overloads all accept null arguments, so there's effectively no clear invariants to enforce on that dimension. While I wanted an Immutable Fluent Builder, I made all the fields nullable. </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;action; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;controller; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">object</span>?&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(<span style="color:blue;">string</span>?&nbsp;action,&nbsp;<span style="color:blue;">string</span>?&nbsp;controller,&nbsp;<span style="color:blue;">object</span>?&nbsp;values) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.action&nbsp;=&nbsp;action; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.controller&nbsp;=&nbsp;controller; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.values&nbsp;=&nbsp;values; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:green;">//&nbsp;...</span></pre> </p> <p> I also gave the <code>UrlBuilder</code> class a public constructor and a private copy constructor. That's the standard way I implement that pattern. </p> <p> Most of the modification methods are straightforward: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithAction(<span style="color:blue;">string</span>&nbsp;newAction) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(newAction,&nbsp;controller,&nbsp;values); } <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithValues(<span style="color:blue;">object</span>&nbsp;newValues) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(action,&nbsp;controller,&nbsp;newValues); }</pre> </p> <p> I wanted to encapsulate <a href="/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper">the suffix-handling behaviour I recently described</a> in the appropriate method: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;WithController(<span style="color:blue;">string</span>&nbsp;newController) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(newController&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(newController)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">const</span>&nbsp;<span style="color:blue;">string</span>&nbsp;controllerSuffix&nbsp;=&nbsp;<span style="color:#a31515;">&quot;controller&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;index&nbsp;=&nbsp;newController.LastIndexOf( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controllerSuffix, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">StringComparison</span>.OrdinalIgnoreCase); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(0&nbsp;&lt;=&nbsp;index) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;newController&nbsp;=&nbsp;newController.Remove(index); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>(action,&nbsp;newController,&nbsp;values); }</pre> </p> <p> The <code>WithController</code> method handles both the case where <code>newController</code> is suffixed by <code>"Controller"</code> and the case where it isn't. I also wrote unit tests to verify that the implementation works as intended. </p> <p> Finally, a Builder should have a method to build the desired object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">Uri</span>&nbsp;BuildAbsolute(<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(url&nbsp;<span style="color:blue;">is</span>&nbsp;<span style="color:blue;">null</span>) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">throw</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">ArgumentNullException</span>(<span style="color:blue;">nameof</span>(url)); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;actionUrl&nbsp;=&nbsp;url.Action( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;action, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;controller, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;values, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;url.ActionContext.HttpContext.Request.Host.ToUriComponent()); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Uri</span>(actionUrl); }</pre> </p> <p> One could imagine also defining a <code>BuildRelative</code> method, but I didn't need it. </p> <h3 id="bec08c507ce7475eb0038723c6f6f8a8"> Generating links <a href="#bec08c507ce7475eb0038723c6f6f8a8" title="permalink">#</a> </h3> <p> Each of the objects shown above are represented by a simple Data Transfer Object: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">LinkDto</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Rel&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">string</span>?&nbsp;Href&nbsp;{&nbsp;<span style="color:blue;">get</span>;&nbsp;<span style="color:blue;">set</span>;&nbsp;} }</pre> </p> <p> My next step was to define an extension method on <code>Uri</code>, so that I could turn a URL into a link: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;Link(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">Uri</span>&nbsp;uri,&nbsp;<span style="color:blue;">string</span>&nbsp;rel) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;{&nbsp;Rel&nbsp;=&nbsp;rel,&nbsp;Href&nbsp;=&nbsp;uri.ToString()&nbsp;}; }</pre> </p> <p> With that function I could now write code like this: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;CreateYearLink() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithAction(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithController(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.BuildAbsolute(Url) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.Link(<span style="color:#a31515;">&quot;urn:year&quot;</span>); }</pre> </p> <p> It's acceptable, but verbose. This only creates the <code>urn:year</code> link; to create the <code>urn:month</code> and <code>urn:day</code> links, I needed similar code. Only the <code>WithValues</code> method calls differed. The calls to <code>WithAction</code> and <code>WithController</code> were identical. </p> <h3 id="e003c737cf604f83bd61dfba8045754b"> Shared Immutable Builder <a href="#e003c737cf604f83bd61dfba8045754b" title="permalink">#</a> </h3> <p> Since <code>UrlBuilder</code> is immutable, I can trivially define a shared instance: </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>&nbsp;calendar&nbsp;= &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">UrlBuilder</span>() &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithAction(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.WithController(<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>));</pre> </p> <p> This enabled me to write more succinct methods for each of the relationship types: </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToYear(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:year&quot;</span>); } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToMonth(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year,&nbsp;month&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:month&quot;</span>); } <span style="color:blue;">internal</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">LinkDto</span>&nbsp;LinkToDay(<span style="color:blue;">this</span>&nbsp;<span style="color:#2b91af;">IUrlHelper</span>&nbsp;url,&nbsp;<span style="color:blue;">int</span>&nbsp;year,&nbsp;<span style="color:blue;">int</span>&nbsp;month,&nbsp;<span style="color:blue;">int</span>&nbsp;day) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;calendar.WithValues(<span style="color:blue;">new</span>&nbsp;{&nbsp;year,&nbsp;month,&nbsp;day&nbsp;}).BuildAbsolute(url).Link(<span style="color:#a31515;">&quot;urn:day&quot;</span>); }</pre> </p> <p> This is possible exactly because <code>UrlBuilder</code> is immutable. Had the Builder been mutable, such sharing would have created an <a href="https://en.wikipedia.org/wiki/Aliasing_(computing)">aliasing</a> bug, as I <a href="/2020/02/10/builder-isomorphisms">previously described</a>. Immutability enables reuse. </p> <h3 id="cc70ea76298547e893d883bcb9983f92"> Conclusion <a href="#cc70ea76298547e893d883bcb9983f92" title="permalink">#</a> </h3> <p> I got my first taste of functional programming around 2010. Since then, when I'm not programming in <a href="https://fsharp.org">F#</a> or <a href="https://www.haskell.org">Haskell</a>, I've steadily worked on identifying good ways to enjoy the benefits of functional programming in C#. </p> <p> Immutability is a fairly low-hanging fruit. It requires more boilerplate code, but apart from that, it's easy to make classes immutable in C#, and Visual Studio has plenty of refactorings that make it easier. </p> <p> Immutability is one of those features you're unlikely to realise that you're missing. When it's not there, you work around it, but when it's there, it simplifies many tasks. </p> <p> The example you've seen in this article relates to the Fluent Builder pattern. At first glance, it seems as though a mutable Fluent Builder has the same capabilities as a corresponding Immutable Fluent Builder. You can, however, build upon shared Immutable Fluent Builders, which you can't with mutable Fluent Builders. </p> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Using the nameof C# keyword with ASP.NET 3 IUrlHelper https://blog.ploeh.dk/2020/08/03/using-the-nameof-c-keyword-with-aspnet-3-iurlhelper 2020-08-03T10:01:00+00:00 Mark Seemann <div id="post"> <p> <em>How to generate links to other resources in a refactoring-friendly way.</em> </p> <p> I recently spent a couple of hours yak-shaving, and despite much Googling couldn't find any help on the internet. I'm surprised that the following problem turned out to be so difficult to figure out, so it may just be that I'm ignorant or that my web search skills failed me that day. On the other hand, if this really is as difficult as I found it, perhaps this article can save some other poor soul an hour or two. </p> <p> I was developing a REST API and wanted to generate some links like these: </p> <p> <pre>{ "links": [ { "rel": "urn:reservations", "href": "http://localhost:53568/reservations" }, { "rel": "urn:year", "href": "http://localhost:53568/calendar/2020" }, { "rel": "urn:month", "href": "http://localhost:53568/calendar/2020/7" }, { "rel": "urn:day", "href": "http://localhost:53568/calendar/2020/7/7" } ] }</pre> </p> <p> Like previous incarnations of the framework, ASP.NET Core 3 has <a href="https://docs.microsoft.com/dotnet/api/microsoft.aspnetcore.mvc.urlhelperextensions.action">an API for generating links to a method on a Controller</a>. I just couldn't get it to work. </p> <h3 id="6c7ec5c13e9942098dc18cbcc43b345c"> Using nameof <a href="#6c7ec5c13e9942098dc18cbcc43b345c" title="permalink">#</a> </h3> <p> I wanted to generate a URL like <code>http://localhost:53568/calendar/2020</code> in a refactoring-friendly way. While ASP.NET wants you to define HTTP resources as methods (<em>actions</em>) on <em>Controllers</em>, the various <code>Action</code> overloads want you to identify these actions and Controllers as strings. What happens if someone renames one of those methods or Controller classes? </p> <p> That's what the C# <code>nameof</code> keyword is for. </p> <p> I naively called the <code>Action</code> method like this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;href&nbsp;=&nbsp;Url.Action( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>), &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Host.ToUriComponent());</pre> </p> <p> Looks good, doesn't it? </p> <p> I thought so, but it didn't work. In the time-honoured tradition of mainstream programming languages, the method just silently fails to return a value and instead returns null. That's not helpful. What might be the problem? No clue is provided. It just doesn't work. </p> <h3 id="3f7f01514e5a4c518ddd37226352970e"> Strip off the suffix <a href="#3f7f01514e5a4c518ddd37226352970e" title="permalink">#</a> </h3> <p> It turns out that the <code>Action</code> method expects the <code>controller</code> argument to <em>not</em> contain the <code>Controller</code> suffix. Not surprisingly, <code>nameof(CalendarController)</code> becomes the string <code>"CalendarController"</code>, which doesn't work. </p> <p> It took me some time to figure out that I was supposed to pass a string like <code>"Calendar"</code>. <em>That</em> works! </p> <p> As a first pass at the problem, then, I changed my code to this: </p> <p> <pre><span style="color:blue;">var</span>&nbsp;controllerName&nbsp;=&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>); <span style="color:blue;">var</span>&nbsp;controller&nbsp;=&nbsp;controllerName.Remove( &nbsp;&nbsp;&nbsp;&nbsp;controllerName.LastIndexOf( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#a31515;">&quot;Controller&quot;</span>, &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">StringComparison</span>.Ordinal)); <span style="color:blue;">var</span>&nbsp;href&nbsp;=&nbsp;Url.Action( &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">nameof</span>(<span style="color:#2b91af;">CalendarController</span>.Get), &nbsp;&nbsp;&nbsp;&nbsp;controller, &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;{&nbsp;year&nbsp;=&nbsp;<span style="color:#2b91af;">DateTime</span>.Now.Year&nbsp;}, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Scheme, &nbsp;&nbsp;&nbsp;&nbsp;Url.ActionContext.HttpContext.Request.Host.ToUriComponent());</pre> </p> <p> That also works, and is more refactoring-friendly. You can rename both the Controller class and the method, and the link should still work. </p> <h3 id="8285d8ffd53f44808a899c33b8712bda"> Conclusion <a href="#8285d8ffd53f44808a899c33b8712bda" title="permalink">#</a> </h3> <p> The <code>UrlHelperExtensions.Action</code> methods expect the <code>controller</code> to be the 'semantic' name of the Controller, if you will - not the actual class name. If you're calling it with values produced with the <code>nameof</code> keyword, you'll have to strip the <code>Controller</code> suffix away. </p> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Task asynchronous programming as an IO surrogate https://blog.ploeh.dk/2020/07/27/task-asynchronous-programming-as-an-io-surrogate 2020-07-27T08:27:00+00:00 Mark Seemann <div id="post"> <p> <em>Is task asynchronous programming a substitute for the IO container? An article for C# programmers.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In the previous articles, you've seen how a type like <code>IO&lt;T&gt;</code> can be used to distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. While it's an effective and elegant solution to the problem, it depends on a convention: that all impure actions return <code>IO</code> objects, which are opaque to pure functions. </p> <p> In reality, .NET base class library methods don't do that, and it's unrealistic that this is ever going to happen. It'd require a breaking reset of the entire .NET ecosystem to introduce this design. </p> <p> A comparable reset did, however, happen a few years ago. </p> <h3 id="dce87c53f2fa4b1db4c05b48e8d65870"> TAP reset <a href="#dce87c53f2fa4b1db4c05b48e8d65870" title="permalink">#</a> </h3> <p> Microsoft introduced the <a href="https://docs.microsoft.com/dotnet/csharp/programming-guide/concepts/async">task asynchronous programming</a> (TAP) model some years ago. Operations that involve <a href="https://en.wikipedia.org/wiki/Input/output">I/O</a> got a new return type. Not <code>IO&lt;T&gt;</code>, but <code>Task&lt;T&gt;</code>. </p> <p> The .NET framework team began a long process of adding asynchronous alternatives to existing APIs that involve I/O. Not as breaking changes, but by adding new, asynchronous methods side-by-side with older methods. <a href="https://docs.microsoft.com/dotnet/api/system.data.sqlclient.sqlcommand.executereaderasync">ExecuteReaderAsync</a> as an alternative to <a href="https://docs.microsoft.com/dotnet/api/system.data.sqlclient.sqlcommand.executereader">ExecuteReader</a>, <a href="https://docs.microsoft.com/dotnet/api/system.io.file.readalllinesasync">ReadAllLinesAsync</a> side by side with <a href="https://docs.microsoft.com/dotnet/api/system.io.file.readalllines">ReadAllLines</a>, and so on. </p> <p> Modern APIs exclusively with asynchronous methods appeared. For example, the <a href="https://docs.microsoft.com/dotnet/api/system.net.http.httpclient">HttpClient</a> class only affords asynchronous I/O-based operations. </p> <p> The TAP reset was further strengthened by the move from .NET to .NET Core. Some frameworks, most notably ASP.NET, were redesigned on a fundamentally asynchronous core. </p> <p> In 2020, most I/O operations in .NET are easily recognisable, because they return <code>Task&lt;T&gt;</code>. </p> <h3 id="3e84ba1d588a42689d7e8741315d22b5"> Task as a surrogate IO <a href="#3e84ba1d588a42689d7e8741315d22b5" title="permalink">#</a> </h3> <p> I/O operations are impure. Either you're receiving input from outside the running process, which is consistently non-deterministic, or you're writing to an external resource, which implies a side effect. It might seem natural to think of <code>Task&lt;T&gt;</code> as a replacement for <code>IO&lt;T&gt;</code>. <a href="https://twitter.com/SzymonPobiega/status/713001580077965312">Szymon Pobiega had a similar idea in 2016</a>, and I <a href="/2016/04/11/async-as-surrogate-io">investigated his idea in an article</a>. This was based on <a href="https://fsharp.org">F#</a>'s <code>Async&lt;'a&gt;</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a>, which is <a href="/2018/09/24/asynchronous-functors">equivalent to <code>Task&lt;T&gt;</code></a> - except when it comes to <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>. </p> <p> Unfortunately, <code>Task&lt;T&gt;</code> is far from a perfect replacement of <code>IO&lt;T&gt;</code>, because the .NET base class library (BCL) still contains plenty of impure actions that 'look' pure. Examples include <a href="https://docs.microsoft.com/dotnet/api/system.console.writeline">Console.WriteLine</a>, the parameterless <a href="https://docs.microsoft.com/dotnet/api/system.random.-ctor">Random constructor</a>, <a href="https://docs.microsoft.com/dotnet/api/system.guid.newguid">Guid.NewGuid</a>, and <a href="https://docs.microsoft.com/dotnet/api/system.datetime.now">DateTime.Now</a> (arguably a candidate for the worst-designed API in the BCL). None of those methods return tasks, which they ought to if tasks should serve as easily recognisable signifiers of impurity. </p> <p> Still, you could write asynchronous <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapters</a> over such APIs. Your <code>Console</code> Adapter might present this API: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Console</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:blue;">string</span>&gt;&nbsp;ReadLine(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;WriteLine(<span style="color:blue;">string</span>&nbsp;value); }</pre> </p> <p> Moreover, the <code>Clock</code> API might look like this: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">Clock</span> { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">Task</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;GetLocalTime(); }</pre> </p> <p> Modern versions of C# enable you to write asynchronous <a href="https://en.wikipedia.org/wiki/Entry_point">entry points</a>, so the <a href="https://en.wikipedia.org/wiki/%22Hello,_World!%22_program">hello world</a> example shown in this article series becomes: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:blue;">async</span>&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;What&#39;s&nbsp;your&nbsp;name?&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;name&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.ReadLine(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;now&nbsp;=&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#2b91af;">Greeter</span>.Greet(now,&nbsp;name); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">await</span>&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(greeting); }</pre> </p> <p> That's nice <a href="/2015/08/03/idiomatic-or-idiosyncratic">idiomatic</a> C# code, so what's not to like? </p> <h3 id="8a11bbfe9cbc4e72bd90ea6b787c0037"> No referential transparency <a href="#8a11bbfe9cbc4e72bd90ea6b787c0037" title="permalink">#</a> </h3> <p> The above <code>Main</code> example is probably as good as it's going to get in C#. I've nothing against that style of C# programming, but you shouldn't believe that this gives you compile-time checking of referential transparency. It doesn't. </p> <p> Consider a simple function like this, written using the <code>IO</code> container shown in previous articles: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;No,&nbsp;I&#39;m&nbsp;not.&quot;</span>; }</pre> </p> <p> Is this method referentially transparent? Surprisingly, despite the apparent side effect, it is. The reason becomes clearer if you write the code so that it explicitly ignores the return value: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;No,&nbsp;I&#39;m&nbsp;not.&quot;</span>; }</pre> </p> <p> The <code>Console.WriteLine</code> method returns an object that represents a computation that might take place. This <code>IO&lt;Unit&gt;</code> object, however, never escapes the method, and thus never runs. The side effect never takes place, which means that the method is referentially transparent. You can replace <code>AmIEvil()</code> with its return value <code>"No, I'm not."</code>, and your program would behave exactly the same. </p> <p> Consider what happens when you replace <code>IO</code> with <code>Task</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;AmIEvil() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Task</span>&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">&quot;Yes,&nbsp;I&nbsp;am.&quot;</span>; }</pre> </p> <p> Is this method a pure function? No, it's not. The problem is that the most common way that .NET libraries return tasks is that the task is already running when it's returned. This is also the case here. As soon as you call this version of <code>Console.WriteLine</code>, the task starts running on a background thread. Even though you ignore the task and return a plain <code>string</code>, the side effect sooner or later takes place. You can't replace a call to <code>AmIEvil()</code> with its return value. If you did, the side effect wouldn't happen, and that would change the behaviour of your program. </p> <p> Contrary to <code>IO</code>, tasks don't guarantee referential transparency. </p> <h3 id="aff1e4cb382b494ea0624ef63311da81"> Conclusion <a href="#aff1e4cb382b494ea0624ef63311da81" title="permalink">#</a> </h3> <p> While it'd be technically possible to make C# distinguish between pure and impure code at compile time, it'd require such a breaking reset to the entire .NET ecosystem that it's unrealistic to hope for. It seems, though, that there's enough overlap with the design of <code>IO&lt;T&gt;</code> and task asynchronous programming that the latter might fill that role. </p> <p> Unfortunately it doesn't, because it fails to guarantee referential transparency. It's better than nothing, though. Most C# programmers have now learned that while <code>Task</code> objects come with a <a href="https://docs.microsoft.com/dotnet/api/system.threading.tasks.task-1.result">Result</a> property, you shouldn't use it. Instead, you should write your entire program using <code>async</code> and <code>await</code>. That, at least, takes you halfway towards where you want to be. </p> <p> The compiler, on the other hand, doesn't help you when it comes to those impure actions that look pure. Neither does it protect you against asynchronous side effects. Diligence, code reviews, and programming discipline are still required if you want to separate pure functions from impure actions. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="aeb6990f553842739915d980119a48d9"> <div class="comment-author">Matt Heuer</div> <div class="comment-content"> <p> This is a great idea. It seems like the only Problem with Tasks is that they are usually already started, either on the current or a Worker Thread. If we return Tasks that are not started yet, then Side-Effects don't happen until we await them. And we have to await them to get their Result or use them in other Tasks. I experimented with a modified GetTime() Method returning a new Task that is not run yet: </p> <pre> public static Task<DateTime> GetTime() => new Task<DateTime>(() => DateTime.Now); </pre> <p> Using a SelectMany Method that ensures that Tasks have been run, the Time is not evaluated until the resulting Task is awaited or another SelectMany is built using the Task from the first SelectMany. The Time of one such Task is also evaluated only once. On repeating Calls the same Result is returned: </p> <pre> public static async Task<TResult> SelectMany<T, TResult>(this Task<T> source , Func<T, Task<TResult>> project) { T t = await source.GetResultAsync(); Task<TResult>? ret = project(t); return await ret.GetResultAsync(); } /// <summary> Asynchronously runs the <paramref name="task"/> to Completion </summary> /// <returns><see cref="Task{T}.Result"/></returns> public static async Task<T> GetResultAsync<T>(this Task<T> task) { switch (task.Status) { case TaskStatus.Created: await Task.Run(task.RunSynchronously); return task.Result; case TaskStatus.WaitingForActivation: case TaskStatus.WaitingToRun: case TaskStatus.Running: case TaskStatus.WaitingForChildrenToComplete: return await task; case TaskStatus.Canceled: case TaskStatus.Faulted: case TaskStatus.RanToCompletion: return task.Result; //let the Exceptions fly... default: throw new ArgumentOutOfRangeException(); } } </pre> <p> Since I/O Operations with side-effects are usually asynchronous anyway, Tasks and I/O are a good match. <br/> Consistenly not starting Tasks and using this SelectMany Method either ensures Method purity or enforces to return the Task. To avoid ambiguity with started Tasks a Wrapper-IO-Class could be constructed, that always takes and creates unstarted Tasks. Am I missing something or do you think this would not be worth the effort? Are there more idiomatic ways to start enforcing purity in C#, except e.g. using the [Pure] Attribute and StyleCop-Warnings for unused Return Values? </p> </div> <div class="comment-date">2021-02-27 22:25 UTC</div> </div> <div class="comment" id="35f7c2e8cdc84a89b79771f541530025"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Matt, thank you for writing. That's essentially <a href="/2016/04/11/async-as-surrogate-io#093a7032a754405a80bc4382d1e2cab9">how F# asynchronous workflows work</a>. </p> </div> <div class="comment-date">2021-03-03 9:05 UTC</div> </div> </div> <hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Closing database connections during test teardown https://blog.ploeh.dk/2020/07/20/closing-database-connections-during-test-teardown 2020-07-20T07:20:00+00:00 Mark Seemann <div id="post"> <p> <em>How to close database connections to SQL Server during integration testing teardown.</em> </p> <p> Whenever I need to run integration tests that involve SQL Server, I have a standard approach that <a href="https://docs.microsoft.com/en-us/archive/blogs/ploeh/sqlexpressdatabaseinstaller">I've evolved since 2007</a>. It involves <ol> <li>setting up a SQL Server Express database before each test</li> <li>running the test</li> <li>tearing down the database</li> </ol> I've explained the basic process in details in my <a href="/outside-in-tdd">Outside-In Test-Driven Development</a> Pluralsight course. You can also see it in action in <a href="https://github.com/ploeh/dependency-injection-revisited">this GitHub repository</a>. </p> <p> One problem with that approach is that SQL Server doesn't allow you to delete a database if it has existing connections. </p> <h3 id="7de0c4f8bd3d40e99740a39b11570593"> Turn off connection pooling <a href="#7de0c4f8bd3d40e99740a39b11570593" title="permalink">#</a> </h3> <p> I usually solve the problem by turning off connection pooling. For an integration test suite, this is fine. I usually use integration testing to verify functionality - not performance. </p> <p> Turning off connection pooling is easily done by setting the flag to <code>false</code> in the connection string: </p> <p> <code>Server=(LocalDB)\MSSQLLocalDB;Database=Booking;Integrated Security=true;Pooling=false</code> </p> <p> This means that when you get to the teardown phase of the test, you can issue a DDL statement to the <code>master</code> database: </p> <p> <pre>IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'Booking') DROP DATABASE [Booking]</pre> </p> <p> When connection pooling is turned off, no other connections are open when you attempt to do that, and the database (here named <code>Booking</code>) is deleted. </p> <h3 id="b599a9242a7d44f1b23241341db054b8"> Forcibly close other connections <a href="#b599a9242a7d44f1b23241341db054b8" title="permalink">#</a> </h3> <p> Recently, however, I ran into a testing scenario where connection pooling had to be turned on. When you turn on connection pooling, however, the above <code>DROP DATABASE</code> statement fails because at least one connection from the pool is still connected to the database. </p> <p> To solve that issue, I forcibly close other connections during teardown: </p> <p> <pre>IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = N'Booking') BEGIN -- This closes existing connections: ALTER DATABASE [Booking] SET SINGLE_USER WITH ROLLBACK IMMEDIATE DROP DATABASE [Booking] END</pre> </p> <p> Surprisingly, turning on connection pooling like this makes the integration tests slower. I suppose it's because throwing other connections off the database involves a bit of negotiation between the server and the clients, and that takes some time. </p> <p> While slower, it does enable you to run the integration tests with connection pooling turned on. When you need it, you need it. </p> <h3 id="6a3cf6b38c0f49a3abb1aa8ccbf50644"> Summary <a href="#6a3cf6b38c0f49a3abb1aa8ccbf50644" title="permalink">#</a> </h3> <p> You can run integration tests against a SQL Server Express database. People do it in various ways, but I've found that setting up and tearing down a pristine database for each test case is a robust and maintainable solution to the problem. </p> <p> SQL Server will not, however, allow you to delete a database if other connections exist. The easiest and fastest solution to that problem is to turn off connection pooling. </p> <p> Sometimes, you can't do that, so instead, you can expand your database teardown script so that it closes existing connections before it deletes the database. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="0a9ea1fffa1c41ae96981ff4625c4985"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> This sounds like a great approach. I have been on projects with tests that involved the database, but none of them were designed as well as this. I will be sure to come back to this post when we add a database to my current project. </p> <p> My understanding is that <a href="https://expressdb.io/sql-server-express-vs-localdb.html#what-is-localdb">SQL Server Express and LocalDB are not the same thing</a>. Are you using SQL Server Express or LocalDB? Do you prefer one over the other for this database testing approach of yours? </p> </div> <div class="comment-date">2020-07-25 19:58 UTC</div> </div> <div class="comment" id="4fd6901499a44fc084d58673aa762cc1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. It's not really my area of expertise. I use the one bundled with Visual Studio, so I suppose that's actually LocalDB, and not SQL Server Express. </div> <div class="comment-date">2020-07-26 13:33 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Implementation of the C# IO container https://blog.ploeh.dk/2020/07/13/implementation-of-the-c-io-container 2020-07-13T06:02:00+00:00 Mark Seemann <div id="post"> <p> <em>Implementation details of the C# IO container.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In the previous articles, you've seen how a type like <code>IO&lt;T&gt;</code> can be used to distinguish between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. </p> <p> The point of the article series is to illustrate the concept that <a href="https://www.haskell.org">Haskell</a> uses to impose the <a href="/2018/11/19/functional-architecture-a-definition">functional interaction law</a> at compile time. The implementation details really aren't important. Still, I believe that I know my readership well enough that a substantial fraction would be left unsatisfied if I didn't share the implementation details. </p> <p> I consider this an appendix to the article series. It's not really what this is all about, but here it is, nonetheless. </p> <h3 id="dcf2521a1b8c40df9d45e97a43cfc546"> Constructor <a href="#dcf2521a1b8c40df9d45e97a43cfc546" title="permalink">#</a> </h3> <p> Based on the public API <a href="/2020/06/15/io-container-in-a-parallel-c-universe">already published</a>, the constructor implementation hardly comes as a surprise. </p> <p> <pre><span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; <span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; }</pre> </p> <p> The <code>IO&lt;T&gt;</code> class is little more than a wrapper around a lazily evaluated function, with the restriction that while you can put a <code>Func&lt;T&gt;</code> object into an <code>IO</code> object, you can never get it out again. Thus, the <code>item</code> is a <code>private</code> class field instead of a public property. </p> <h3 id="50d90cc423f04f5bb4e31a0d6c9b12c8"> SelectMany <a href="#50d90cc423f04f5bb4e31a0d6c9b12c8" title="permalink">#</a> </h3> <p> The <code>SelectMany</code> method is a little more tricky: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;selector(item()).item()); }</pre> </p> <p> To guarantee <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>, we don't want the method to trigger evaluation of the lazy value, so the <code>selector</code> has to run inside a new lazy computation. This produces a lazy <code>IO</code> value that the method then has to unwrap inside another lazy computation. Such a translation from <code>Func&lt;IO&lt;TResult&gt;&gt;</code> to a new <code>IO</code> object with a <code>Func&lt;TResult&gt;</code> inside it is reminiscent of what in Haskell is known as a <em>traversal</em>. </p> <h3 id="b6c088710f6340689376b0ddf3b7df01"> UnsafePerformIO <a href="#b6c088710f6340689376b0ddf3b7df01" title="permalink">#</a> </h3> <p> Finally, the <code>UnsafePerformIO</code> method isn't part of the API, but as explained in the <a href="/2020/07/06/referential-transparency-of-io">previous article</a>, this is the special method that the hypothetical parallel-world framework calls on the <code>IO&lt;Unit&gt;</code> returned by <code>Main</code> methods. </p> <p> <pre><span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item(); }</pre> </p> <p> Since only the framework is supposed to call this method, it's <code>internal</code> by design. The only thing it does is to force evaluation of the lazy <code>item</code>. </p> <h3 id="38830b00fbbf4911b8a7ebd723e138ea"> Conclusion <a href="#38830b00fbbf4911b8a7ebd723e138ea" title="permalink">#</a> </h3> <p> Most of the implementation of <code>IO&lt;T&gt;</code> is straightforward, with the single exception of <code>SelectMany</code>, which has to jump through a few hoops to keep the behaviour lazy until it's activated. </p> <p> Once more I want to point out that the purpose of this article series is to explain how a type system like Haskell's guarantees referential transparency. C# could do the same, but it'd require that all impure actions in all libraries in the entire .NET ecosystem were to return <code>IO</code> values. That's not going to happen, but something similar already has happened. Read on in the next article. </p> <p> <strong>Next:</strong> <a href="/2020/07/27/task-asynchronous-programming-as-an-io-surrogate">Task asynchronous programming as an IO surrogate</a>. </p> </div> <div id="comments"> <hr> <h2 id="comments-header"> Comments </h2> <div class="comment" id="8fea4b2ac22d4211bd91d352c7b4e55e"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <p> In your <a href="https://blog.ploeh.dk/2020/07/13/implementation-of-the-c-io-container/">previous post</a>, you said </p> <blockquote> Haskell is a lazily evaluated language. That's an important piece of the puzzle, so I've modelled the <code>IO&lt;T&gt;</code> example so that it only wraps <code>Lazy</code> values. That emulates Haskell's behaviour in C#. </blockquote> <p> After several days, I finally feel like I fully understand this. </p> <p> The concept of lazy has serveral slightly different definitions depending on the context. Haskell is a lazily evaluated language in the sense that its evaluation strategy is <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_need">call by need</a>. In C#, both <code>Lazy&lt;T&gt;</code> and <code>Func&lt;T&gt;</code> are lazy in the sense that neither actually contains a <code>T</code>, but both could produce a <code>T</code> if asked to do so. The difference is the presence or absence of caching. I remember all this by saying that <code>Lazy&lt;T&gt;</code> is lazy with caching and <code>Func&lt;T&gt;</code> is lazy without caching. So <code>Lazy&lt;T&gt;</code> is to call by need as <code>Func&lt;T&gt;</code> is to <a href="https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name">call by name</a>. </p> <p> Therefore, <code>Lazy&lt;T&gt;</code> is the correct choice if we want to model or emulate the evaluation strategy of Haskell in C#. What about Haskell's <code>IO&lt;T&gt;</code>? Is it lazy with caching or lazy without caching? My guess was lazy without caching, but I finally installed the ghc Haskell compiler and compiled Haskell on my machine for the first time in order to test this. I think this example shows that Haskell's <code>IO&lt;T&gt;</code> is lazy without caching. </p> <p> <pre>-- output: xx main = let io = putStr "x" in do { io ; io }</pre> </p> <p> I think this would be equivalent C# code in this parallel world that you have created. </p> <p> <pre>// output: x static IO<Unit> MainIO(string[] args) { var io = Console.Write("x"); return from _1 in io from _2 in io select Unit.Instance; }</pre> </p> <p> What makes me think that I fully understand this now is that I think I see where you are going. I think you already knew all this and decided to model Haskell's <code>IO&lt;T&gt;</code> using <code>Lazy&lt;T&gt;</code> anyway because <code>Task&lt;T&gt;</code> is also lazy with caching just like <code>Lazy&lt;T&gt;</code>, and your next post will discuss using <code>Task&lt;T&gt;</code> as a surrogate for Haskell's <code>IO&lt;T&gt;</code>. I think you want your C# implementation of <code>IO&lt;T&gt;</code> to be more like C#'s <code>Task&lt;T&gt;</code> than Haskell's <code>IO&lt;T&gt;</code>. </p> <p> Thank you for including such a gem for me to think about...and enough motivation to finally put Haskell on my machine! </p> </div> <div class="comment-date">2020-07-13 20:46 UTC</div> </div> <div class="comment" id="ba8de284a15c49529bb10390c33d2aa1"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> Tyson, thank you for writing. You've almost turned my blog into a peer-reviewed journal, and you've just pointed out a major blunder of mine 👍 </p> <p> I think I was mislead by the name <code>Lazy</code>, my attention was elsewhere, and I completely forgot about the memoisation that both <code>Lazy&lt;T&gt;</code> and <code>Task&lt;T&gt;</code> employ. It does turn out to be problematic in this context. Take the following example: </p> <p> <pre><span style="color:blue;">static</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;Main(<span style="color:blue;">string</span>[]&nbsp;args) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;getTime&nbsp;=&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getTime.SelectMany(t1&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(t1.Ticks.ToString()).Select(u1&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Thread</span>.Sleep(2); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#2b91af;">Unit</span>.Instance; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}).SelectMany(u2&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;getTime).SelectMany(t2&nbsp;=&gt; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(t2.Ticks.ToString()))); }</pre> </p> <p> Notice that this example <em>reuses</em> <code>getTime</code> twice. We'd like any <code>IO&lt;T&gt;</code> value to represent an impure computation, so evaluating it twice with a 2 millisecond delay in between ought to yield two different results. </p> <p> Due to the memoisation built into <code>Lazy&lt;T&gt;</code>, the first value is reused. That's not the behaviour we'd like to see. </p> <p> While this is a rather big error on my part, it's fortunately only of a technical nature (I think). As you imply, the correct implementation is to use <code>Func&lt;T&gt;</code> rather than <code>Lazy&lt;T&gt;</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;selector(item()).item()); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item(); &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> This addresses the above reuse bug. With this implementation, the above <code>Main</code> method prints two different values, even though it reuses <code>getTime</code>. </p> <p> Haskell <code>IO</code> doesn't memoise values, so this <code>Func</code>-based implementation better emulates the Haskell behaviour, which is actually what I wanted to do all along. </p> <p> This mistake of mine is potentially so confusing that I think that it's best if I go back and edit the other posts in this articles series. Before I do that, though, I'd like to get your feedback. </p> <p> Does this look better to you, or do you see other problems with it? </p> </div> <div class="comment-date">2020-07-19 14:32 UTC</div> </div> <div class="comment" id="f14bcdf1d46245278ef8694fd6b73d3b"> <div class="comment-author"><a href="https://about.me/tysonwilliams">Tyson Williams</a></div> <div class="comment-content"> <blockquote> Tyson, thank you for writing. You've almost turned my blog into a peer-reviewed journal, and you've just pointed out a major blunder of mine 👍 </blockquote> <p> You're welcome. I am certainly a peer, and I benefit greatly from closly reviewing your posts. They always give me so much to think about. I am happy that we both benefit from this :) </p> <blockquote> <p> While this is a rather big error on my part, it's fortunately only of a technical nature (I think). ... </p> <p> Haskell <code>IO</code> doesn't memoise values, so this <code>Func</code>-based implementation better emulates the Haskell behaviour, which is actually what I wanted to do all along. </p> <p> This mistake of mine is potentially so confusing that I think that it's best if I go back and edit the other posts in this articles series. Before I do that, though, I'd like to get your feedback. </p> <p> Does this look better to you, or do you see other problems with it? </p> </blockquote> <p> Yes, your <code>Func&lt;T&gt;</code>-based implementation better emulates Haskell's <code>IO&lt;T&gt;</code>. My guess was that you had used <code>Lazy&lt;T&gt;</code> with its caching behavior in mind. I do think it is a minor issue. I can't think of any code that I would write on purpose that would depend on this difference. </p> <p> I think editing the previous posts depends on exactly how you want to suggesst <code>Task&lt;T&gt;</code> as an <code>IO&lt;T&gt;</code> surrogate. </p> <p> From a purely teaching perspective, I think I prefer to first implement Haskell's <code>IO&lt;T&gt;</code> in C# using <code>Func&lt;T&gt;</code>, then suggest this implemention is essentialy the same as <code>Task&lt;T&gt;</code>, then point out the caching difference for those that are still reading. It would be a shame to lose some readers eariler by pointing out the difference too soon. I wouldn't expect you lost any readres in your current presentation that includes the caching difference but without mentioning it. </p> <p> <code>Func&lt;T&gt;</code>, <code>Lazy&lt;T&gt;</code>, and <code>Task&lt;T&gt;</code> are all lazy (in the sense that none contain a <code>T</code> but all could produce one if requested. Here are their differences: <ul> <li><code>Func&lt;T&gt;</code> is synchronous without caching,</li> <li><code>Lazy&lt;T&gt;</code> is synchronous with caching, and</li> <li><code>Task&lt;T&gt;</code> is asynchronous with caching.</li> </ul> We are missing a type that is asynchronous without caching. We can create such behavior though using <code>Func&lt;T&gt;</code> and <code>Task&lt;T&gt;</code>. The nested type <code>Func&lt;Task&lt;T&gt;&gt;</code> can asynchronously produce a <code>T</code> without caching. Maybe this would be helpful in this article series. </p> <p> Overall though, I don't know of any other potential changes to consider. </p> </div> <div class="comment-date">2020-07-19 21:19 UTC</div> </div> <div class="comment" id="689c56123f7b469a9796e0793f48b294"> <div class="comment-author"><a href="/">Mark Seemann</a></div> <div class="comment-content"> <p> For any reader following the discussion after today (July 24, 2020), it may be slightly confusing. Based on <a href="#8fea4b2ac22d4211bd91d352c7b4e55e">Tyson Williams' feedback</a>, I've edited the article series with <a href="#ba8de284a15c49529bb10390c33d2aa1">the above implementation</a>. The previous incarnation of the article series had this implementation of <code>IO&lt;T&gt;</code>: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">sealed</span>&nbsp;<span style="color:blue;">class</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">T</span>&gt; { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">private</span>&nbsp;<span style="color:blue;">readonly</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>(<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">T</span>&gt;&nbsp;item) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">this</span>.item&nbsp;=&nbsp;item; &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">public</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&nbsp;SelectMany&lt;<span style="color:#2b91af;">TResult</span>&gt;(<span style="color:#2b91af;">Func</span>&lt;<span style="color:#2b91af;">T</span>,&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;&nbsp;selector) &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;res&nbsp;=&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;&gt;(()&nbsp;=&gt;&nbsp;selector(item.Value)); &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;( &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">new</span>&nbsp;<span style="color:#2b91af;">Lazy</span>&lt;<span style="color:#2b91af;">TResult</span>&gt;(()&nbsp;=&gt;&nbsp;res.Value.item.Value)); &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">internal</span>&nbsp;<span style="color:#2b91af;">T</span>&nbsp;UnsafePerformIO() &nbsp;&nbsp;&nbsp;&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;item.Value; &nbsp;&nbsp;&nbsp;&nbsp;} }</pre> </p> <p> As this discussion reveals, the memoisation performed by <code>Lazy&lt;T&gt;</code> causes problems. After thinking it through, I've decided to retroactively change the articles in the series. This is something that I rarely do, but in this case I think is the best way forward, as the <code>Lazy</code>-based implementation could be confusing to new readers. </p> <p> Readers interested in the history of these articles can peruse <a href="https://github.com/ploeh/ploeh.github.com">the Git log</a>. </p> </div> <div class="comment-date">2020-07-24 8:22 UTC</div> </div> </div><hr> This blog is totally free, but if you like it, please consider <a href="https://blog.ploeh.dk/support">supporting it</a>. Referential transparency of IO https://blog.ploeh.dk/2020/07/06/referential-transparency-of-io 2020-07-06T05:56:00+00:00 Mark Seemann <div id="post"> <p> <em>How the IO container guarantees referential integrity. An article for object-oriented programmers.</em> </p> <p> This article is part of <a href="/2020/06/08/the-io-container">an article series about the IO container in C#</a>. In <a href="/2020/06/15/io-container-in-a-parallel-c-universe">a previous article</a> you got a basic introduction to the <code>IO&lt;T&gt;</code> <a href="https://bartoszmilewski.com/2014/01/14/functors-are-containers">container</a> I use to explain how a type system like <a href="https://www.haskell.org">Haskell</a>'s distinguishes between <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a> and impure actions. </p> <p> The whole point of the <code>IO</code> container is to effectuate the <a href="/2018/11/19/functional-architecture-a-definition">functional interaction law</a>: <em>a pure function mustn't be able to invoke an impure activity.</em> This rule follows from <a href="https://en.wikipedia.org/wiki/Referential_transparency">referential transparency</a>. </p> <p> The practical way to think about it is to consider the two rules of pure functions: <ul> <li>Pure functions must be deterministic</li> <li>Pure functions may have no side effects</li> </ul> In this article, you'll see how <code>IO&lt;T&gt;</code> imposes those rules. </p> <h3 id="3b648ec5b32a42e4b608dbc2e8861a86"> Determinism <a href="#3b648ec5b32a42e4b608dbc2e8861a86" title="permalink">#</a> </h3> <p> Like in the previous articles in this series, you must imagine that you're living in a parallel universe where all impure library functions return <code>IO&lt;T&gt;</code> objects. By elimination, then, methods that return naked values must be pure functions. </p> <p> Consider the <code>Greet</code> function from the <a href="/2020/06/15/io-container-in-a-parallel-c-universe">previous article</a>. Since it returns a plain <code>string</code>, you can infer that it must be a pure function. What prevents it from doing something impure? </p> <p> What if you thought that passing <code>now</code> as an argument is a silly design. Couldn't you just call <code>Clock.GetLocalTime</code> from the method? </p> <p> Well, yes, in fact you can: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">DateTime</span>&gt;&nbsp;now1&nbsp;=&nbsp;<span style="color:#2b91af;">Clock</span>.GetLocalTime(); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> This compiles, but is only the first refactoring step you have in mind. Next, you want to extract the <code>DateTime</code> from <code>now1</code> so that you can get rid of the <code>now</code> parameter. Alas, you now run into an insuperable barrier. <a href="/2019/02/04/how-to-get-the-value-out-of-the-monad">How do you get the DateTime out of the IO?</a> </p> <p> You can't. By design. </p> <p> While you can call the <code>GetLocalTime</code> method, you can't <em>use</em> the return value. The only way you can use it is by composing it with <code>SelectMany</code>, but that still accomplishes nothing unless you return the resulting <code>IO</code> object. If you do that, though, you've now turned the entire <code>Greet</code> method into an impure action. </p> <p> You can't perform any non-deterministic behaviour inside a pure function. </p> <h3 id="9f62e432e0c74fda93aff427774c0183"> Side effects <a href="#9f62e432e0c74fda93aff427774c0183" title="permalink">#</a> </h3> <p> How does <code>IO&lt;T&gt;</code> protect against side effects? In the <code>Greet</code> method, couldn't you just write to the console, like the following? </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsAfternoon(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;afternoon&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsEvening(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;evening&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(<span style="color:blue;">string</span>.IsNullOrWhiteSpace(name)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">.&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">return</span>&nbsp;<span style="color:#a31515;">$&quot;</span>{greeting}<span style="color:#a31515;">,&nbsp;</span>{name.Trim()}<span style="color:#a31515;">.&quot;</span>; }</pre> </p> <p> This also compiles, despite our best efforts. That's unfortunate, but, as you'll see in a moment, it doesn't violate referential transparency. </p> <p> In Haskell or <a href="https://fsharp.org">F#</a> equivalent code would make the compiler complain. Those compilers don't have special knowledge about <code>IO</code>, but they can see that an action returns a value. F# generates a compiler warning if you ignore a return value. In Haskell the story is a bit different, but the result is the same. Those compilers complain because you try to ignore the return value. </p> <p> You can get around the issue using the language's wildcard pattern. This tells the compiler that you're actively ignoring the result of the action. You can do the same in C#: </p> <p> <pre><span style="color:blue;">public</span>&nbsp;<span style="color:blue;">static</span>&nbsp;<span style="color:blue;">string</span>&nbsp;Greet(<span style="color:#2b91af;">DateTime</span>&nbsp;now,&nbsp;<span style="color:blue;">string</span>&nbsp;name) { &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:#2b91af;">IO</span>&lt;<span style="color:#2b91af;">Unit</span>&gt;&nbsp;_&nbsp;=&nbsp;<span style="color:#2b91af;">Console</span>.WriteLine(<span style="color:#a31515;">&quot;Side&nbsp;effect!&quot;</span>); &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">var</span>&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Hello&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<span style="color:blue;">if</span>&nbsp;(IsMorning(now)) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;greeting&nbsp;=&nbsp;<span style="color:#a31515;">&quot;Good&nbsp;morning&quot;</span>; &nbsp;&nbsp;&nbsp;&nbsp;<spa