Decoupling decisions from effects by Mark Seemann
Functional programming emphasises pure functions, but sometimes decisions must be made based on impure data. The solution is to decouple decisions and effects.
Functional programmers love pure functions. Not only do they tend to be easy to reason about, they are also intrinsically testable. It'd be wonderful if we could build entire systems only from pure functions, but every functional programmer knows that the world is impure. Instead, we strive towards implementing as much of our code base as pure functions, so that an application is impure only at its boundaries.
The more you can do this, the more testable the system becomes. One rule of thumb about unit testing that I often use is that if a particular candidate for unit testing has a cyclomatic complexity of 1, it may be acceptable to skip unit testing it. Instead, we can consider such a unit a humble unit. If you can separate decisions from effects (which is what functional programmers often call impurities), you can often make the impure functions humble.
In other words: put all logic in pure functions that can be unit tested, and implement impure effects as humble functions that you don't need to unit test.
You want to see an example. So do I!
Example: conditional reading from file #
In a recent discussion, Jamie Cansdale asks how I'd design and unit test something like the following C# method if I could instead redesign it in F#.
public static string GetUpperText(string path) { if (!File.Exists(path)) return "DEFAULT"; var text = File.ReadAllText(path); return text.ToUpperInvariant(); }
Notice how this method contains two impure operations: File.Exists
and File.ReadAllText
. Decision logic seems interleaved with IO. How can decisions be separated from effects?
(For good measure I ought to point out that obviously, the above example is so simple that by itself, it almost doesn't warrant testing. Think of it as a stand-in for a more complex problem.)
With a statement-based language like C#, it can be difficult to see how to separate decision logic from effects without introducing interfaces, but with expression-based languages like F#, it becomes close to trivial. In this article, I'll show you three alternatives.
All three alternatives, however, make use of the same function for turning text into upper case:
// string -> string let getUpper (text : string) = text.ToUpperInvariant ()
Obviously, this function is so trivial that it's hardly worth testing, but remember to think about it as a stand-in for a more complex problem. It's a pure function, so it's easy to unit test:
[<Theory>] [<InlineData("foo", "FOO")>] [<InlineData("bar", "BAR")>] let ``getUpper returns correct value`` input expected = let actual = getUpper input expected =! actual
This test uses xUnit.net 2.1.0 and Unquote 3.1.2. The =!
operator is a custom Unquote operator; you can read it as must equal; that is: expected must equal actual. It'll throw an exception if this isn't the case.
Custom unions #
Languages like F# come with algebraic data types, which means that in addition to complex structures, they also enable you to express types as alternatives. This means that you can represent a decision as one or more alternative pure values.
Although the examples you'll see later in this article are simpler, I think it'll be helpful to start with an ad hoc solution to the problem. Here, the decision is to either read from a file, or return a default value. You can express that using a custom discriminated union:
type Action = ReadFromFile of string | UseDefault of string
This type models two mutually exclusive cases: either you decide to read from the file identified by a file path (string
), or your return a default value (also modelled as a string
).
Using this Action
type, you can write a pure function that makes the decision:
// string -> bool -> Action let decide path fileExists = if fileExists then ReadFromFile path else UseDefault "DEFAULT"
This function takes two arguments: path
(a string
) and fileExists
(a bool
). If fileExists
is true
, it returns the ReadFromFile
case; otherwise, it returns the UseDefault
case.
Notice how this function neither checks whether the file exists, nor does it attempt to read the contents of the file. It only makes a decision based on input, and returns information about this decision as output. This function is pure, so (as I've claimed numerous times) is easy to unit test:
[<Theory>] [<InlineData("ploeh.txt")>] [<InlineData("fnaah.txt")>] let ``decide returns correct result when file exists`` path = let actual = decide path true ReadFromFile path =! actual [<Theory>] [<InlineData("ploeh.txt")>] [<InlineData("fnaah.txt")>] let ``decide returns correct result when file doesn't exist`` path = let actual = decide path false UseDefault "DEFAULT" =! actual
One unit test function exercises the code path where the file exists, whereas the other test exercises the code path where it doesn't. Straightforward.
There's still some remaining work, because you need to somehow compose your pure functions with File.Exists
and File.ReadAllText
. You also need a way to extract the string value from the two cases of Action
. One way to do that is to introduce another pure function:
// (string -> string) -> Action -> string let getValue f = function | ReadFromFile path -> f path | UseDefault value -> value
This is a function that returns the UseDefault
data for that case, but invokes a function f
in the ReadFromFile
case. Again, since this function is pure it's easy to unit test it, but I'll leave that as an exercise.
You now have all the building blocks required to compose a function similar to the above GetUpperText
C# method:
// string -> string let getUpperText path = path |> File.Exists |> decide path |> getValue (File.ReadAllText >> getUpper)
This implementation pipes path
into File.Exists
, which returns a Boolean value indicating whether the file exists. This Boolean value is then piped into decide path
, which (as you may recall) returns an Action
. That value is finally piped into getValue (File.ReadAllText >> getUpper)
. Recall that getValue
will only invoke the function argument when the Action
is ReadFromFile
, so File.ReadAllText >> getUpper
is only executed in this case.
Notice how decisions and effectful functions are interleaved. All the decision functions are covered by unit tests; only File.Exists
and File.ReadAllText
aren't covered, but I find it reasonable to treat these as humble functions.
Either #
Normally, decisions often involve a choice between two alternatives. In the above example, you saw how the alternatives were named ReadFromFile
and UseDefault
. Since a choice between two alternatives is so common, there's a well-known 'pattern' that gives you general-purpose tools to model decisions. This is known as the Either monad.
The F# core library doesn't (yet) come with an implementation of the Either monad, but it's easy to add. In this example, I'm using code from Scott Wlaschin's railway-oriented programming, although slightly modified, and including only the most essential building blocks for the example:
type Result<'TSuccess, 'TFailure> = | Success of 'TSuccess | Failure of 'TFailure module Result = // ('a -> Result<'b, 'c>) -> Result<'a, 'c> -> Result<'b, 'c> let bind f = function | Success succ -> f succ | Failure fail -> Failure fail // ('a -> 'b) -> Result<'a, 'c> -> Result<'b, 'c> let map f = function | Success succ -> Success (f succ) | Failure fail -> Failure fail // ('a -> bool) -> 'a -> Result<'a, 'a> let split f x = if f x then Success x else Failure x // ('a -> 'b) -> ('c -> 'b) -> Result<'a, 'c> -> 'b let either f g = function | Success succ -> f succ | Failure fail -> g fail
In fact, the bind
and map
functions aren't even required for this particular example, but I included them anyway, because otherwise, readers already familiar with the Either monad would wonder why they weren't there.
All these functions are generic and pure, so they are easy to unit test. I'm not going to show you the unit tests, however, as I consider the functions belonging to that Result
module as reusable functions. This is a module that would ship as part of a well-tested library. In fact, it'll soon be added to the F# core library.
With the already tested getUpper
function, you now have all the building blocks required to implement the desired functionality:
// string -> string let getUpperText path = path |> Result.split File.Exists |> Result.either (File.ReadAllText >> getUpper) (fun _ -> "DEFAULT")
This composition pipes path
into Result.split
, which uses File.Exists
as a predicate to decide whether the path should be packaged into a Success
or Failure
case. The resulting Result<string, string>
is then piped into Result.either
, which invokes File.ReadAllText >> getUpper
in the Success
case, and the anonymous function in the Failure
case.
Notice how, once again, the impure functions File.Exists
and File.ReadAllText
are used as humble functions, but interleaved with testable, pure functions that make all the decisions.
Maybe #
Sometimes, a decision isn't so much between two alternatives as it's a decision between something that may exist, but also may not. You can model this with the Maybe monad, which in F# comes in the form of the built-in option
type.
In fact, so much is already built in (and tested by the F# development team) that you almost don't need to add anything yourself. The only function you could consider adding is this:
module Option = // 'a -> 'a option -> 'a let defaultIfNone def x = defaultArg x def
As you can see, this function simply swaps the arguments for the built-in defaultArg
function. This is done to make it more pipe-friendly. This function will most likely be added in a future version of F#.
That's all you need:
// string -> string let getUpperText path = path |> Some |> Option.filter File.Exists |> Option.map (File.ReadAllText >> getUpper) |> Option.defaultIfNone "DEFAULT"
This composition starts with the path
, puts it into a Some
case, and pipes that option
value into Option.filter File.Exists
. This means that the Some
case will only stay a Some
value if the file exists; otherwise, it will be converted to a None
value. Whatever the option
value is, it's then piped into Option.map (File.ReadAllText >> getUpper)
. The composed function File.ReadAllText >> getUpper
is only executed in the Some
case, so if the file doesn't exist, the function will not attempt to read it. Finally, the option
value is piped into Option.defaultIfNone
, which returns the mapped value, or "DEFAULT"
if the value was None
.
Like in the two previous examples, the decision logic is implemented by pure functions, whereas the impure functions File.Exists
and File.ReadAllText
are handled as humble functions.
Summary #
Have you noticed a pattern in all the three examples? Decisions are separated from effects using discriminated unions (both the above Action
, Result<'TSuccess, 'TFailure>
, and the built-in option
are discriminated unions). In my experience, as long as you need to decide between two alternatives, the Either or Maybe monads are often sufficient to decouple decision logic from effects. Often, I don't even need to write any tests, because I compose my functions from the known, well-tested functions that belong to the respective monads.
If your decision has to branch between three or more alternatives, you can consider a custom discriminated union. For this particular example, though, I think I prefer the third, Maybe-based composition, but closely followed by the Either-based composition.
In this article, you saw three examples of how to decouple decision from effects; and I didn't even show you the Free monad!
Comments
Mark,
I can't understand how can the
getValue
function be pure. While I agree that it's easy to test, it's still the higher order function and it's purity depends on the purity of function passed as the argument. Even in Your example it takesFile.ReadAllText >> getUpper
which actually reaches to a file on the disk and I perceive it as reaching to an external shared state. Is there something I misunderstood?Grzegorz, thank you for writing. You make a good point, and in a sense you're correct. F# doesn't enforce purity, and this is both an advantage and a disadvantage. It's an advantage because it makes it easier for programmers migrating from C# to make a gradual transition to a more functional programming style. It's also an advantage exactly because it relies on the programmer's often-faulty reasoning to ensure that code is properly functional.
Functions in F# are only pure if they're implemented to be pure. For any given function type (signature) you can always create an impure function that fits the type. (If nothing else, you can always write "Hello, world!" to the console, before returning a value.)
The result of this is that few parts of F# are pure in the sense that you imply. Even
List.map
could be impure, if passed an impure function. In other words, higher-order functions in F# are only pure if composed of exclusively pure parts.Clearly, this is in stark contrast to Haskell, where purity is enforced at the type level. In Haskell, a throw-away, poorly designed mini-API like the
Action
type and associated functions shown here wouldn't even compile. The Either and Maybe examples, on the other hand, would.My assumption here is that function composition happens at the edge of the application - that is, in an impure (
IO
) context.