Functional programming emphasises pure functions, but sometimes decisions must be made based on impure data. The solution is to decouple decisions and effects.

Functional programmers love pure functions. Not only do they tend to be easy to reason about, they are also intrinsically testable. It'd be wonderful if we could build entire systems only from pure functions, but every functional programmer knows that the world is impure. Instead, we strive towards implementing as much of our code base as pure functions, so that an application is impure only at its boundaries.

The more you can do this, the more testable the system becomes. One rule of thumb about unit testing that I often use is that if a particular candidate for unit testing has a cyclomatic complexity of 1, it may be acceptable to skip unit testing it. Instead, we can consider such a unit a humble unit. If you can separate decisions from effects (which is what functional programmers often call impurities), you can often make the impure functions humble.

In other words: put all logic in pure functions that can be unit tested, and implement impure effects as humble functions that you don't need to unit test.

You want to see an example. So do I!

Example: conditional reading from file #

In a recent discussion, Jamie Cansdale asks how I'd design and unit test something like the following C# method if I could instead redesign it in F#.

public static string GetUpperText(string path)
    if (!File.Exists(path)) return "DEFAULT";
    var text = File.ReadAllText(path);
    return text.ToUpperInvariant();

Notice how this method contains two impure operations: File.Exists and File.ReadAllText. Decision logic seems interleaved with IO. How can decisions be separated from effects?

(For good measure I ought to point out that obviously, the above example is so simple that by itself, it almost doesn't warrant testing. Think of it as a stand-in for a more complex problem.)

With a statement-based language like C#, it can be difficult to see how to separate decision logic from effects without introducing interfaces, but with expression-based languages like F#, it becomes close to trivial. In this article, I'll show you three alternatives.

All three alternatives, however, make use of the same function for turning text into upper case:

// string -> string
let getUpper (text : string) = text.ToUpperInvariant ()

Obviously, this function is so trivial that it's hardly worth testing, but remember to think about it as a stand-in for a more complex problem. It's a pure function, so it's easy to unit test:

let ``getUpper returns correct value`` input expected =
    let actual = getUpper input
    expected =! actual

This test uses 2.1.0 and Unquote 3.1.2. The =! operator is a custom Unquote operator; you can read it as must equal; that is: expected must equal actual. It'll throw an exception if this isn't the case.

Custom unions #

Languages like F# come with algebraic data types, which means that in addition to complex structures, they also enable you to express types as alternatives. This means that you can represent a decision as one or more alternative pure values.

Although the examples you'll see later in this article are simpler, I think it'll be helpful to start with an ad hoc solution to the problem. Here, the decision is to either read from a file, or return a default value. You can express that using a custom discriminated union:

type Action = ReadFromFile of string | UseDefault of string

This type models two mutually exclusive cases: either you decide to read from the file identified by a file path (string), or your return a default value (also modelled as a string).

Using this Action type, you can write a pure function that makes the decision:

// string -> bool -> Action
let decide path fileExists =
    if fileExists
    then ReadFromFile path
    else UseDefault "DEFAULT"

This function takes two arguments: path (a string) and fileExists (a bool). If fileExists is true, it returns the ReadFromFile case; otherwise, it returns the UseDefault case.

Notice how this function neither checks whether the file exists, nor does it attempt to read the contents of the file. It only makes a decision based on input, and returns information about this decision as output. This function is pure, so (as I've claimed numerous times) is easy to unit test:

let ``decide returns correct result when file exists`` path =
    let actual = decide path true
    ReadFromFile path =! actual
let ``decide returns correct result when file doesn't exist`` path =
    let actual = decide path false
    UseDefault "DEFAULT" =! actual

One unit test function exercises the code path where the file exists, whereas the other test exercises the code path where it doesn't. Straightforward.

There's still some remaining work, because you need to somehow compose your pure functions with File.Exists and File.ReadAllText. You also need a way to extract the string value from the two cases of Action. One way to do that is to introduce another pure function:

// (string -> string) -> Action -> string
let getValue f = function
    | ReadFromFile path -> f path
    | UseDefault value  -> value

This is a function that returns the UseDefault data for that case, but invokes a function f in the ReadFromFile case. Again, since this function is pure it's easy to unit test it, but I'll leave that as an exercise.

You now have all the building blocks required to compose a function similar to the above GetUpperText C# method:

// string -> string
let getUpperText path =
    |> File.Exists
    |> decide path
    |> getValue (File.ReadAllText >> getUpper)

This implementation pipes path into File.Exists, which returns a Boolean value indicating whether the file exists. This Boolean value is then piped into decide path, which (as you may recall) returns an Action. That value is finally piped into getValue (File.ReadAllText >> getUpper). Recall that getValue will only invoke the function argument when the Action is ReadFromFile, so File.ReadAllText >> getUpper is only executed in this case.

Notice how decisions and effectful functions are interleaved. All the decision functions are covered by unit tests; only File.Exists and File.ReadAllText aren't covered, but I find it reasonable to treat these as humble functions.

Either #

Normally, decisions often involve a choice between two alternatives. In the above example, you saw how the alternatives were named ReadFromFile and UseDefault. Since a choice between two alternatives is so common, there's a well-known 'pattern' that gives you general-purpose tools to model decisions. This is known as the Either monad.

The F# core library doesn't (yet) come with an implementation of the Either monad, but it's easy to add. In this example, I'm using code from Scott Wlaschin's railway-oriented programming, although slightly modified, and including only the most essential building blocks for the example:

type Result<'TSuccess, 'TFailure> =
    | Success of 'TSuccess
    | Failure of 'TFailure
module Result =
    // ('a -> Result<'b, 'c>) -> Result<'a, 'c> -> Result<'b, 'c>
    let bind f = function
        | Success succ -> f succ
        | Failure fail -> Failure fail
    // ('a -> 'b) -> Result<'a, 'c> -> Result<'b, 'c>
    let map f = function
        | Success succ -> Success (f succ)
        | Failure fail -> Failure fail
    // ('a -> bool) -> 'a -> Result<'a, 'a>
    let split f x = if f x then Success x else Failure x
    // ('a -> 'b) -> ('c -> 'b) -> Result<'a, 'c> -> 'b
    let either f g = function
        | Success succ -> f succ
        | Failure fail -> g fail

In fact, the bind and map functions aren't even required for this particular example, but I included them anyway, because otherwise, readers already familiar with the Either monad would wonder why they weren't there.

All these functions are generic and pure, so they are easy to unit test. I'm not going to show you the unit tests, however, as I consider the functions belonging to that Result module as reusable functions. This is a module that would ship as part of a well-tested library. In fact, it'll soon be added to the F# core library.

With the already tested getUpper function, you now have all the building blocks required to implement the desired functionality:

// string -> string
let getUpperText path =
    |> Result.split File.Exists
    |> Result.either (File.ReadAllText >> getUpper) (fun _ -> "DEFAULT")

This composition pipes path into Result.split, which uses File.Exists as a predicate to decide whether the path should be packaged into a Success or Failure case. The resulting Result<string, string> is then piped into Result.either, which invokes File.ReadAllText >> getUpper in the Success case, and the anonymous function in the Failure case.

Notice how, once again, the impure functions File.Exists and File.ReadAllText are used as humble functions, but interleaved with testable, pure functions that make all the decisions.

Maybe #

Sometimes, a decision isn't so much between two alternatives as it's a decision between something that may exist, but also may not. You can model this with the Maybe monad, which in F# comes in the form of the built-in option type.

In fact, so much is already built in (and tested by the F# development team) that you almost don't need to add anything yourself. The only function you could consider adding is this:

module Option =
    // 'a -> 'a option -> 'a
    let defaultIfNone def x = defaultArg x def

As you can see, this function simply swaps the arguments for the built-in defaultArg function. This is done to make it more pipe-friendly. This function will most likely be added in a future version of F#.

That's all you need:

// string -> string
let getUpperText path =
    |> Some
    |> Option.filter File.Exists
    |> (File.ReadAllText >> getUpper)
    |> Option.defaultIfNone "DEFAULT"

This composition starts with the path, puts it into a Some case, and pipes that option value into Option.filter File.Exists. This means that the Some case will only stay a Some value if the file exists; otherwise, it will be converted to a None value. Whatever the option value is, it's then piped into (File.ReadAllText >> getUpper). The composed function File.ReadAllText >> getUpper is only executed in the Some case, so if the file doesn't exist, the function will not attempt to read it. Finally, the option value is piped into Option.defaultIfNone, which returns the mapped value, or "DEFAULT" if the value was None.

Like in the two previous examples, the decision logic is implemented by pure functions, whereas the impure functions File.Exists and File.ReadAllText are handled as humble functions.

Summary #

Have you noticed a pattern in all the three examples? Decisions are separated from effects using discriminated unions (both the above Action, Result<'TSuccess, 'TFailure>, and the built-in option are discriminated unions). In my experience, as long as you need to decide between two alternatives, the Either or Maybe monads are often sufficient to decouple decision logic from effects. Often, I don't even need to write any tests, because I compose my functions from the known, well-tested functions that belong to the respective monads.

If your decision has to branch between three or more alternatives, you can consider a custom discriminated union. For this particular example, though, I think I prefer the third, Maybe-based composition, but closely followed by the Either-based composition.

In this article, you saw three examples of how to decouple decision from effects; and I didn't even show you the Free monad!



I can't understand how can the getValue function be pure. While I agree that it's easy to test, it's still the higher order function and it's purity depends on the purity of function passed as the argument. Even in Your example it takes File.ReadAllText >> getUpper which actually reaches to a file on the disk and I perceive it as reaching to an external shared state. Is there something I misunderstood?

2016-10-14 09:06 UTC

Grzegorz, thank you for writing. You make a good point, and in a sense you're correct. F# doesn't enforce purity, and this is both an advantage and a disadvantage. It's an advantage because it makes it easier for programmers migrating from C# to make a gradual transition to a more functional programming style. It's also an advantage exactly because it relies on the programmer's often-faulty reasoning to ensure that code is properly functional.

Functions in F# are only pure if they're implemented to be pure. For any given function type (signature) you can always create an impure function that fits the type. (If nothing else, you can always write "Hello, world!" to the console, before returning a value.)

The result of this is that few parts of F# are pure in the sense that you imply. Even could be impure, if passed an impure function. In other words, higher-order functions in F# are only pure if composed of exclusively pure parts.

Clearly, this is in stark contrast to Haskell, where purity is enforced at the type level. In Haskell, a throw-away, poorly designed mini-API like the Action type and associated functions shown here wouldn't even compile. The Either and Maybe examples, on the other hand, would.

My assumption here is that function composition happens at the edge of the application - that is, in an impure (IO) context.

2016-10-15 09:02 UTC

Wish to comment?

You can add a comment to this post by sending me a pull request. Alternatively, you can discuss this post on Twitter or somewhere else with a permalink. Ping me with the link, and I may respond.


Monday, 26 September 2016 21:51:00 UTC


"Our team wholeheartedly endorses Mark. His expert service provides tremendous value."
Hire me!
Published: Monday, 26 September 2016 21:51:00 UTC