An example of state-based testing in Haskell by Mark Seemann
How do you do state-based testing when state is immutable? You use the State monad.
This article is an instalment in an article series about how to move from interaction-based testing to state-based testing. In the previous article, you saw an example of an interaction-based unit test written in C#. The problem that this article series attempts to address is that interaction-based testing can lead to what xUnit Test Patterns calls Fragile Tests, because the tests get coupled to implementation details, instead of overall behaviour.
My experience is that functional programming is better aligned with unit testing because functional design is intrinsically testable. While I believe that functional programming is no panacea, it still seems to me that we can learn many valuable lessons about programming from it.
People often ask me about F# programming: How do I know that my F# code is functional?
I sometimes wonder that myself, about my own F# code. One can certainly choose to ignore such a question as irrelevant, and I sometimes do, as well. Still, in my experience, asking such questions can create learning opportunities.
The best answer that I've found is: Port the F# code to Haskell.
Haskell enforces referential transparency via its compiler. If Haskell code compiles, it's functional. In this article, then, I take the problem from the previous article and port it to Haskell.
The code shown in this article is available on GitHub.
A function to connect two users #
In the previous article, you saw implementation and test coverage of a piece of software functionality to connect two users with each other. This was a simplification of the example running through my two Clean Coders videos, Church Visitor and Preserved in translation.
In contrast to the previous article, we'll start with the implementation of the System Under Test (SUT).
post :: Monad m => (a -> m (Either UserLookupError User)) -> (User -> m ()) -> a -> a -> m (HttpResponse User) post lookupUser updateUser userId otherUserId = do userRes <- first (\case InvalidId -> "Invalid user ID." NotFound -> "User not found.") <$> lookupUser userId otherUserRes <- first (\case InvalidId -> "Invalid ID for other user." NotFound -> "Other user not found.") <$> lookupUser otherUserId connect <- runExceptT $ do user <- ExceptT $ return userRes otherUser <- ExceptT $ return otherUserRes lift $ updateUser $ addConnection user otherUser return otherUser return $ either BadRequest OK connect
This is as direct a translation of the C# code as makes sense. If I'd only been implementing the desired functionality in Haskell, without having to port existing code, I'd designed the code differently.
post function uses partial application as an analogy to dependency injection, but in order to enable potentially impure operations to take place, everything must happen inside of some monad. While the production code must ultimately run in the
IO monad in order to interact with a database, tests can choose to run in another monad.
In the C# example, two dependencies are injected into the class that defines the
Post method. In the above Haskell function, these two dependencies are instead passed as function arguments. Notice that both functions return values in the monad
The intent of the
lookupUser argument is that it'll query a database with a user ID. It'll return the user if present, but it could also return a
UserLookupError, which is a simple sum type:
data UserLookupError = InvalidId | NotFound deriving (Show, Eq)
If both users are found, the function connects the users and calls the
updateUser function argument. The intent of this 'dependency' is that it updates the database. This is recognisably a Command, since its return type is
m () - unit (
()) is equivalent to
State-based testing #
How do you unit test such a function? How do you use Mocks and Stubs in Haskell? You don't; you don't have to. While the
post method can be impure (when
IO), it doesn't have to be. Functional design is intrinsically testable, but that proposition depends on purity. Thus, it's worth figuring out how to keep the
post function pure in the context of unit testing.
IO implies impurity, most common monads are pure. Which one should you choose? You could attempt to entirely 'erase' the monadic quality of the
post function with the
Identity monad, but if you do that, you can't verify whether or not
updateUser was invoked.
While you could write an ad-hoc Mock using, for example, the
Writer monad, it might be a better choice to investigate if something closer to state-based testing would be possible.
In an object-oriented context, state-based testing implies that you exercise the SUT, which mutates some state, and then you verify that the (mutated) state matches your expectations. You can't do that when you test a pure function, but you can examine the state of the function's return value. The
State monad is an obvious choice, then.
A Fake database #
State monad is parametrised on the state type as well as the normal 'value type', so in order to be able to test the
post function, you'll have to figure out what type of state to use. The interactions implied by the
updateUser arguments are those of database interactions. A Fake database seems an obvious choice.
For the purposes of testing the
post function, an in-memory database implemented using a
Map is appropriate:
type DB = Map Integer User
This is simply a dictionary keyed by
Integer values and containing
User values. You can implement compatible
updateUser functions with
State DB as the
updateUser function is the easiest one to implement:
updateUser :: User -> State DB () updateUser user = modify $ Map.insert (userId user) user
This simply inserts the
user into the database, using the
userId as the key. The type of the function is compatible with the general requirement of
User -> m (), since here,
lookupUser Fake implementation is a bit more involved:
lookupUser :: String -> State DB (Either UserLookupError User) lookupUser s = do let maybeInt = readMaybe s :: Maybe Integer let eitherInt = maybe (Left InvalidId) Right maybeInt db <- get return $ eitherInt >>= maybe (Left NotFound) Right . flip Map.lookup db
First, consider the type. The function takes a
String value as an argument and returns a
State DB (Either UserLookupError User). The requirement is a function compatible with the type
a -> m (Either UserLookupError User). This works when
m is, again,
The entire function is written in
do notation, where the inferred
Monad is, indeed,
State DB. The first line attempts to parse the
String into an
Integer. Since the built-in
readMaybe function returns a
Maybe Integer, the next line uses the
maybe function to handle the two possible cases, converting the
Nothing case into the
Left InvalidId value, and the
Just case into a
It then uses the
get function to access the database
db, and finally attempt a
lookup against that
maybe is used to convert the
Maybe value returned by
Map.lookup into an
Happy path test case #
This is all you need in terms of Test Doubles. You now have test-specific
updateUser functions that you can pass to the
Like in the previous article, you can start by exercising the happy path where a user successfully connects with another user:
testProperty "Users successfully connect" $ \ user otherUser -> runStateTest $ do put $ Map.fromList [toDBEntry user, toDBEntry otherUser] actual <- post lookupUser updateUser (show $ userId user) (show $ userId otherUser) db <- get return $ isOK actual && any (elem otherUser . connectedUsers) (Map.lookup (userId user) db)
Here I'm inlining test cases as anonymous functions - this time expressing the tests as QuickCheck properties. I'll later return to the
runStateTest helper function, but first I want to focus on the test body itself. It's written in
do notation, and specifically, it runs in the
State DB monad.
otherUser are input arguments to the property. These are both
User values, since the test also defines
Arbitrary instances for that type (not shown in this article; see the source code repository for details).
The first step in the test is to 'save' both users in the Fake database. This is easily done by converting each
User value to a database entry:
toDBEntry :: User -> (Integer, User) toDBEntry = userId &&& id
Recall that the Fake database is nothing but an alias over
Map Integer User, so the only operation required to turn a
User into a database entry is to extract the key.
The next step in the test is to exercise the SUT, passing the test-specific
updateUser Test Doubles to the
post function, together with the user IDs converted to
In the assert phase of the test, it first extracts the current state of the database, using the
State library's built-in
get function. It then verifies that
actual represents a
200 OK value, and that the
user entry in the database now contains
otherUser as a connected user.
Missing user test case #
While there's one happy-path test case, there's four other test cases left. One of these is when the first user doesn't exist:
testProperty "Users don't connect when user doesn't exist" $ \ (Positive i) otherUser -> runStateTest $ do let db = Map.fromList [toDBEntry otherUser] put db let uniqueUserId = show $ userId otherUser + i actual <- post lookupUser updateUser uniqueUserId (show $ userId otherUser) assertPostFailure db actual
What ought to trigger this test case is that the 'first' user doesn't exist, even if the
otherUser does exist. For this reason, the test inserts the
otherUser into the Fake database.
Since the test is a QuickCheck property,
i could be any positive
Integer value - including the
otherUser. In order to properly exercise the test case, however, you'll need to call the
post function with a
uniqueUserId - thas it: an ID which is guaranteed to not be equal to the
otherUser. There's several options for achieving this guarantee (including, as you'll see soon, the
==> operator), but a simple way is to add a non-zero number to the number you need to avoid.
You then exercise the
post function and, as a verification, call a reusable
assertPostFailure :: (Eq s, Monad m) => s -> HttpResponse a -> StateT s m Bool assertPostFailure stateBefore resp = do stateAfter <- get let stateDidNotChange = stateBefore == stateAfter return $ stateDidNotChange && isBadRequest resp
This function verifies that the state of the database didn't change, and that the response value represents a
400 Bad Request HTTP response. This verification doesn't actually verify that the error message associated with the
BadRequest case is the expected message, like in the previous article. This would, however, involve a fairly trivial change to the code.
Missing other user test case #
Similar to the above test case, users will also fail to connect if the 'other user' doesn't exist. The property is almost identical:
testProperty "Users don't connect when other user doesn't exist" $ \ (Positive i) user -> runStateTest $ do let db = Map.fromList [toDBEntry user] put db let uniqueOtherUserId = show $ userId user + i actual <- post lookupUser updateUser (show $ userId user) uniqueOtherUserId assertPostFailure db actual
Since this test body is so similar to the previous test, I'm not going to give you a detailed walkthrough. I did, however, promise to describe the
runStateTest helper function:
runStateTest :: State (Map k a) b -> b runStateTest = flip evalState Map.empty
Since this is a one-liner, you could also write all the tests by simply in-lining that little expression, but I thought that it made the tests more readable to give this function an explicit name.
It takes any
State (Map k a) b and runs it with an empty map. Thus, all
State-valued functions, like the tests, must explicitly put data into the state. This is also what the tests do.
Notice that all the tests return
State values. For example, the
assertPostFailure function returns
StateT s m Bool, of which
State s Bool is an alias. This fits
State (Map k a) b when
Map k a, which again is aliased to
DB. Reducing all of this, the tests are simply functions that return
Invalid user ID test cases #
Finally, you can also cover the two test cases where one of the user IDs is invalid:
testProperty "Users don't connect when user Id is invalid" $ \ s otherUser -> isIdInvalid s ==> runStateTest $ do let db = Map.fromList [toDBEntry otherUser] put db actual <- post lookupUser updateUser s (show $ userId otherUser) assertPostFailure db actual , testProperty "Users don't connect when other user Id is invalid" $ \ s user -> isIdInvalid s ==> runStateTest $ do let db = Map.fromList [toDBEntry user] put db actual <- post lookupUser updateUser (show $ userId user) s assertPostFailure db actual
Both of these properties take a
s as input. When QuickCheck generates a
String, that could be any
String value. Both tests require that the value is an invalid user ID. Specifically, it mustn't be possible to parse the string into an
Integer. If you don't constrain QuickCheck, it'll generate various strings, including e.g.
"8" and other strings that can be parsed as numbers.
In the above
"Users don't connect when user doesn't exist" test, you saw how one way to explicitly model constraints on data is to project a seed value in such a way that the constraint always holds. Another way is to use QuickCheck's built-in
==> operator to filter out undesired values. In this example, both tests employ the
isIdInvalid :: String -> Bool isIdInvalid s = let userInt = readMaybe s :: Maybe Integer in isNothing userInt
isIdInvalid with the
==> operator guarantees that
s is an invalid ID.
While state-based testing may, at first, sound incompatible with strictly functional programming, it's not only possible with the State monad, but even, with good language support, easily done.
The tests shown in this article aren't concerned with the interactions between the SUT and its dependencies. Instead, they compare the initial state with the state after exercising the SUT. Comparing values, even complex data structures such as maps, tends to be trivial in functional programming. Immutable values typically have built-in structural equality (in Haskell signified by the automatic
Eq type class), which makes comparing them trivial.
Now that we know that state-based testing is possible even with Haskell's enforced purity, it should be clear that we can repeat the feat in F#.