Error-accumulating composable assertions in C# by Mark Seemann

Perhaps the list monoid is all you need for non-short-circuiting assertions.

This article is the second instalment in a small articles series about applicative assertions. It explores a way to compose assertions in such a way that failure messages accumulate rather than short-circuit. It assumes that you've read the article series introduction and the previous article.

Unsurprisingly, the previous article showed that you can use an applicative functor to create composable assertions that don't short-circuit. It also concluded that, in C# at least, the API is awkward.

This article explores a simpler API.

A clue left by the proof of concept #

The previous article's proof of concept left a clue suggesting a simpler API. Consider, again, how the rather horrible RunAssertions method decides whether or not to throw an exception:

string errors = composition.Match(
    onFailure: f => string.Join(Environment.NewLine, f),
    onSuccess: _ => string.Empty);
 
if (!string.IsNullOrEmpty(errors))
    throw new Exception(errors);

Even though Validated<F, S> is a sum type, the RunAssertions method declines to take advantage of that. Instead, it reduces composition to a simple type: A string. It then decides to throw an exception if the errors value is not null or empty.

This suggests that using a sum type may not be necessary to distinguish between the success and the failure case. Rather, an empty error string is all it takes to indicate success.

Non-empty errors #

The proof-of-concept assertion type is currently defined as Validated with a particular combination of type arguments: Validated<IReadOnlyCollection<string>, Unit>. Consider, again, this Match expression:

string errors = composition.Match(
    onFailure: f => string.Join(Environment.NewLine, f),
    onSuccess: _ => string.Empty);

Does an empty string unambiguously indicate success? Or is it possible to arrive at an empty string even if composition actually represents a failure case?

You can arrive at an empty string from a failure case if the collection of error messages is empty. Consider the type argument that takes the place of the F generic type: IReadOnlyCollection<string>. A collection of this type can be empty, which would also cause the above Match to produce an empty string.

Even so, the proof-of-concept works in practice. The reason it works is that failure cases will never have empty assertion messages. We know this because (in the proof-of-concept code) only two functions produce assertions, and they each populate the error message collection with a string. You may want to revisit the AssertTrue and AssertEqual functions in the previous article to convince yourself that this is true.

This is a good example of knowledge that 'we' as developers know, but the code currently doesn't capture. Having to deal with such knowledge taxes your working memory, so why not encapsulate such information in the type itself?

How do you encapsulate the knowledge that a collection is never empty? Introduce a NotEmptyCollection collection. I'll reuse the class from the article Semigroups accumulate and add a Concat instance method:

public NotEmptyCollection<T> Concat(NotEmptyCollection<T> other)
{
    return new NotEmptyCollection<T>(Head, Tail.Concat(other).ToArray());
}

Since the two assertion-producing functions both supply an error message in the failure case, it's trivial to change them to return Validated<NotEmptyCollection<string>, Unit> - just change the types used:

public static Validated<NotEmptyCollection<string>, Unit> AssertTrue(
    this bool condition,
    string message)
{
    return condition
        ? Succeed<NotEmptyCollection<string>, Unit>(Unit.Value)
        : Fail<NotEmptyCollection<string>, Unit>(new NotEmptyCollection<string>(message));
}
 
public static Validated<NotEmptyCollection<string>, Unit> AssertEqual<T>(
    T expected,
    T actual)
{
    return Equals(expected, actual)
        ? Succeed<NotEmptyCollection<string>, Unit>(Unit.Value)
        : Fail<NotEmptyCollection<string>, Unit>(
            new NotEmptyCollection<string>($"Expected {expected}, but got {actual}."));
}

This change guarantees that the RunAssertions method only produces an empty errors string in success cases.

Error collection isomorphism #

Assertions are still defined by the Validated sum type, but the success case carries no information: Validated<NotEmptyCollection<T>, Unit>, and the failure case is always guaranteed to contain at least one error message.

This suggests that a simpler representation is possible: One that uses a normal collection of errors, and where an empty collection indicates an absence of errors:

public class Asserted<T>
{
    public Asserted() : this(Array.Empty<T>())
    {
    }
 
    public Asserted(T error) : this(new[] { error })
    {
    }
 
    public Asserted(IReadOnlyCollection<T> errors)
    {
        Errors = errors;
    }
 
    public Asserted<T> And(Asserted<T> other)
    {
        if (other is null)
            throw new ArgumentNullException(nameof(other));
 
        return new Asserted<T>(Errors.Concat(other.Errors).ToList());
    }
 
    public IReadOnlyCollection<T> Errors { get; }
}

The Asserted<T> class is scarcely more than a glorified wrapper around a normal collection, but it's isomorphic to Validated<NotEmptyCollection<T>, Unit>, which the following two functions prove:

public static Asserted<T> FromValidated<T>(this Validated<NotEmptyCollection<T>, Unit> v)
{
    return v.Match(
        failures => new Asserted<T>(failures),
        _ => new Asserted<T>());
}
 
public static Validated<NotEmptyCollection<T>, Unit> ToValidated<T>(this Asserted<T> a)
{
    if (a.Errors.Any())
    {
        var errors = new NotEmptyCollection<T>(
            a.Errors.First(),
            a.Errors.Skip(1).ToArray());
        return Validated.Fail<NotEmptyCollection<T>, Unit>(errors);
    }
    else
        return Validated.Succeed<NotEmptyCollection<T>, Unit>(Unit.Value);
}

You can translate back and forth between Validated<NotEmptyCollection<T>, Unit> and Asserted<T> without loss of information.

A collection, however, gives rise to a monoid, which suggests a much simpler way to compose assertions than using an applicative functor.

Asserted truth #

You can now rewrite the assertion-producing functions to return Asserted<string> instead of Validated<NotEmptyCollection<string>, Unit>.

public static Asserted<string> True(bool condition, string message)
{
    return condition ? new Asserted<string>() : new Asserted<string>(message);
}

This Asserted.True function returns no error messages when condition is true, but a collection with the single element message when it's false.

You can use it in a unit test like this:

var assertResponse = Asserted.True(
    deleteResp.IsSuccessStatusCode,
    $"Actual status code: {deleteResp.StatusCode}.");

You'll see how assertResponse composes with another assertion later in this article. The example continues from the previous article. It's the same test from the same code base.

Asserted equality #

You can also rewrite the other assertion-producing function in the same way:

public static Asserted<string> Equal(object expected, object actual)
{
    if (Equals(expected, actual))
        return new Asserted<string>();
 
    return new Asserted<string>($"Expected {expected}, but got {actual}.");
}

Again, when the assertion passes, it returns no errors; otherwise, it returns a collection with a single error message.

Using it may look like this:

var getResp = await api.CreateClient().GetAsync(address);
var assertState = Asserted.Equal(HttpStatusCode.NotFound, getResp.StatusCode);

At this point, each of the assertions are objects that represent a verification step. By themselves, they neither pass nor fail the test. You have to execute them to reach a verdict.

Evaluating assertions #

The above code listing of the Asserted<T> class already shows how to combine two Asserted<T> objects into one. The And instance method is a binary operation that, together with the parameterless constructor, makes Asserted<T> a monoid.

Once you've combined all assertions into a single Asserted<T> object, you need to Run it to produce a test outcome:

public static void Run(this Asserted<string> assertions)
{
    if (assertions?.Errors.Any() ?? false)
    {
        var messages = string.Join(Environment.NewLine, assertions.Errors);
        throw new Exception(messages);
    }
}

If there are no errors, Run does nothing; otherwise it combines all the error messages together and throws an exception. As was also the case in the previous article, I've allowed myself a few proof-of-concept shortcuts. The framework design guidelines admonishes against throwing System.Exception. It might be more appropriate to introduce a new Exception type that also allows enumerating the error messages.

The entire assertion phase of the test looks like this:

var assertResponse = Asserted.True(
    deleteResp.IsSuccessStatusCode,
    $"Actual status code: {deleteResp.StatusCode}.");
var getResp = await api.CreateClient().GetAsync(address);
var assertState = Asserted.Equal(HttpStatusCode.NotFound, getResp.StatusCode);
assertResponse.And(assertState).Run();

You can see the entire test in the previous article. Notice how the two assertion objects are first combined into one with the And binary operation. The result is a single Asserted<string> object on which you can call Run.

Like the previous proof of concept, this assertion passes and fails in the same way. It's possible to compose assertions and collect error messages, instead of short-circuiting on the first failure, even without an applicative functor.

Method chaining #

If you don't like to come up with variable names just to make assertions, it's also possible to use the Asserted API's fluent interface:

var getResp = await api.CreateClient().GetAsync(address);
Asserted
    .True(
        deleteResp.IsSuccessStatusCode,
        $"Actual status code: {deleteResp.StatusCode}.")
    .And(Asserted.Equal(HttpStatusCode.NotFound, getResp.StatusCode))
    .Run();

This isn't necessarily better, but it's an option.

Conclusion #

While it's possible to design non-short-circuiting composable assertions using an applicative functor, it looks as though a simpler solution might solve the same problem. Collect error messages. If none were collected, interpret that as a success.

As I wrote in the introduction article, however, this may not be the last word. Some assertions return values that can be used for other assertions. That's a scenario that I have not yet investigated in this light, and it may change the conclusion. If so, I'll add more articles to this small article series. As I'm writing this, though, I have no such plans.

Did I just, in a roundabout way, write that more research is needed?

Next: Built-in alternatives to applicative assertions.

Comments

• ← Previous
• Archive
• Next →