Serializing restaurant tables in F# by Mark Seemann
Using System.Text.Json, with and without Reflection.
This article is part of a short series of articles about serialization with and without Reflection. In this instalment I'll explore some options for serializing JSON with F# using the API built into .NET: System.Text.Json. I'm not going use Json.NET in this article, but I've done similar things with that library in the past, so what's here is, at least, somewhat generalizable.
Natural numbers #
Before we start investigating how to serialize to and from JSON, we must have something to serialize. As described in the introductory article we'd like to parse and write restaurant table configurations like this:
{
"singleTable": {
"capacity": 16,
"minimalReservation": 10
}
}
On the other hand, I'd like to represent the Domain Model in a way that encapsulates the rules governing the model, making illegal states unrepresentable.
As the first step, we observe that the numbers involved are all natural numbers. In F# it's both idiomatic and easy to define a predicative data type:
type NaturalNumber = private NaturalNumber of int
Since it's defined with a private constructor we need to also supply a way to create valid values of the type:
module NaturalNumber = let tryCreate n = if n < 1 then None else Some (NaturalNumber n)
In this, as well as the other articles in this series, I've chosen to model the potential for errors with Option values. I could also have chosen to use Result if I wanted to communicate information along the 'error channel', but sticking with Option makes the code a bit simpler. Not so much in F# or Haskell, but once we reach C#, applicative validation becomes complicated.
There's no loss of generality in this decision, since both Option and Result are applicative functors.
> NaturalNumber.tryCreate -1;;
val it: NaturalNumber option = None
> let x = NaturalNumber.tryCreate 42;;
val x: NaturalNumber option = Some NaturalNumber 42
The tryCreate function enables client developers to create NaturalNumber values, and due to the F#'s default equality and comparison implementation, you can even compare them:
> let y = NaturalNumber.tryCreate 2112;;
val y: NaturalNumber option = Some NaturalNumber 2112
> x < y;;
val it: bool = true
That's it for natural numbers. Three lines of code. Compare that to the Haskell implementation, which required eight lines of code. This is mostly due to F#'s private keyword, which Haskell doesn't have.
Domain Model #
Modelling a restaurant table follows in the same vein. One invariant I would like to enforce is that for a 'single' table, the minimal reservation should be a NaturalNumber less than or equal to the table's capacity. It doesn't make sense to configure a table for four with a minimum reservation of six.
In the same spirit as above, then, define this type:
type Table = private | SingleTable of NaturalNumber * NaturalNumber | CommunalTable of NaturalNumber
Once more the private keyword makes it impossible for client code to create instances directly, so we need a pair of functions to create values:
module Table = let trySingle capacity minimalReservation = option { let! cap = NaturalNumber.tryCreate capacity let! min = NaturalNumber.tryCreate minimalReservation if cap < min then return! None else return SingleTable (cap, min) } let tryCommunal = NaturalNumber.tryCreate >> Option.map CommunalTable
Notice that trySingle checks the invariant that the capacity must be greater than or equal to the minimalReservation.
Again, notice how much easier it is to define a predicative type in F#, compared to Haskell.
This isn't a competition between languages, and while F# certainly scores a couple of points here, Haskell has other advantages.
The point of this little exercise, so far, is that it encapsulates the contract implied by the Domain Model. It does this by using the static type system to its advantage.
JSON serialization by hand #
At the boundaries of applications, however, there are no static types. Is the static type system still useful in that situation?
For a long time, the most popular .NET library for JSON serialization was Json.NET, but these days I find the built-in API offered in the System.Text.Json namespace adequate. This is also the case here.
The original rationale for this article series was to demonstrate how serialization can be done without Reflection, so I'll start there and return to Reflection later.
In this article series, I consider the JSON format fixed. A single table should be rendered as shown above, and a communal table should be rendered like this:
{ "communalTable": { "capacity": 42 } }
Often in the real world you'll have to conform to a particular protocol format, or, even if that's not the case, being able to control the shape of the wire format is important to deal with backwards compatibility.
As I outlined in the introduction article you can usually find a more weakly typed API to get the job done. For serializing Table to JSON it looks like this:
let serializeTable = function | SingleTable (NaturalNumber capacity, NaturalNumber minimalReservation) -> let j = JsonObject () j["singleTable"] <- JsonObject () j["singleTable"]["capacity"] <- capacity j["singleTable"]["minimalReservation"] <- minimalReservation j.ToJsonString () | CommunalTable (NaturalNumber capacity) -> let j = JsonObject () j["communalTable"] <- JsonObject () j["communalTable"]["capacity"] <- capacity j.ToJsonString ()
In order to separate concerns, I've defined this functionality in a new module that references the module that defines the Domain Model. The serializeTable function pattern-matches on SingleTable and CommunalTable to write two different JsonObject objects, using the JSON API's underlying Document Object Model (DOM).
JSON deserialization by hand #
You can also go the other way, and when it looks more complicated, it's because it is. When serializing an encapsulated value, not a lot can go wrong because the value is already valid. When deserializing a JSON string, on the other hand, all sorts of things can go wrong: It might not even be a valid string, or the string may not be valid JSON, or the JSON may not be a valid Table representation, or the values may be illegal, etc.
Here I found it appropriate to first define a small API of parsing functions, mostly in order to make the object-oriented API more composable. First, I need some code that looks at the root JSON object to determine which kind of table it is (if it's a table at all). I found it appropriate to do that as a pair of active patterns:
let private (|Single|_|) (node : JsonNode) = match node["singleTable"] with | null -> None | tn -> Some tn let private (|Communal|_|) (node : JsonNode) = match node["communalTable"] with | null -> None | tn -> Some tn
It turned out that I also needed a function to even check if a string is a valid JSON document:
let private tryParseJson (candidate : string) = try JsonNode.Parse candidate |> Some with | :? System.Text.Json.JsonException -> None
If there's a way to do that without a try/with expression, I couldn't find it. Likewise, trying to parse an integer turns out to be surprisingly complicated:
let private tryParseInt (node : JsonNode) = match node with | null -> None | _ -> if node.GetValueKind () = JsonValueKind.Number then try node |> int |> Some with | :? FormatException -> None // Thrown on decimal numbers else None
Both tryParseJson and tryParseInt are, however, general-purpose functions, so if you have a lot of JSON you need to parse, you can put them in a reusable library.
With those building blocks you can now define a function to parse a Table:
let tryDeserializeTable (candidate : string) = match tryParseJson candidate with | Some (Single node) -> option { let! capacity = node["capacity"] |> tryParseInt let! minimalReservation = node["minimalReservation"] |> tryParseInt return! Table.trySingle capacity minimalReservation } | Some (Communal node) -> option { let! capacity = node["capacity"] |> tryParseInt return! Table.tryCommunal capacity } | _ -> None
Since both serialisation and deserialization is based on string values, you should write automated tests that verify that the code works, and in fact, I did. Here are a few examples:
[<Fact>] let ``Deserialize single table for 4`` () = let json = """{"singleTable":{"capacity":4,"minimalReservation":3}}""" let actual = tryDeserializeTable json Table.trySingle 4 3 =! actual [<Fact>] let ``Deserialize non-table`` () = let json = """{"foo":42}""" let actual = tryDeserializeTable json None =! actual
Apart from module declaration and imports etc. this hand-written JSON capability requires 46 lines of code, although, to be fair, some of that code (tryParseJson and tryParseInt) are general-purpose functions that belong in a reusable library. Can we do better with static types and Reflection?
JSON serialisation based on types #
The static JsonSerializer class comes with Serialize<T> and Deserialize<T> methods that use Reflection to convert a statically typed object to and from JSON. You can define a type (a Data Transfer Object (DTO) if you will) and let Reflection do the hard work.
In Code That Fits in Your Head I explain how you're usually better off separating the role of serialization from the role of Domain Model. One way to do that is exactly by defining a DTO for serialisation, and let the Domain Model remain exclusively to model the rules of the application. The above Table type plays the latter role, so we need new DTO types:
type CommunalTableDto = { Capacity : int } type SingleTableDto = { Capacity : int; MinimalReservation : int } type TableDto = { CommunalTable : CommunalTableDto option SingleTable : SingleTableDto option }
One way to model a sum type with a DTO is to declare both cases as option fields. While it does allow illegal states to be representable (i.e. both kinds of tables defined at the same time, or none of them present) this is only par for the course at the application boundary.
While you can serialize values of that type, by default the generated JSON doesn't have the right format:
> val dto: TableDto = { CommunalTable = Some { Capacity = 42 }
SingleTable = None }
> JsonSerializer.Serialize dto;;
val it: string = "{"CommunalTable":{"Capacity":42},"SingleTable":null}"
There are two problems with the generated JSON document:
- The casing is wrong
- The null value shouldn't be there
None of those are too hard to address, but it does make the API a bit more awkward to use, as this test demonstrates:
[<Fact>] let ``Serialize communal table via reflection`` () = let dto = { CommunalTable = Some { Capacity = 42 }; SingleTable = None } let actual = JsonSerializer.Serialize ( dto, JsonSerializerOptions ( PropertyNamingPolicy = JsonNamingPolicy.CamelCase, IgnoreNullValues = true )) """{"communalTable":{"capacity":42}}""" =! actual
You can, of course, define this particular serialization behaviour as a reusable function, so it's not a problem that you can't address. I just wanted to include this, since it's part of the overall work that you have to do in order to make this work.
JSON deserialisation based on types #
To allow parsing of JSON into the above DTO the Reflection-based Deserialize method pretty much works out of the box, although again, it needs to be configured. Here's a passing test that demonstrates how that works:
[<Fact>] let ``Deserialize single table via reflection`` () = let json = """{"singleTable":{"capacity":4,"minimalReservation":2}}""" let actual = JsonSerializer.Deserialize<TableDto> ( json, JsonSerializerOptions ( PropertyNamingPolicy = JsonNamingPolicy.CamelCase )) { CommunalTable = None SingleTable = Some { Capacity = 4; MinimalReservation = 2 } } =! actual
There's only difference in casing, so you'd expect the Deserialize method to be a Tolerant Reader, but no. It's very particular about that, so the JsonNamingPolicy.CamelCase configuration is necessary. Perhaps the API designers found that explicit is better than implicit.
In any case, you could package that in a reusable Deserialize function that has all the options that are appropriate in a particular code context, so not a big deal. That takes care of actually writing and parsing JSON, but that's only half the battle. This only gives you a way to parse and serialize the DTO. What you ultimately want is to persist or dehydrate Table data.
Converting DTO to Domain Model, and vice versa #
As usual, converting a nice, encapsulated value to a more relaxed format is safe and trivial:
let toTableDto = function | SingleTable (NaturalNumber capacity, NaturalNumber minimalReservation) -> { CommunalTable = None SingleTable = Some { Capacity = capacity MinimalReservation = minimalReservation } } | CommunalTable (NaturalNumber capacity) -> { CommunalTable = Some { Capacity = capacity }; SingleTable = None }
Going the other way is fundamentally a parsing exercise:
let tryParseTableDto candidate = match candidate.CommunalTable, candidate.SingleTable with | Some { Capacity = capacity }, None -> Table.tryCommunal capacity | None, Some { Capacity = capacity; MinimalReservation = minimalReservation } -> Table.trySingle capacity minimalReservation | _ -> None
Such an operation may fail, so the result is a Table option. It could also have been a Result<Table, 'something>, if you wanted to return information about errors when things go wrong. It makes the code marginally more complex, but doesn't change the overall thrust of this exploration.
Ironically, while tryParseTableDto is actually more complex than toTableDto it looks smaller, or at least denser.
Let's take stock of the type-based alternative. It requires 26 lines of code, distributed over three DTO types and the two conversions tryParseTableDto and toTableDto, but here I haven't counted configuration of Serialize and Deserialize, since I left that to each test case that I wrote. Since all of this code generally stays within 80 characters in line width, that would realistically add another 10 lines of code, for a total around 36 lines.
This is smaller than the DOM-based code, although at the same magnitude.
Conclusion #
In this article I've explored two alternatives for converting a well-encapsulated Domain Model to and from JSON. One option is to directly manipulate the DOM. Another option is take a more declarative approach and define types that model the shape of the JSON data, and then leverage type-based automation (here, Reflection) to automatically parse and write the JSON.
I've deliberately chosen a Domain Model with some constraints, in order to demonstrate how persisting a non-trivial data model might work. With that setup, writing 'loosely coupled' code directly against the DOM requires 46 lines of code, while taking advantage of type-based automation requires 36 lines of code. Contrary to the Haskell example, Reflection does seem to edge out a win this round.
