Modification to F# discriminated unions implementation to support treating Fields as a Record

Modification to F# discriminated unions implementation to support treating Fields as a Record

Serialization of F# discriminated unions in the current Json.NET implementation and in the option presented in issue #1547 result in unfavorable output in my opinion. Below I will list problems, propose solutions, and present a semi-trivial case with sample code to support these opinions. I have already begun work on this issue here.

Unfavorable result 1 - Current implemtation

Example of F# code and resultant Json from the current implementation

type DUSerializationTest =
    | CaseA of id:int * name:string * dateTimeStamp:DateTime
    | CaseB of string * int
    | CaseC of description:string * DUSerializationTest

let serializeTest =
    CaseA (1, "Carl", DateTime.Now)
    |> JsonConvert.SerializeObject

let serializeTest2 =
    CaseB ("CaseB Item1 Example", 2)
    |> JsonConvert.SerializeObject

let serializeTest3 =
    CaseC ("Recursive Type Definition Example", CaseB ("CaseB Item1", 2))
    |> JsonConvert.SerializeObject

{
    "Case": "CaseA",
    "Fields": [
        1,
        "Carl",
        "2018-03-28T00:54:08.0652638+00:00"
    ]
}

{
    "Case": "CaseB",
    "Fields": [
        "CaseB Item1 Example",
        2
    ]
}

{
    "Case": "CaseC",
    "Fields": [
        "Recursive Type Definition Example",
        {
            "Case": "CaseB",
            "Fields": [
                "CaseB Item1",
                2
            ]
        }
    ]
}

I find this to be functional but not usable in scenarios where interoperability or discovery is important. Having a simple array of “Fields” as a series of ordered values without labels does not give any kind of hint of intent to the consumer.

Furthermore, it is my experience that when looking at API documentation or working with 3rd parties (who sometimes just send sample Json payloads as documentation) it is common practice to use a tool like json2csharp to generate classes at least as a basis for implementing the interaction. Doing this on the output detailed above results in the following class definition.

public class RootObject
{
    public string Case { get; set; }
    public List<object> Fields { get; set; }
}

As you can see the List<object> Fields property leaves us with more responsibility being placed on the consuming developer to get the implementation correct based on documentation. It would be preferable to have Json that would result in generated code that is named and explictly typed.

Unfavorable result 2 - Issue #1547 proposed struct-based implementation

While issue #1547 presents a good potential solution to the 1st unfavorable result I believe there are three problems which should prevent this library from adopting it.

Issue 1 - Current usage

The current implementation of serialization of Discriminated Unions by Json.NET is already published and available for common use; this means utilizing the existing [Struct] common attribute of Discriminated Unions would mean that developers who have implemented and potentially persisted [Struct] attributed unions in data stores would unexpectedly and unexplainably (without researching) have to deal with their programs not behaving properly or data loss.

Issue 2 - Limited to [Struct]

Outside of the obvious desire one would have to minimize restrictions on functionality the [Struct] attribute limits the ability to have a recursive type definition.

Issue 3 - “Case” becomes a reserved word with no compiler warning

The following F# would result in two instances of “case” being used within the Json output.

[<Struct>]
type DUSerializationCaseTest =
    | CaseA of case:string * dateTimeStamp:DateTime
    | CaseB of Case:string * DateTimeStamp:DateTime

let serializeTest =
    CaseA ("Not ideal", DateTime.Now)
    |> JsonConvert.SerializeObject

let serializeTest2 =
    CaseB ("Problem", DateTime.Now)
    |> JsonConvert.SerializeObject

Example 1 - technically valid because of the case-sensitivity of Json which, because of the common F# convention to use camelCasing in union case fields, could have a low impact on the majority of users.

{
    "Case": "CaseA",
    "case": "Not ideal",
    "dateTimeStamp": "2018-03-28T00:54:08.0652638+00:00"
}

Example 2 - invalid, duplicate key

{
    "Case": "CaseA",
    "Case": "Problem",
    "DateTimeStamp": "2018-03-28T00:54:08.0652638+00:00"
}

Proposed Solution

Begin by solving the “current usage” problem listed above by leaving the default functionality and adding an attribute which can be used to decorate Discriminated Unions so that they may be treated differently when serializing and deserializing.

public class DiscriminatedUnionFieldsAsRecordAttribute : Attribute { }

Modify the output of serializing a Discriminated Union decorated in this fashion in the following ways:

• Leave the “Case” and “Fields” top level keyword paradigm but change “Fields” to “Record”. This is beneficial to isolate and differentiate the label:value pairs of the union case from the wrapper identifiers.
• Treat the fields of the Case being serialized as a record. This is beneficial for interoperability, discovery, and readability purposes. Additionally, the implementation of the deserialization of the “Record” does not have to follow a strict ordering of values since we can match on the labels and create the union case more safely.

Example of proposed changes

[<DiscriminatedUnionFieldsAsRecordAttribute>]
type DUSerializationTest =
    | CaseA of id:int * name:string * dateTimeStamp:DateTime
    | CaseB of string * int
    | CaseC of description:string * DUSerializationTest

let serializeTest =
    CaseA (1, "Carl", DateTime.Now)
    |> JsonConvert.SerializeObject

Results in the following Json

{
    "Case": "CaseA",
    "Record": {
        "id": 1,
        "name": "Carl",
        "dateTimeStamp": "2018-03-28T00:54:08.0652638+00:00"
    }
}

Example of json2csharp output when these changes have been applied

public class Record
{
    public int id { get; set; }
    public string name { get; set; }
    public DateTime dateTimeStamp { get; set; }
}

public class RootObject
{
    public string Case { get; set; }
    public Record Record { get; set; }
}

As I hope you will agree, the results of the proposed modification yield better results.

Sample Usage

This concept uses a Discriminated Union to model events for an ordering system. The goal is that these events can be folded into a state representation of an “Order” which a different system will display, modify, and return changes as a series of events. A benefit of utilizing a Discriminated Union for our model here is that we get to enforce completeness when doing operations on these events (e.g. folding events to state).

Model of events

type OrderEventData = {id:Guid; dateTimeStamp:DateTime; orderId:string; reason:string}

[<DiscriminatedUnionFieldsAsRecordAttribute>]
type OrderEvents =
  | OrderCreated of orderEvent:OrderEventData
  | CustomerChanged of orderEvent:OrderEventData * customerId:string
  | ItemAdded of orderEvent:OrderEventData * itemId:string * row:Guid
  | ItemDeleted of orderEvent:OrderEventData * row:Guid
  | ItemQuantityChange of orderEvent:OrderEventData * row:Guid * newQuantity:decimal
  | ItemPriceChange of orderEvent:OrderEventData * row:Guid * newPrice:decimal

With sample data serialized from above as “documentation” it becomes easy to run that through something like json2csharp and do some light renaming / plumbing to get a useful base for interoperability.

Sample “documentation” (note how the “Case” gives us info on the “Record” naming we’ll use below)

{
    "Case": "CustomerChanged",
    "Record": {
        "orderEvent": {
            "id": "626d7012-acb0-4eec-99dd-072e2dd34e6d",
            "dateTimeStamp": "2018-03-28T00:54:08.061706+00:00",
            "orderId": "12345",
            "reason": "user selected"
        },
        "customerId": "10080"
    }
}

{
    "Case": "ItemAdded",
    "Record": {
        "orderEvent": {
            "id": "8d15144d-bf40-4c15-b0a7-2f5ec77f8847",
            "dateTimeStamp": "2018-03-28T00:54:08.0621004+00:00",
            "orderId": "12345",
            "reason": "user selected"
        },
        "itemId": "80511",
        "row": "9ca04207-2c77-4d3a-9d38-1f6f3a632fb9"
    }
}

Generated and modified C# which could be created entirely from the context of the above “documentation” though I will admit I think it would be important to point out to a consumer that the “Case” value is significant.

    //Create some base abstract types to support the event model
    public abstract class EventBase { }
    public abstract class EventRecordBase { }


    //Limited this example to two of the six cases defined in the F# code since this will take up many lines
    public class OrderEventMetadata
    {
        public string id { get; set; }
        public DateTime dateTimeStamp { get; set; }
        public string orderId { get; set; }
        public string reason { get; set; }
    }

    public class CustomerChangedEvent : EventBase
    {
        public OrderEventMetadata orderEvent { get; set; }
        public string customerId { get; set; }
    }

    public class ItemAddedEvent : EventBase
    {
        public OrderEventMetadata orderEvent { get; set; }
        public string itemId { get; set; }
        public string row { get; set; }
    }


    //Define the wrapper for events in support of the Case / Record root object
    public abstract class TypedEventRecord<T> : EventRecordBase where T : EventBase
    {
        public TypedEventRecord() => Case = GetCase();

        protected abstract string GetCase();

        public string Case { get; set; }
        public T Record { get; set; }
    }

    public class CustomerChangedRecord : TypedEventRecord<CustomerChangedEvent>
    {
        protected override string GetCase() => "CustomerChanged";
    }

    public class ItemAddedRecord : TypedEventRecord<ItemAddedEvent>
    {
        protected override string GetCase() => "ItemAdded";
    }

    //Create a factory for taking in events and producing event records
    public static class EventRecordFactory
    {
        public static EventRecordBase CreateEventRecord(EventBase record)
        {
            if (record.GetType() == typeof(CustomerChangedEvent))
            {
                return new CustomerChangedRecord()
                {
                    Record = (CustomerChangedEvent)record
                };
            }
            if (record.GetType() == typeof(ItemAddedEvent))
            {
                return new ItemAddedRecord()
                {
                    Record = (ItemAddedEvent)record
                };
            }

            throw new ArgumentException("Record is not a valid supported event type.", "record");
        }
    }

[DaveEmmerson]

DUs having poor serialization is a pain. I’ve been looking at how to work around it, but didn’t think about a PR to here. It’s a good idea.

Your use case actually looks similar to mine - want to be able to sensibly serialize events.

+1 from me

[bartelink]

👍 Contributions are definitely welcome in FsCodec.

In general it’s trying to be minimal, and have a story about STJ vs NSJ interop (which makes the full semantics in the OP probably a stretch).

Having said that, if there is an NSJ converter that is likely to be useful in multiple codebases, I don’t necessarily have a problem with it living there, even without an equivalent STJ side implementation.

But, most dearly of all, I’d love to see that glorious exception you showed, (which saved you, and people following in your footsteps from a random ill thought out behavior) could also be implemented for NSJ!