Is it possible to deserialize unknown keys?

I have currently an issue where I need to be able to deserialize JSON that has a defined structure but also allows for arbitary extensions. So I need to be able to work with unknown keys and unknown types. Now I would like to somehow store those unknown keys with their values and later, when needed serialize them into their original form again.

My first thought was to create some map property in which all the unknown keys and their values are put. When serializing that map I probably need to use a JsonTransformingSerializer since I want to keep the original structure and without that the map would be serialized as object.

The problem of how to store unknown keys and their values remain. I searched a lot but I could not find an answer to that. Does there exist a feature that deals with unknown keys or do I need to write a custom deserializer for that and what would an example of that look like dealing with unknown keys? Or do I need to approach this problem in a different way?

At the moment this is the approach I am experimenting with: My test class: @Serializable data class testSerializationVC (val knownKey: String, val elements: JsonObject) My test json: jsonToParse = """{"knownKey":"knownKey", "knownKey":"knownKey", "elements": {"arbitraryValue1":"json", "arbitraryValue2": 1, "arbitraryValue3": "arbitraryValue3"}}""" Created Object: testSerializationVC(knownKey=knownKey, elements={"arbitraryValue1":"json","arbitraryValue2":1,"arbitraryValue3":"arbitraryValue3"}) Output of serialization: {"knownKey":"knownKey", "elements":{"arbitraryValue1":"json","arbitraryValue2":1,"arbitraryValue3":"arbitraryValue3"}}

My goal is to parse JSON like: jsonToParse = """{"knownKey":"knownKey", "arbitraryValue1":"json", "arbitraryValue2": 1, "arbitraryValue3": "arbitraryValue3"}""" And deserilze the object back into: {"knownKey":"knownKey", "arbitraryValue1":"json", "arbitraryValue2": 1, "arbitraryValue3": "arbitraryValue3"}

[matax87]

Thanks to @pdvrieze suggestions I was able to accomplish the same challenge asked from @MMairinger. Please note that I’m using version 0.20.0. You’ll have to use JsonDecoder/JsonEncoder instead of JsonInput/JsonOutput in order to make it work on version 1.0.0.

Example:

@Serializable(with = FiltersConfigSerializer::class)
data class FiltersConfig(
    val name: String? = null,
    val age: Int? = null,
    val from: Date? = null,
    val unknownFilters: Map<String, JsonElement>? = null
)

@Serializable
data class Date(val timestamp: Long)

@Serializer(forClass = FiltersConfig::class)
internal object FiltersConfigSerializer : KSerializer<FiltersConfig> {

    private val stringToJsonElementSerializer = MapSerializer(String.serializer(), JsonElement.serializer())

    override val descriptor: SerialDescriptor = stringToJsonElementSerializer.descriptor

    override fun deserialize(decoder: Decoder): FiltersConfig {
        // Decoder -> JsonInput
        require(decoder is JsonInput) // this class can be decoded only by Json
        val json = decoder.json
        val filtersMap = decoder.decode(stringToJsonElementSerializer)

        val name = filtersMap["name"]?.let {
            json.fromJson(String.serializer(), it)
        }
        val age = filtersMap["age"]?.let {
            json.fromJson(Int.serializer(), it)
        }
        val from = filtersMap["from"]?.let {
            json.fromJson(Date.serializer(), it)
        }
        val knownKeys =
            setOf("name", "age", "from")
        val unknownFilters = filtersMap.filter { (key, _) -> ! knownKeys.contains(key) }

        return FiltersConfig(name, age, from, unknownFilters)
    }

    override fun serialize(encoder: Encoder, value: FiltersConfig) {
        // Encoder -> JsonOutput
        require(encoder is JsonOutput) // This class can be encoded only by Json
        val json = encoder.json
        val map: MutableMap<String, JsonElement> = mutableMapOf()

        value.name?.let { map["name"] = json.toJson(String.serializer(), it) }
        value.age?.let { map["age"] = json.toJson(Int.serializer(), it) }
        value.from?.let { map["from"] = json.toJson(Date.serializer(), it) }
        value.unknownFilters?.let { map.putAll(it) }

        encoder.encode(stringToJsonElementSerializer, map)
    }
}

[pdvrieze]

By far the simplest approach to do this would be to have a custom deserializer that does the following:

It first delegates to a serializer of a map of string,JsonElement values. For the known keys you then invoke the appropriate deserializers from the jsonElement and store them in some temporary. The unknown keys go into some map of unknown values. You create the instance of the result class and return it.

Note that it requires you delegate the descriptor to be that of a map (with the correct serialkind)

To serialize you do the same thing in reverse, you first serialize all the properties into a map (you serialize to an in-memory structure, not the encoder passed as function parameter). Then you add all the “unknown” values to the same map. Finally you serialize the map (against the encoder).

In short, you do nested serialization into/from an intermediate representation.