Proto2 `required` fields that contain default values are not serialized correctly

My proto is as follows

message Update {
    repeated Prop prop = 1;
    message Prop {
        oneof prop_update_type {
            StandardProp standard_prop = 2;
            RegularProp regular_prop = 5;
        }
    message StandardProp {
        optional string id = 1;
        optional string title = 2;
    }

    message RegularProp {
        required string name = 1;
        required bool is_array = 2;
        optional Value value = 3;
    }
}
message Value {
    oneof value {
        string string_value = 1;
        bool boolean_value = 2;
        int32 integer_value = 3;
        int64 long_value = 4;
        float float_value = 5;
        double double_value = 6;
    }
}

The Update object protoMarshal() is sent across Kafka and on the consumer side, I am getting exceptions which are on java platform. The exceptions are different in different projects.

Note: I am getting exception only when Prop contains RegularProp.

Below listed bullet points are the exception messages I am getting in java projects which deserialize using Update.parseFrom(data)

Project - 1 Value cannot cast to Update Project - 2 prop[0].regular_prop.is_array not present

Is this due to the presence of Value in RegularProp?

The same proto is used by other Kafka producers in different projects (Javascript, java, etc.) and working fine. So to repeat the exception is only for Kafka message sent from Kotlin to other platform and only when using RegularProp

[garyp]

@sujithkris This seems like a bug in pbandk. That’s why it works in Java 🙂 pbandk isn’t doing the correct thing for proto2 required fields that contain default values.

At least that’s my hypothesis. If you could share the binary data, that’d help me confirm the hypothesis. But I should be able to also write a quick unit test to test this scenario.

[garyp]

@sujithkris I was able to reproduce the bug with a unit test. Unfortunately it doesn’t look like pbandk ever properly supported required proto2 fields (despite the documentation), so this is going to be a bit more involved to fix than I expected. It won’t make the 0.9.0 release (which should be this or next week), but we’ll try to get it in for the 0.10.0 release.

Also, to clarify, the bug only impacts proto2 required fields that contain default values (as you have in your example up above). proto2 optional fields with default values and proto3 fields with default values should be working correctly.