Generate purely typescript interfaces / plain JSON representation

Unfortunately, discussions aren’t available for this repo and i’m not sure if this feature is out of scope or not.

Basically, I noticed that for large protocols, protobuf bundle size can go to upwards of 50kB. Considering that JSON serialization/deserialization is also accelerated by the browser’s native code, I feel like it may be more beneficial for us to use purely the JSON representation of the protobuf for RPCs.

Is it possible for protobuf-ts to support generating exclusively typescript interfaces and using JSON.parse / JSON.stringify for serialization? This could significantly lower bundle size for web apps.

[timostamm]

This is actually already available! At least the core functionality. It is not documented and the API is not streamlined at all, but it works.

import { Interpreter } from '@protobuf-ts/plugin/build/interpreter';
import { DescriptorRegistry, FileDescriptorSet } from '@protobuf-ts/plugin-framework';
import { LongType, ScalarType, UnknownMessage } from '@protobuf-ts/runtime';
import { readFileSync } from 'fs';

// for the input, we need file descriptors:
// you can generate a file descriptor set with: protoc msg-scalar.proto --descriptor_set_out msg-scalar.bin
// or write them by hand in the JSON format
const buffer = readFileSync("./msg-scalar.bin");

// we need a descriptor registry:
const fileDescriptorSet = FileDescriptorSet.fromBinary(buffer);
const registry = DescriptorRegistry.createFrom(fileDescriptorSet.file[0]);

// now we create the interpreter.
// it can create message types dynamically from the descriptors in the registry.
const interpreter = new Interpreter(registry, {
    normalLongType: LongType.STRING,
    oneofKindDiscriminator: 'oneofKind',
    synthesizeEnumZeroValue: 'UNSPECIFIED$',
});


// this is how you get a dynamic message type
const fooType = interpreter.getMessageType('spec.ScalarValuesMessage');

// you can use it just like the generated messsage types,
// but of course there will be no interface for the message available.
// there is a special type for those messages, the `UnknownMessage`:
const foo: UnknownMessage = fooType.create();

// you can use reflection to work with the message
for (let field of fooType.fields) {
    console.log("field:", field.name, "value:", foo[field.localName]);
    if (field.kind === 'scalar' && field.T === ScalarType.INT32) {
        foo[field.localName] = 123;
    }
}

// you can use all message type methods:
const json = fooType.toJsonString(foo);
console.log(json)

This means it is possible to create something like proto-loader with protobuf-ts. There just needs to be a piece of code that reads proto loader json files and turns them into file descriptors.

[timostamm]

I do have an idea to reduce code size further. At the moment, all functionality is packed into MessageType and can’t be shaken off if you don’t need it.

Moving the methods to standalone functions would mean that webpack output for example would not contain code for the JSON format if you only use the binary format.

This opens up several design questions though. I think this might be thing for v3…