Using InternalMetadata class in Armeria

Hi all.

As some of you may have seen, Armeria team mantains a Java implementation of gRPC that runs on top of the Armeria server framework, a generic HTTP/1+2 server framework built on Netty (made by our good friend @trustin himself 😃 )

We interface with protoc stubs by creating Call objects and starting them - after started, a Call goes through io.grpc business logic as normal in most cases with surprisingly few gotchas. The Call interface is a quite nice plugin point.

https://github.com/line/armeria/blob/master/grpc/src/main/java/com/linecorp/armeria/server/grpc/ArmeriaServerCall.java

We have one major caveat though, no support for the Metadata class

https://github.com/line/armeria/blob/master/grpc/src/main/java/com/linecorp/armeria/server/grpc/GrpcService.java#L77

While Armeria has its own methods for adding response trailers, we’re finding more and more users would like to be able to use it so their existing gRPC business logic can run as-is on Armeria, and if possible I’d like to achieve that. It means I need to use InternalMetadata to be able to create and serialize Metadata.

Is it kosher for Armeria to use this class? I understand it is for “specifically supported transport packages” - if Armeria could be considered a specifically supported transport package, that would be great! But if it’s not possible to have official support like that, if it’s “at your own risk but should be ok” that’d be fine too. For context, the first version of our implementation used a lot of io.grpc.internal classes to try to minimize code duplication - that was naturally a nightmare to maintain and I’m hoping not to repeat my mistake 😉

Also, as an alternative or in addition, any thoughts on making Metadata an interface and letting implementations control the serialization themselves? If that happened, we wouldn’t need to use InternalMetadata either.

Thanks!

References

• One of several slack threads
• https://github.com/line/armeria/issues/1788
• https://github.com/line/armeria/pull/1790

[anuraaga]

Thanks for this discussion. Indeed, newMetadata and serialize are the two important methods (I used the base64 encoder in the linked PR but it’s very easy to remove that).

For reference, Armeria uses the unsafe word to indicate APIs that are advanced and dangerous / hard to use but still supported. I borrowed the practice from protobuf. It might make sense here if going with exposing Stable-but-not-for-normal-use APIs. Aside from Metadata, I’m sure some grpc-java users would appreciate an unsafe way to access aliased bytestrings without copies.

I did come to the same conclusion that it is possible to use the public APIs by converting all header names into Key but it’s very obtuse, and probably very slow even if escape analysis kicks in. So hoping to avoid that 😃

As far as API suggestions, some that come to mind are

• I was surprised that Metadata didn’t handle making values ASCII and the transport layer does it (notably base encoding binary values). Maybe it’s because while the gRPC wire format defines Custom-Metadata to only have ASCII values, the class Metadata maybe doesn’t have that restriction. But I was sort of expecting it to.

• It was unclear whether newMetadata should be passed all headers or only ones that don’t start with : or grpc-. Based on the spec I would expect the latter but couldn’t find that logic in the grpc-java transports. In a fit API it would be nice if that’s clearer.

• Having an API that returns an iterator of both raw header names and values would solve a lot. Armeria would use it when serializing but I think users could also find it useful. Is there a way for them right now to log Metadata in a generic way without keys (trash keys work but are quite hard to reason about)?

• Could investigate the real value of using byte[][] instead of String[]. APIs like an iterator become much more practical if doing so. I’d expect it to have similar performance on Java 11 but it’s true that Java 8 might suffer.

• The above points become moot since Armeria would have to implement all Metadata logic, but I think it could make a lot of sense if Metadata became an interface with the public APIs and the implementation class stayed internal. Then transports could implement it as they see fit (e.g., it might make sense to back Metadata with Netty’s Http2Headers itself)

Hope this info helps. Let me know if I can help with anything else. If we decide on a direction I’d be happy to send a PR for it too.

[ejona86]

We probably need to make defensive copy of the args for outside users.

Sounds good. And that will mirror serialize().

The names of the methods should probably indicate that it’s meant for transports, not so much end users.

I thought we’d put “unsafe” in the names, and then documentation describe it is really only usable by transports. I don’t think I want to start creating many Transport* classes in the stable API; I’d rather it not show up that high-level in the javadoc. Unless it was a single UnsafeMethods/TransportMethods class which exposed many such methods.

Were you thinking using a separate class to hide it from the majority of Bazel users?