Defining a Skylark py_proto_library via proto_library

Given https://github.com/bazelbuild/bazel/issues/2626, I was wondering whether it would be possible to add a pure Skylark py_proto_library built on proto_library.

I’ve played around with rules and had a look at how rules_go does this with toolchains (https://github.com/bazelbuild/rules_go/pull/803/files). These basically involve you maintaining an additional dependency tree (which proto_library is supposed to solve) and having your custom py_proto_library applied across this tree. The most promising approach to avoiding this seems to be to use aspects.

It’s pretty clear how aspects could be used to invoke protoc to generate the _pb2.py files. What’s less clear is how to use these outputs, which are unknown ahead of time and come from the proto_library transitive dep provider, in conjunction with the native py_library rule (I think this is the sandwich problem?).

Any guidance here from the Bazel proto or Python teams?

[dayfine]

As far as I’ve tried, none of these py_proto_library works to my satisfaction (so are some other rules that I’ve tried, like go_proto_library). My expectation for these rules come from google3, and it’s frustrating the bazel versions don’t quite work the same way.

What I would expect with a lang_proto_library and which is not currently how these rules work:

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
    deps = [
        "bar_proto",
        "@com_google_protobuf//:timestamp_proto",
    ],
)

py_proto_library(
    name = "foo_py_pb2",
    deps = [":foo_proto"],
)

proto_library(
    name = "bar_proto",
    srcs = ["bar.proto"],
)

py_proto_library(
    name = "bar_py_pb2",
    deps = [":bar_proto"],
)

# Using foo_py_pb2 without fuss
py_library(
    deps = [":foo_py_pb2"],
)

Basically:

1. lang_proto_library only needs foo_proto in dependencies, not other bar_py_pb2. The Go rule and some flavors of the python rule do not work like this.
2. When using the proto foo in python, I only need foo_py_pb2 as the dependency of the users, i.e. the target would include all its transitive dependencies. The grpc version of py_proto_library fails with this.

[awbraunstein]

What’s the Bazel team’s philosophy on rules like the py_proto_library rule being requested here. Currently there are implementations from the protobuf team, grpc team, as well as other users such as http://github.com/stackb/rules_proto mentioned above. Is the end goal for the bazel team to own these rules? Or should they be owned by some other entity?

As a user of these rules it becomes difficult to understand which one is the best/right one to use since they all tend to function in the same way (except some use the proto_libaray and some don’t.)