question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Best practice for including external .proto files in container build images

See original GitHub issue

I am kind of uncertain what to do when referencing *.proto from outside the project. Example:

<Protobuf Include="..\..\protos\example.proto" GrpcServices="Client" />

My Dockerfile is included in the project directory src\example-client\Dockerfile, and the docker build context is src\example-client. I’m trying to avoid changing the docker context to the root of the mono repo. The build will fail because docker cannot reference files outside of the build context (aka, COPY ../../protos .). I tried searching around and came accross issue #183, but that seems like it would result in the same issue just with a nuget reference.

Some examples show manually copying/pasting the ..\..\protos\example.proto to src\example-client\Protos\example.proto and updating the include to:

<Protobuf Include="Protos\example.proto" GrpcServices="Client" />

In order to avoid copy/pasting manually I updated my build to include:

<Target BeforeTargets="PreBuildEvent" Condition="'$(Configuration)' != 'Release'" Name="CopyProtos">
  <Copy SourceFiles="..\..\protos\example.proto" DestinationFolder="Protos" ContinueOnError="true" />
</Target>

It worked, but not ideal. I looked at how gRPC clients are handled with Go, and it looks like the generated *.pb.go and *_grpc.pb.go files are generally just checked into source control. So I modified my build to do this:

<ItemGroup Condition="'$(Configuration)' != 'Release'">
  <Protobuf Include="..\..\protos\example.proto" OutputDir="proto"
            GrpcServices="Client" CompileOutputs="false" />
</ItemGroup>

This worked and felt like the right approach, until I read it might be an anti-pattern.

The tooling package Grpc.Tools is required to generate the C# assets from *.proto files. The generated assets (files):

  • Are generated on an as-needed basis each time the project is built.
  • Aren’t added to the project or checked into source control.
  • Are a build artifact contained in the obj directory.

– <cite>C# Tooling support for .proto files</cite>

So is this actually an anti-pattern in this instance, or are other teams doing similar approaches? Are there better solutions?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
JunTaoLuocommented, Jan 5, 2021

The reason we recommend generating the classes is recommended is that it is kept in sync with the proto files. Checking in the generated files is fine if you regenerate those files each time the proto files are changed. This could be accidentally missed or become a burden which is why we recommend against it. However, if you need it in your scenario, that approach is acceptable too.

0reactions
chwarrcommented, Jan 5, 2021

You’ll also want to regenerate the .cs files if protoc changes (e.g., a different version is used), not just on changes to the .proto files. That’s a dependency that often gets missed when writing build system rules, and I’ve fixed this bug a number of times myself. 😀

Also, keep in mind that protoc and the ProtoBuf library need to be kept in sync with each other. New code gen may rely on newer APIs in the library and vice-versa. (This may not be strictly necessary in all cases, but mismatches have caused problems in the past.)

Read more comments on GitHub >

github_iconTop Results From Across the Web

protocol buffers - Sharing proto files in docker?
Generate the back docker container first, then use multi-stage build to pull the .proto files from the back container to the front container....
Read more >
Best practices for building docker images for microservices?
Just use a git submodule and expose interface descriptors via a separate git repo. This way both your server and client side can...
Read more >
Running a gRPC Service with Docker
Multistage build​​ One of Docker's best practice is keeping the image size small, by having only the binary file then we make our...
Read more >
API Best Practices | Protocol Buffers Documentation
API Best Practices · Precisely, Concisely Document Most Fields and Messages · Use Different Messages for Wire and Storage · For Mutations, Support...
Read more >
How to Compile Protobufs When You Have Friends Who ...
One way to fix this is to switch to a hermetic build system with protobuf support, like Bazel. But you'd have to rewrite...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found