Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

GRPC server always return `RawOutputContents` even if the `InferInputTensor::contents` is specified.

See original GitHub issue

Description GRPC server always return RawOutputContents even if the InferInputTensor::contents is specified.

Triton Information r21.03 container

To Reproduce In src/clients/go/grpc_simple_client.go, change the input from RawInputContents to InferInputTensor::contents and just run the sample.

Expected behavior InferOutputTensor::Contents should be returned.

Is this behavior by design or a bug?

Issue Analytics

State:
Created 2 years ago
Comments:5 (4 by maintainers)

Top GitHub Comments

1reaction

tanmayv25commented, Jun 2, 2021

The return by raw output contents is by design for performance consideration. The parsing of data to contents field will be an expensive operation on the upstream. Hence, tritonserver will directly return the output as a raw data blob. Provided the blob, user can always parse out the data into individual elements.

@zouyxdut Do you have a specific use case where you require output to be returned as explicit contents field? Or was there any indication that using InferInputTensor::contents would return results as InferOutputTensor::Contents in triton docs? If latter is true, we can fix the docs to prevent giving the wrong idea.

0reactions

tanmayv25commented, Jun 7, 2021

Closing the issue… Please file a new issue if you would like to request for output to be returned as explicit array within InferOutputTensor::Contents.