Unable to grpcurl to inference.GRPCInferenceService/ServerLive

Description I am trying to establish a health check for a grpc triton application, but I am unable to run grpcurl on the inference.GRPCInferenceService/ServerLive rpc endpoint path. When I try this:

grpcurl -plaintext localhost:8001 inference.GRPCInferenceService/ServerLive

it returns this:

Error invoking method "inference.GRPCInferenceService.ServerLive": failed to query for service descriptor "inference.GRPCInferenceService": server does not support the reflection API

Even running

grpcurl -plaintext localhost:8001 list

results in

Failed to list services: server does not support the reflection API

Is the ServerLive health endpoint supposed to be exposed via grpc reflection? Or is there another way to reach the ServerLive endpoint? Or is there yet another way to get grpc service health?

Triton Information What version of Triton are you using? Triton version 2.8.0 docker image: nvcr.io/nvidia/tritonserver:21.03-py3

[slizb]

Thank you for the feedback! Is there a Dockerfile where I can add these flags to the triton build? I see a Dockerfile.QA, and a Dockerfile.sdk in the root directory of the repo, but its not clear if one of these is more appropriate.

[tanmayv25]

I see a Dockerfile.QA, and a Dockerfile.sdk in the root directory of the repo, but its not clear if one of these is more appropriate.

These docker files don’t build the tritonserver application. Look at the build.py to see how the tritonserver application gets built. You may need to add an additional flag to build the application with grpc reflection.

Specifically, the grpc_endpoint gets built via cmake here: https://github.com/triton-inference-server/server/blob/main/src/servers/CMakeLists.txt#L48

The main application gets built here: https://github.com/triton-inference-server/server/blob/main/src/servers/CMakeLists.txt#L257

As per the links in https://github.com/triton-inference-server/server/issues/3465#issuecomment-944358027, it appears linking grpc++_reflection to the grpc endpoint or main application should do the trick.