Config.pbtx for Efficientdet-D0

Description I am deploying detection model Efficientdet-D0 via tensorflow savedmodel, and facing some issues while deploying the model using nvcr.io/nvidia/tritonserver:20.10-py3. It is about the dimension of output tensor, it is [1, -1, 7] and triton-server expects -1 in first dimension, I also tried to make it [-1, 7] as well, but it is still not working.

My config.pbtxt file

name: "EfficientDet"
platform: "tensorflow_savedmodel"
max_batch_size: 1
input [
   {
      name: "image_arrays:0"
      data_type: TYPE_UINT8
      format: FORMAT_NHWC
      dims: [-1, -1, 3]
   }
]
output [
   {
      name: "detections:0"
      data_type: TYPE_FP32
      dims: [1, -1, 7 ]
   }
]

I am using docker container nvcr.io/nvidia/tritonserver:20.10-py3

You can get saved_model from here : [https://tfhub.dev/tensorflow/efficientdet/d0/1]

I expected config.pbtxt to work with output tensor having dimensions of [1, 7].

[deadeyegoodwin]

Please read this section of the documentation carefully: https://github.com/triton-inference-server/server/blob/master/docs/model_configuration.md#inputs-and-outputs

Setting max_batch_size > 0 has a specific requirements on your model inputs and outputs. You should also consider not providing a config.pbtxt and using --strict-model-config=false and see what Triton generates for you: https://github.com/triton-inference-server/server/blob/master/docs/model_configuration.md#auto-generated-model-configuration

[tanmayv25]

@AkashDharani It looks like the format of saved mode is TF2.0 saved model. Can you try invoking the server with --backend-config=tensorflow,version=2 ?