How do I create properly formatted input_data_file for warmup

I’ve read over the docs, model_config.proto, as well as the warmup tests, and I don’t quite understand the file format for input_data_file.

I’m using the tensorflow backend, latest release of triton-inference-server. My config.pbtxt is as follows:

  platform: "tensorflow_savedmodel"
  max_batch_size: 0
  input [
    {
      name: "a"
      data_type: TYPE_STRING
      dims: [ 1 ]
      reshape: { shape: [ 1 ] }
    },
    {
      name: "b"
      data_type: TYPE_STRING
      dims: [ 1 ]
      reshape: { shape: [ 1 ] }
    },
    {
      name: "c"
      data_type: TYPE_STRING
      dims: [ 1 ]
      reshape: { shape: [ 1 ] }
    },
        {
      name: "d"
      data_type: TYPE_BOOL
      dims: [ 1 ]
      reshape: { shape: [ 1 ] }
    }
  ]
  output [
    {
      name: "output_0"
      data_type: TYPE_INT32
      dims: [ 1 ]
      reshape: { shape: [ 1 ] }
    }
  ]
  model_warmup [
    {
        name: "warmup_data"
        batch_size: 1
        inputs: {
            key: "a"
            value: {
                data_type: TYPE_STRING
                dims: [ 1 ]
                input_data_file: "raw_a"
            }
        }
        inputs: {
            key: "b"
            value: {
                data_type: TYPE_STRING
                dims: [ 1 ]
                input_data_file: "raw_b"
            }
        }
        inputs: {
            key: "c"
            value: {
                data_type: TYPE_STRING
                dims: [ 1 ]
                input_data_file: "raw_c"
            }
        }
        inputs: {
            key: "d"
            value: {
                data_type: TYPE_BOOL
                dims: [ 1 ]
                input_data_file: "raw_d"
            }
        }
    }
]

I’m struggling with how to create a raw file in the necessary format. I tried the following:

import numpy as np
np.array(["string_for_arg_a"]).tofile("raw_a")

But when I run tritonserver, I get the following error:

E0124 00:21:54.463405 1261 triton_model_instance.cc:86] warmup error: Invalid argument - unexpected number of string elements 2 for inference input 'a', expecting 1

[lminer]

I worked! Thank you.

[tanmayv25]

@lminer You must write the serialized item to the file and not the ndarray object itself.

Try the following:

import numpy as np
from tritonclient.utils import serialize_byte_tensor


  serialized = serialize_byte_tensor(
      np.array(["string_for_arg_a".encode("utf-8")], dtype=object)
  )
  with open("raw_a", "wb") as fh:
    fh.write(serialized.item())

Read more about the utility and what it returns from here https://github.com/triton-inference-server/client/blob/main/src/python/library/tritonclient/utils/__init__.py#L187