Can't get the shape right
See original GitHub issueDescription
So were using an EfficientNet model and off the shelf its config looks like this (note the input dimensions):
{
"name": "super-grouper",
"platform": "tensorflow_savedmodel",
"backend": "tensorflow",
"version_policy": {
"latest": {
"num_versions": 1
}
},
"max_batch_size": 0,
"input": [
{
"name": "image_tensor",
"data_type": "TYPE_FP32",
"format": "FORMAT_NONE",
"dims": [
-1,
456,
456,
3
],
"is_shape_tensor": false,
"allow_ragged_batch": false
}
],
"output": [
{
"name": "logits",
"data_type": "TYPE_FP32",
"dims": [
-1,
152
],
"label_filename": "",
"is_shape_tensor": false
}
],
"optimization": {
"priority": "PRIORITY_DEFAULT",
"input_pinned_memory": {
"enable": true
},
"output_pinned_memory": {
"enable": true
}
},
"instance_group": [
{
"name": "super-grouper",
"kind": "KIND_GPU",
"count": 1,
"gpus": [
0
],
"profile": []
}
],
"default_model_filename": "model.savedmodel",
"cc_model_filenames": {},
"metric_tags": {},
"parameters": {},
"model_warmup": []
}
Triton Information
Triton client 2.2.0
To Reproduce
So i can’t seem to get the input shape so that it would be accepted. First i tried a batch of 8 images following the image_client.py example i np.stack()'ed them before sending:
tritonclientutils.InferenceServerException: got unexpected numpy array shape [8, 456, 456, 3], expected [-1, 456, 456, 3]
then i tried to using np.expand_dims
first as this worked in v1, still no joy
… got unexpected numpy array shape [8, 1, 456, 456, 3], expected [-1, 456, 456, 3]
reduced the batch to a single image (our use case anyway)
… got unexpected numpy array shape [1, 456, 456, 3], expected [-1, 456, 456, 3]
Expected behavior
One of these methods to work!?!?!?
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (3 by maintainers)
Thanks @tanmayv25. I released that once I set the
max_batch_size
> 0 it removes the -1 dim. for me in the configuration JSON (which I was using for setting the input dims. Thanks again!Sure here is my client for you client:
I refactored my client more in line with one of your examples and I’m getting correct inference when the model has
max_batch_size=0
and only 3 dims. But I can’t get inference for models with a -1 batch dimension even after settingmax_batch_size > 0
with a -1 4th dim. Always getting back from the server:HA! I did manage to get a batch though but the model config has
max_batch_size=0
with `dims: [-1 456 456 3] i force the batch size with the client to 8? I guess because the model is handling the batching and not Triton. So that makes sense.