REST API V2 call is shutting down server

I’m using docker image nvcr.io/nvidia/tritonserver:20.03.1-py3

I created a mobilenetv2 saved model to run with triton (it works under tensorflow serving).

I was trying to call it via api-v2 with the following: (echo -n ‘{ “inputs”: [{ “name” : “input_image”, “shape”: “-1”, “datatype” : “STRING”, “data” : [{“b64” : "’; base64 example-integration-lib/www/assets/cowboyhat.jpg; echo ‘" }] }] }’) |
curl -i -w ‘\nTotal: %{time_total}s\n’ -H “Content-Type: application/json” -d @- http://localhost:8000/v2/models/mobilenetv2/versions/5/infer

The server reports an error and shutdowns: tritonserver: /usr/include/rapidjson/document.h:1471: rapidjson::SizeType rapidjson::GenericValue<Encoding, Allocator>::Size() const [with Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; rapidjson::SizeType = unsigned int]: Assertion `IsArray()’ failed.

here is my config.pbtxt

name: “mobilenetv2”

platform: “tensorflow_savedmodel”

input [ { name: “input_image” data_type: TYPE_STRING dims: [-1] } ]

output [ { name: “label” data_type: TYPE_STRING dims: [5]
}, { name: “confidence” data_type: TYPE_FP32 dims: [5] } ]

version_policy: { all { }}

I can understand if I’m passing a wrong parameter, but the server shouldn’t just shutdown if it can’t parse my request.

[CoderHam]

Thanks for bringing this to our attention @cloudhan. As @deadeyegoodwin said we have hardened the HTTP endpoint and will add testing to ensure it remains tolerant. Regarding your specific issue I can confirm it was fixed in 20.06. If the shape contains negative values the server will catch that error and report it instead of crashing. (Note that we will be making the error message more descriptive.)

Request:

curl --data "{\"inputs\":[{\"name\":\"INPUT0\",\"shape\":[-1,16],\"datatype\":\"INT32\",\"parameters\":{},\"data\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]},{\"name\":\"INPUT1\",\"shape\":[1,16],\"datatype\":\"INT32\",\"parameters\":{}, \"data\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]}]}" http://localhost:8000/v2/models/simple/infer -v

Response:

{"error":"attempt to access JSON non-unsigned-integer as unsigned-integer"}

[deadeyegoodwin]

The HTTP endpoint is significantly hardened in the 20.06 release. But we are going to test your specific failure to see how it is handled.