Python backend returning wrong results converting from NHWC to NCHW

Description I’ve created a model with python backend doing simple preprocessing (substract and transpose)

Model.pbtxt looks like this:

name: "face_detector_preprocess"
backend: "python"

input [
  {
    name: "image"
    data_type: TYPE_UINT8
    dims: [ -1, 320, 320, 3]
  }
]
output [
  {
    name: "preprocessed_image"
    data_type: TYPE_FP32
    dims: [ -1, 3, 320, 320 ]
  }
]
instance_group [{
    count: 1
    kind: KIND_CPU }]

The execute method in the model.py looks like this:

    def execute(self, requests):
        output_dtype = self.output_dtype
        responses = []
        for request in requests:
            input_ = pb_utils.get_input_tensor_by_name(request, "image")
            in_img = torch.from_numpy(input_.as_numpy())
            in_img = torch.sub(in_img, torch.Tensor([104, 117, 123]))
            in_img = in_img.permute(0,3,1,2)
            out = in_img.numpy().astype(dtype=np.float32)
            output = pb_utils.Tensor("preprocessed_image", out.astype(output_dtype))
            inference_response = pb_utils.InferenceResponse(output_tensors=[output])
            responses.append(inference_response)
        return responses

Running over an example image I get a result that looks like this:

Doing this same step outside of Triton gives me a result as expected:

The problem I find is in the permute step I send my image as NWHC and I want to return them as NCHW so then I can just chain this as an ensemble model (the next model uses NCHW). If I comment: in_img = in_img.permute(0,3,1,2) The image is OK but in NWHC order so I don’t understand what could be happening because doing the same without triton server works fine.

Triton Information 21.04

I’m using a Triton container which only adds numpy and torch:

FROM nvcr.io/nvidia/tritonserver:21.04-py3

RUN pip install numpy torch

[Tabrizian]

I created a PR to fix this. Meanwhile, you can use np.ascontiguousarray before creating the Tensor object to avoid this issue.

[ajaichemmanam]

Similar issue for me. NCHW image NHWC image

model config

name: "pre_process" backend: "python" input[ { name: "PREPROCESS_INPUT0" data_type: TYPE_UINT8 format: FORMAT_NHWC dims: [-1, -1, 3] } ] output [ { name: "PREPROCESS_OUTPUT0" data_type: TYPE_FP32 dims: [-1, 3, 512, 512] } ] output [ { name: "PREPROCESS_OUTPUT1" data_type: TYPE_UINT8 dims: [-1, -1, -1, 3] } ] instance_group [{ kind: KIND_CPU }] I used numpy for processing the image

for request in requests: img = pb_utils.get_input_tensor_by_name(request, "PREPROCESS_INPUT0").as_numpy() img_x = cv2.resize(img, (512,512)) nchw = img_x.transpose(2, 0, 1).reshape(1, 3, 512, 512).astype(np.float32) nhwc = np.expand_dims(img, 0) out_tensor_0 = pb_utils.Tensor("PREPROCESS_OUTPUT0", nchw) out_tensor_1 = pb_utils.Tensor("PREPROCESS_OUTPUT1", nhwc) inference_response = pb_utils.InferenceResponse(output_tensors=[out_tensor_0, out_tensor_1]) responses.append(inference_response)

and on Client side

NCHW_image = results.as_numpy('PREPROCESS_OUTPUT0').reshape(1,3,512,512) CHW = np.squeeze(NCHW_image) print(CHW.shape) CHW = CHW.transpose(1, 2, 0).astype(np.uint8) cv2.imwrite('./NCHW.jpg', CHW) NHWC_image = results.as_numpy('PREPROCESS_OUTPUT1') #-1, -1, -1, 3 HWC = np.squeeze(NHWC_image) cv2.imwrite('./NHWC.jpg', HWC)

Sorry I couldn’t properly format the code in github. Confirmed these issue on triton 21.04 and 21.05. Same code runs without any issues on 21.03