Segmentation fault (core dumped) on ensemble model from Triton (GPU) to Python Backend (CPU)

Description Error in ensemble model when postprocessing python backend on cpu from tensorrt GPU input “Segmentation fault (core dumped)”

However, if I send the output from tensorrt to the client, and the resend back to the server it perfectly works. It seems to be something reported in previous issues and not completely solved, at least for tritonserver in jetpack 4.4:

https://github.com/triton-inference-server/python_backend/pull/30

Triton Information triton server 2.6 for jetson. I have built the last python backend which includes this last PR:

https://github.com/triton-inference-server/python_backend/pull/30

Are you using the Triton container or did you build it yourself? I built myself .

To Reproduce

The config.pbtx

name: "ensemblePMG_basque"
platform: "ensemble"
input [
  {
    name: "INPUT0"
    data_type: TYPE_FP32
    dims: [ 1, -1 ]
  }
]

output [
  {
    name: "OUTPUT0"
    data_type: TYPE_STRING
    dims: [ -1 ]
  }
]

ensemble_scheduling {
  step [
    {
      model_name: "preprocess"
      model_version: -1
      input_map {
        key: "INPUT0"
        value: "INPUT0"
      }
      output_map {
        key: "FEATURES"
        value: "FEATURES"
      }
    },
    {
      model_name: "BasqueQ10x5"
      model_version: -1
      input_map {
        key: "FEATURES"
        value: "FEATURES"
      }
      output_map {
        key: "LOGITS"
        value: "LOGITS"
      }
    },
    {
      model_name: "greedy"
      model_version: -1
      input_map {
        key: "LOGITS"
        value: "LOGITS"
      }
      output_map {
        key: "OUTPUT0"
        value: "OUTPUT0"
      }
    }
  ]
}

Last logs before fail :

I0201 09:40:12.665871 14297 infer_request.cc:502] prepared: [0x0x7ee00840f0] request id: 1, model: BasqueQ10x5, requested version: -1, actual version: 1, flags: 0x0, correlation id: 0, batch size: 0, priority: 0, timeout (us): 0
original inputs:
[0x0x7ee0003968] input: FEATURES, type: FP32, original shape: [1,64,464], batch + shape: [1,64,464], shape: [1,64,464]
override inputs:
inputs:
[0x0x7ee0003968] input: FEATURES, type: FP32, original shape: [1,64,464], batch + shape: [1,64,464], shape: [1,64,464]
original requested outputs:
LOGITS
requested outputs:
LOGITS


I0201 09:40:12.665983 14297 python.cc:926] TRITONBACKEND_ModelInstanceExecute: model instance name preprocess_0 released 1 requests
I0201 09:40:12.666197 14297 plan_backend.cc:2322] Running BasqueQ10x5_0_0_gpu0 with 1 requests
I0201 09:40:12.666463 14297 plan_backend.cc:3207] Optimization profile default [0] is selected for BasqueQ10x5_0_0_gpu0
I0201 09:40:12.666887 14297 plan_backend.cc:2721] Context with profile default [0] is being executed for BasqueQ10x5_0_0_gpu0
I0201 09:40:12.689471 14297 infer_response.cc:139] add response output: output: LOGITS, type: FP32, shape: [1,232,37]
I0201 09:40:12.689572 14297 ensemble_scheduler.cc:509] Internal response allocation: LOGITS, size 34336, addr 0xf00e80000, memory type 2, type id 0
I0201 09:40:13.047937 14297 ensemble_scheduler.cc:524] Internal response release: size 34336, addr 0xf00e80000
I0201 09:40:13.048183 14297 infer_request.cc:502] prepared: [0x0x7f34038470] request id: 1, model: greedy, requested version: -1, actual version: 1, flags: 0x0, correlation id: 0, batch size: 0, priority: 0, timeout (us): 0
original inputs:
[0x0x7f3400eca8] input: LOGITS, type: FP32, original shape: [1,232,37], batch + shape: [1,232,37], shape: [1,232,37]
override inputs:
inputs:
[0x0x7f3400eca8] input: LOGITS, type: FP32, original shape: [1,232,37], batch + shape: [1,232,37], shape: [1,232,37]
original requested outputs:
OUTPUT0
requested outputs:
OUTPUT0

I0201 09:40:13.048582 14297 pinned_memory_manager.cc:158] pinned memory deallocation: addr 0x101070090
Segmentation fault (core dumped)

[ivangtorre]

@Tabrizian I will work in a minimal example and upload it on the following days. Thanks for the support

[Tabrizian]

Closing. Reopen if you still saw the issue.