Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dynamically loaded models don't work with ensemble

See original GitHub issue

Description When using dynamically loaded models via the load model API, the ensemble will not pick them up.

I0712 13:09:16.608657 1 model_repository_manager.cc:843] AsyncUnload() 'resize'
I0712 13:09:16.608668 1 model_repository_manager.cc:1136] TriggerNextAction() 'resize' version 1: 2
I0712 13:09:16.608673 1 model_repository_manager.cc:1216] Unload() 'resize' version 1
I0712 13:09:16.608674 1 model_repository_manager.cc:1223] unloading: resize:1
I0712 13:09:16.608698 1 model_repository_manager.cc:843] AsyncUnload() 'test'
E0712 13:09:16.608702 1 model_repository_manager.cc:1551] Invalid argument: ensemble test contains models that are not available: resize
I0712 13:09:16.608707 1 model_repository_manager.cc:713] VersionStates() 'resize'

Triton Information

docker container 22.06-py3

To Reproduce

file:1/dali,py

import nvidia.dali as dali
import nvidia.dali.types as types
from nvidia.dali.plugin.triton import autoserialize


@autoserialize
@dali.pipeline_def(batch_size=3, num_threads=1, device_id=0)
def pipe():
    images = dali.fn.external_source(device="gpu", name="DALI_INPUT_0")
    images = dali.fn.resize(images, resize_x=1280, dtype=types.FLOAT)
    return images

resize/config.json

{
  "name": "resize",
  "version_policy": {
    "latest": {
      "num_versions": 1
    }
  },
  "max_batch_size": 256,
  "input": [
    {
      "name": "DALI_INPUT_0",
      "data_type": "TYPE_UINT8",
      "dims": [
        "-1",
        "-1",
        "3"
      ]
    }
  ],
  "output": [
    {
      "name": "DALI_OUTPUT_0",
      "data_type": "TYPE_FP32",
      "dims": [
        "-1",
        "-1",
        "3"
      ]
    }
  ],
  "model_warmup": [
    {}
  ],
  "backend": "dali"
}

ensemble/config.json

{
  "name": "test",
  "platform": "ensemble",
  "version_policy": {
    "latest": {
      "num_versions": 1
    }
  },
  "input": [
    {
      "name": "images",
      "data_type": "TYPE_UINT8",
      "dims": [
        "1",
        "-1",
        "-1",
        "3"
      ]
    }
  ],
  "output": [
    {
      "name": "resized",
      "data_type": "TYPE_FP32",
      "dims": [
        "1",
        "-1",
        "-1",
        "3"
      ]
    }
  ],
  "ensemble_scheduling": {
    "step": [
      {
        "model_name": "resize",
        "model_version": "-1",
        "input_map": {
          "DALI_INPUT_0": "images"
        },
        "output_map": {
          "DALI_OUTPUT_0": "resized"
        }
      }
    ]
  }
}

Using:

client.load_model("resize", config=<resize.json>, files={"file:1/daly.py": <daly.py>})
client.load_model("test", config=<ensemble.json>, files={"file:1/trigger": b""})

Expected behavior To just load the ensemble model as a version of resize was loaded already

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:8 (5 by maintainers)

Top GitHub Comments

1reaction

kthuicommented, Jul 20, 2022

Filed a ticket for us to investigate further.

0reactions

GuanLuocommented, Jul 20, 2022

E0715 07:37:27.810673 1 model_repository_manager.cc:1571] failed to load model ‘test’: failed to open directory /tmp/folderV5JZli

Seems like there is something wrong when setting up the ensemble model directory, will investigate

Top Results From Across the Web

Angular 5 - load modules (that are not known at compile time ...

The router exposes config property, holding its current config, and a resetConfig(routes: Routes) method for reseting configuration. You can ...

Process.Modules does not list dynamically loaded assemblies ...

For a diagnostic tool I'm writing, I need to check if a target process has loaded a given assembly. To do so, I...

Dynamic Parallelism in TorchScript - PyTorch

A good way to demonstrate how these work is by way of an example: ... The modules in the # ensemble are run...

Dynamic Load Modules - IBM

Dynamic load modules provide the following functions: Load, refresh, and delete installation load modules, which are not part of the IBM® base JES2...

Angular coding style guide

... default export for its file which facilitates lazy loading with the router. ... from '@angular/platform-browser-dynamic'; interface Hero { id: number; ...