Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failed to load model

See original GitHub issue

I am trying to use memory analyzer for the resnet50_netdef model. It seems that I can start the tritonserver but cannot load the model. What may be the problem here?

kingsleyl@prm-dgx-05:/gpfs/fs1/kingsley/server/docs/examples$ docker run -v /var/run/docker.sock:/var/run/docker.sock -v /gpfs/fs1/kingsley/server/docs/examples/embed_model_repository:/models -v /gpfs/fs1/kingsley/server/docs/examples/results:/results --net=host memory-analyzer:latest --batch 1,2,4 --concurrency 1,2,4 --model-names resnet50_netdef --model-folder /models --export --export-path /results/
Failed to load resnet50_netdef on inference server: skipping model

Server Only:
Model                         Batch               Concurrency         Throughput          Max Memory Util(%)  Max GPU Util(%)     Max BAR1(MB)        Max Framebuffer(MB)
triton-server                 0                   0                   0 infer/sec         0                   0                   8                   308

Models:
Model                         Batch               Concurrency         Throughput          Max Memory Util(%)  Max GPU Util(%)     Max BAR1(MB)        Max Framebuffer(MB)

By the way, I can start the tritonserver with resnet50_netdef model successfully in the environment inside the nvcr.io/nvidia/tritonserver:20.09-py3 container.

Issue Analytics

State:
Created 3 years ago
Comments:5

Top GitHub Comments

1reaction

dyastremskycommented, Oct 29, 2020

Found it! You actually need to use the absolute path to the model directory thrice in the command for now, due to the way Docker is used here. The command would be:

docker run -v /var/run/docker.sock:/var/run/docker.sock -v /gpfs/fs1/kingsley/server/docs/examples/embed_model_repository:/gpfs/fs1/kingsley/server/docs/examples/embed_model_repository -v /gpfs/fs1/kingsley/server/docs/examples/results:/results --net=host memory-analyzer:latest --batch 1,2,4 --concurrency 1,2,4 --model-names resnet50_netdef --model-folder /gpfs/fs1/kingsley/server/docs/examples/embed_model_repository --export --export-path /results/

0reactions

KingsleyLiu-NVcommented, Oct 30, 2020

Thanks a lot for your help. I think this issue can be closed. @dyastremsky