question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Inference with Pretrained Models error

See original GitHub issue

Expected results

image

Actual results

What did you observe instead? image


E0212 10:57:39.824651  1567 init_intrinsics_check.cc:54] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0212 10:57:39.824681  1567 init_intrinsics_check.cc:54] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0212 10:57:39.824687  1567 init_intrinsics_check.cc:54] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
WARNING cnn.py:  40: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
INFO net.py:  57: Loading weights from: /tmp/detectron-download-cache/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl
I0212 10:58:21.947926  1567 net_dag_utils.cc:118] Operator graph pruning prior to chain compute took: 0.000193389 secs
I0212 10:58:21.948194  1567 net_dag.cc:61] Number of parallel execution chains 63 Number of operators = 402
I0212 10:58:21.970666  1567 net_dag_utils.cc:118] Operator graph pruning prior to chain compute took: 0.000247387 secs
I0212 10:58:21.970948  1567 net_dag.cc:61] Number of parallel execution chains 30 Number of operators = 358
I0212 10:58:21.973315  1567 net_dag_utils.cc:118] Operator graph pruning prior to chain compute took: 2.0227e-05 secs
I0212 10:58:21.973356  1567 net_dag.cc:61] Number of parallel execution chains 5 Number of operators = 18
INFO infer_simple.py: 111: Processing demo/15673749081_767a7fa63a_k.jpg -> /tmp/detectron-visualizations/15673749081_767a7fa63a_k.jpg.pdf
terminate called after throwing an instance of 'caffe2::EnforceNotMet'
  what():  [enforce fail at context_gpu.h:105] status == CUDNN_STATUS_SUCCESS. 1 vs 0. , Error at: /var/lib/jenkins/workspace/caffe2/core/context_gpu.h:105: CUDNN_STATUS_NOT_INITIALIZED Error from operator:
input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernel" i: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "pad" i: 3 } arg { name: "order" s: "NCHW" } arg { name: "stride" i: 2 } device_option { device_type: 1 cuda_gpu_id: 0 } engine: "CUDNN"
*** Aborted at 1518433102 (unix time) try "date -d @1518433102" if you are using GNU date ***
PC: @     0x7f11fd7b1428 gsignal
*** SIGABRT (@0x61f) received by PID 1567 (TID 0x7f1170168700) from PID 1567; stack trace: ***
    @     0x7f11fd7b14b0 (unknown)
    @     0x7f11fd7b1428 gsignal
    @     0x7f11fd7b302a abort
    @     0x7f11f772884d __gnu_cxx::__verbose_terminate_handler()
    @     0x7f11f77266b6 (unknown)
    @     0x7f11f7726701 std::terminate()
    @     0x7f11f7751d38 (unknown)
    @     0x7f11fdb4d6ba start_thread
    @     0x7f11fd88341d clone
    @                0x0 (unknown)
Aborted (core dumped)

Detailed steps to reproduce

I am using caffe2 + Detecron docker image but when I am trying to Inference with Pretrained Models I got these errors. Caffe2 latest Docker image using GPU support:

nvidia-docker run -it caffe2ai/caffe2:latest /bin/bash

Build the image:

cd $Detectron/docker
docker build -t detectron:c2-cuda9-cudnn7 .

make new docker

nvidia-docker run -it detectron1 detectron:c2-cuda9-cudnn7 /bin/bash

Install the COCO API

git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI
make install

build the python modules

cd $Detectron/lib && make && cd ..

Making Inferences

python2 tools/infer_simple.py \
    --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
    --output-dir /tmp/detectron-visualizations \
    --image-ext jpg \
    --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
    demo

System information

  • Operating system: Ubuntu 16.04
  • CUDA version: 9.0
  • cuDNN version: 7.0
  • GPU : TitanX

Issue Analytics

  • State:open
  • Created 6 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
ir413commented, Feb 12, 2018

Hi @moh3th1, I don’t quite follow the steps you’re taking in order to use the Detectron docker image.

As described in INSTALL.md, to build the Detectron docker image it is sufficient to:

cd $DETECTRON/docker
docker build -t detectron:c2-cuda9-cudnn7 .

You can then run the demo using:

nvidia-docker run --rm -it detectron:c2-cuda9-cudnn7 python2 tools/infer_simple.py \
    --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
    --output-dir /tmp/detectron-visualizations \
    --image-ext jpg \
    --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
    demo
0reactions
remcovacommented, Oct 9, 2018

Same error here, does someone have a solution for this?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Trying: Inference with pretrained models, getting the error ...
I am trying to test the "Inference with pre-trained models", on the MMaction documentation. I am getting the error below:.
Read more >
Inference with pretrained models
Inference with pretrained models. We provide testing scripts to evaluate a whole dataset (Cityscapes, PASCAL VOC, ADE20k, etc.), and also some high-level ...
Read more >
Inference on your own videos using pre-trained models
Hi there, trying to run the inference.py script from this tutorial. . I tried on both my PC and a server, but I'm...
Read more >
Saving and loading models for inference in PyTorch
There are two approaches for saving and loading models for inference in PyTorch. The first is saving and loading the state_dict , and...
Read more >
Using PyTorch Models with Elastic Inference
Tracing models may lead to tensor creation on a specific device. When this happens, you may get errors when loading the model onto...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found