Check failed: output->dims() == Input(i).dims() (detectron model -> caffe2 model -> onnx model)

hi! I run ./tools/convert_pkl_to_pb.py to convert detectron model to caffe2 model successfully. Then I run the code to convert caffe2 model to onnx model:

import onnx
import caffe2.python.onnx.frontend
from caffe2.proto import caffe2_pb2

# We need to provide type and shape of the model inputs,
# see above Note section for explanation
data_type = onnx.TensorProto.FLOAT
data_shape = (1, 3, 720, 1280)
value_info = {
    'data': (data_type, data_shape)
}

predict_net = caffe2_pb2.NetDef()
with open('model.pb', 'rb') as f:
# with open('predict_net.pb', 'rb') as f:
    predict_net.ParseFromString(f.read())


init_net = caffe2_pb2.NetDef()
with open('model_init.pb', 'rb') as f:
# with open('init_net.pb', 'rb') as f:
    init_net.ParseFromString(f.read())

onnx_model = caffe2.python.onnx.frontend.caffe2_net_to_onnx_model(
    predict_net,
    init_net,
    value_info,
)

onnx.checker.check_model(onnx_model)

I encounter the issue: RuntimeError: [enforce fail at operator.cc:213] op. Cannot create operator of type 'BatchPermutation' on the device 'CPU'. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing. Operator def: input: "roi_feat_shuffled" input: "rois_idx_restore_int32" output: "roi_feat" name: "" type: "BatchPermutation" device_option { } engine:

Is it because that BatchPermutation belongs to detectron? I try to add these code:

from caffe2.python import dyndep
detectron_ops_lib = '/home/user/pytorch/build/lib/libcaffe2_detectron_ops_gpu.so'
dyndep.InitOpsLibrary(detectron_ops_lib)

The issue disappear. But I encounter the new issue:

RuntimeError: [enforce fail at utility_ops.h:275] . Check failed: output->dims() == Input(i).dims().Description: Input #1, input dimension:[1, 256, 46, 80] should match output dimension: [1, 256, 45, 80]
Error from operator: 
input: "fpn_inner_res4_5_sum_lateral" input: "fpn_inner_res4_5_sum_topdown" output: "fpn_inner_res4_5_sum" name: "" type: "Sum" device_option { } engine: ""frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*) + 0x76 (0x7ff40d388b86 in /home/user/.local/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libc10.so)

Why the input dimension and output dimention are different? Could you please help me? Thanks!

[gangulypritha]

@pkuxwguan Hi, What do you mean by adding “im_info” information to blob? The image size which I used was (1,3,800,800) and caffe2.python.onnx.frontend.caffe2_net_to_onnx_model(…) function returns a RuntimeError :

" RuntimeError: [enforce fail at generate_proposals_op.cc:243] im_info_tensor.sizes() == (vector<int64_t>{num_images, 3}). [0] vs 1 3 Error from operator: input: “rpn_cls_probs_cpu” input: “rpn_bbox_pred_cpu” input: “im_info” input: “anchor” output: “rpn_rois” output: “rpn_roi_probs” name: “” type: “GenerateProposals” arg { name: “nms_thresh” f: 0.7 } arg { name: “min_size” f: 0 } arg { name: “spatial_scale” f: 0.0625 } arg { name: “correct_transform_coords” i: 1 } arg { name: “post_nms_topN” i: 1000 } arg { name: “pre_nms_topN” i: 6000 }frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*) + 0x68 (0x7f786ea37d78 in /usr/local/lib/libc10.so)

[fuzzyBatman]

@lilichu

I change the input data to (1, 3, 800, 800) as you say, it is ok. But I also encounter the same issue:

RuntimeError: [enforce fail at generate_proposals_op.cc:243] im_info_tensor.sizes() == (vector<int64_t>{num_images, 3}). [0] vs 1 3
Error from operator: 
input: "rpn_cls_probs_fpn2" input: "rpn_bbox_pred_fpn2" input: "im_info" input: "anchor2" output: "rpn_rois_fpn2" output: "rpn_roi_probs_fpn2" name: "" type: "GenerateProposals" arg { name: "nms_thresh" f: 0.7 } arg { name: "min_size" f: 0 } arg { name: "spatial_scale" f: 0.25 } arg { name: "correct_transform_coords" i: 1 } arg { name: "post_nms_topN" i: 1000 } arg { name: "pre_nms_topN" i: 1000 } device_option { } engine: ""

I get that same error with ONNX and also when I try to run the caffe2 model on my Intel CPU.