Load ONNX model

I managed to export some models from the model zoo into ONNX format. However, I have difficulties getting it to work with torchreid. In torchtools.py, instead of torch.load(), I added checkpoint = onnx.load(fpath). This resulted in the following error:

File "yolov5_deepsort\reid models\deep-person-reid\torchreid\utils\torchtools.py", line 280, in load_pretrained_weights
    if 'state_dict' in checkpoint:
TypeError: argument of type 'ModelProto' is not iterable

Any advice?

[mikel-brostrom]

Good news @KaiyangZhou, @Rm1n90, @HeChengHui!

I have a working multibackend (ONNX, OpenVINO and TFLite) class for for the ReID models that I manged to export (mobilenet, resnet50 and osnet models) with my export script. My export pipeline is as follows: PT --> ONNX --> OpenVINO --> TFLite. osnet models fails in the OpenVINO export; mobilenet and resnet50 models go all the way through. Feel free to experiment with it, it is in working condition as shown by my CI pipeline. Don’t forget to drop a PR if you have any improvements! 😄

[mikel-brostrom]

Did you time the model in ONNX, OPENVINO and TFLITE to see how long will take the tracking to do the job compare to pytorch version?

Inference time for the different frameworks is highly dependent on which HW you run it on. The chosen export frameworks should be deployment-plaform specific.

• ONNX is an all-around format for all type of CPUs
• OpenVINO should be the way to go for Deep Learning inference on Intel CPUs, Intel integrated GPUs, and Intel Vision Processing Units (VPUs)
• TFLITE is for mobile and IoT devices