human-pose-estimation-0004 architecture openpose does not run

The new human-pose-estimation-000(2-4) models do not work when architecture is set to openpose. They run when architecture is set to associative embedding, but the accuracy is very poor - much worse than the human-pose-estimation-0001 model. Given that they have a higher mAP, I think there must be some bug here.

When I run the demo with python human_pose_estimation.py -i video.avi -m human-pose-estimation-0004.xml -d CPU -at openpose, I get the following error:

Traceback (most recent call last):
  File "human_pose_estimation.py", line 285, in <module>
    sys.exit(main() or 0)
  File "human_pose_estimation.py", line 185, in main
    poses, scores = hpes[mode].postprocess(raw_outputs, frame_meta)
  File "/home/walt/workspace/libs/open_model_zoo/demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/model.py", line 209, in postprocess
    poses, scores = self.decoder(heatmaps, nms_heatmaps, pafs)
  File "/home/walt/workspace/libs/open_model_zoo/demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/decoder_openpose.py", line 49, in __call__
    pafs = np.transpose(pafs, (0, 2, 3, 1))
  File "<__array_function__ internals>", line 5, in transpose
  File "/home/walt/.local/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 650, in transpose
    return _wrapfunc(a, 'transpose', axes)
  File "/home/walt/.local/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 61, in _wrapfunc
    return bound(*args, **kwds)
ValueError: axes don't match array

If I manually set the input height to 368 : python human_pose_estimation.py -i video.avi -m human-pose-estimation-0004.xml -d CPU -at openpose --tsize 368 I get the following error:

Traceback (most recent call last):
  File "human_pose_estimation.py", line 285, in <module>
    sys.exit(main() or 0)
  File "human_pose_estimation.py", line 259, in main
    hpes[mode](frame, next_frame_id, {'frame': frame, 'start_time': start_time})
  File "/home/walt/workspace/libs/open_model_zoo/demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/model.py", line 134, in __call__
    self.reshape_net(inputs)
  File "/home/walt/workspace/libs/open_model_zoo/demos/python_demos/human_pose_estimation_demo/human_pose_estimation_demo/model.py", line 71, in reshape_net
    self.net.reshape(input_shapes)
  File "ie_api.pyx", line 1437, in openvino.inference_engine.ie_api.IENetwork.reshape
RuntimeError: Check 'PartialShape::broadcast_merge_into( pshape, node->get_input_partial_shape(i), autob)' failed at ngraph/core/src/op/util/elementwise_args.cpp:49:
While validating node 'v1::Add 2056 (2023[0]:f32{1,21,92,164}, 2055/Interpolate[0]:f32{1,21,96,168}) -> (f32{?,21,92,164})' with friendly_name '2056':
Argument shapes are inconsistent.

[druzhkov-paul]

@renziver Thanks for bringing this up.

A few notes on this:

• Lightweight OpenPose model (human-pose-estimation-0001) is still available, if you find it more accurate for a particular use case, you may still use it.
• Cross domain transferability of models is a tricky topic. Docs are referring to the metrics obtained on MS COCO val AP for the models trained on the training subset of the same dataset. Code for metrics computation and models testing in general is available as a part of Accuracy Checker. Let us know if you find some inconsistencies there.
• AP (a.k.a. MS COCO AP, or average mean average precision, or average precision) accounts errors of many different types. Hence improvement/degradation in different aspects of models’ predictions may contribute differently to overall AP values. So higher AP value don’t guarantee that model is better in all aspect.
• Don’t forget to try different confidence threshold values for different algos.
• In general Associative Embedding like networks have less restrictions on the merge of far away joints into a single pose, which might be the main cause of errors that can be observed in the images you’ve shared. I believe it can be fixed to some extent by putting some heuristics to the post-processing code.

[druzhkov-paul]

@renziver, yes we have plans to release the training code for this kind of models.