[ Visualization: MuCo/MuPoTS ]

Hi,

Wonderful work, and thanks for sharing the code.

I would like to know If you have provided any code to visualize the MuCo dataset and if I need to transform the coordinates to visualize them like in demo.py (is the input provided from MuCo or MuPoTS dataset)?

• Question I: My goal is to visualize the joints on top of the original image taken from MuCo as keypoints? I tried to do the same thing provided in vis_keypoints() but I do not know why I can not see any joints written on top of the image by OpenCV.

• Question II: the attribute root_cam is the same attribute used in the demo.py as root_depth_list ?

` # Image taken from MuCo Dataset

for i, n in enumerate(centers_id):

    n = int(n[0])
    img_path = dataset[n]["img_path"]
    original_img = cv2.imread(img_path)
    original_img_height, original_img_width = original_img.shape[:2]

    # normalized camera intrinsics
    focal = [1500, 1500] # x-axis, y-axis
    princpt = [original_img_width/2, original_img_height/2]
    img, img2bb_trans = generate_patch_image(original_img, np.array(bbox_imgs[i]), False, 1.0, 0.0, False)
    imgs_cropped.append(img)

    # inverse affine transform (restore the crop and resize)
    pose_3d = dataset[n]["joint_img"]
    pose_3d[:, 0] = pose_3d[:,0] / cfg.output_shape[1] * cfg.input_shape[1]
    pose_3d[:, 1] = pose_3d[:,1] / cfg.output_shape[0] * cfg.input_shape[0]
    pose_3d_xy1 = np.concatenate((pose_3d[:, :2], np.ones_like(pose_3d[:, :1])), 1)
    img2bb_trans_001 = np.concatenate((img2bb_trans, np.array([0, 0, 1]).reshape(1, 3)))
    pose_3d[:, :2] = np.dot(np.linalg.inv(img2bb_trans_001), pose_3d_xy1.transpose(1, 0)).transpose(1,0)[:, :2]
    pose_2d = pose_3d[:, :2].copy()
    list2d.append(pose_2d)

    # root-relative discretized depth -> absolute continuous depth
    pose_3d[:,2] = (pose_3d[:,2] / cfg.depth_dim * 2 - 1) * (cfg.bbox_3d_shape[0]/2) + root_depth_list[0]
    pose_3d = pixel2cam(pose_3d, focal, princpt)
    list3d.append(pose_3d.copy())

    # Visualize 2d poses
    vis_img = original_img.copy()
    for i in range(len(list2d)):
        vis_kps = np.zeros((3, joint_num))
        vis_kps[0, :] = list2d[i][:, 0]
        vis_kps[1, :] = list2d[i][:, 1]
        vis_kps[2, :] = 1
        vis_img = vis_keypoints(vis_img, vis_kps, skeleton)
        name = folder + "/Center_{}_ID_{}.png".format(str(i), str(n))
        cv2.imwrite(name, vis_img)

`

Thanks !

[mks0601]

No, I meant you don’t have to do this: (pose_3d[:,2] / cfg.depth_dim * 2 - 1) * (cfg.bbox_3d_shape[0]/2). This directly add root_depth_list[2] to joint_img[:,2]

[mks0601]

A1. MuCo dataset has its focal lengths. The focal lengths in here are just a normalized focal lengths. I used the normalized ones as images in the wild (e.g., images of MSCOCO datasets) do not provide focal lengths. You should set focal lengths of MuCo dataset for the projection and visualization.

A2. I’m not sure where the root_cam is from, but I guess root_cam is a x-, y-, and z-axis coordinate of human root joint in camera-centered coordinate system. The root_depth_list contains root joint depth of all persons in the input image. Therefore, root_cam[2] is identical to one of root_depth_list.