Heatmaps Do not look as expexted????

Hello,

I have trained ROMP on the pw3d dataset for 150 epochs and from SCRATCH. Here are my hyperparameters:

ARGS:
 tab: 'V1_hrnet' 
 dataset: 'pw3d'
 GPUS: 0,1,2,3
 distributed_training: False
 model_version: 1
 match_preds_to_gts_for_supervision: True

 master_batch_size: -1
 val_batch_size: 16
 batch_size: 16
 epoch: 150
 nw: 4
 nw_eval: 2
 lr: 0.00005
 exp: 'ROMP'              # experiment name
 save_dir: './results'           # Path to results directory

 fine_tune: True
 fix_backbone_training_scratch: False
 eval: True
 supervise_global_rot: False

 model_return_loss: True
 collision_aware_centermap: True
 collision_factor: 0.2
 homogenize_pose_space: True
 shuffle_crop_mode: True
 shuffle_crop_ratio_2d: 0.1
 shuffle_crop_ratio_3d: 0.4

 merge_smpl_camera_head: False
 head_block_num: 2

 backbone: 'hrnet'
 centermap_size: 64
 centermap_conf_thresh: 0.2

 model_path: None

loss_weight:
 MPJPE: 200.
 PAMPJPE: 360.
 P_KP2D: 400.
 Pose: 80.
 Shape: 6.
 Prior: 1.6
 CenterMap: 160.

sample_prob:
 pw3d: 1

The model seems to be well converged and here is the loss function:

However, when I look at the Center-map heatmaps it doe s not look good at all:

I expected to get a plot similar to below that I obtained by fine-tuning the ROMP with HRNet-32 as the backbone (V1_hrnet_3dpwft.sh):

Any thought or advice on why this is the case and what I further need to do to improve the center map heatmap?

Thanks in advance,

[Arthur151]

Hi, really sorry about that I missed this issue.

The 2D pose and body center heatmap are estimated by two individual heads. We need to supervise the estimated body center heatmap with the ground truth heatmap We calculate the ground truth body center location in heatmap using the ground truth 2d pose for supervising the estimated body center heatmap. The reason why we also use an individual head to estimate 2D pose heatmaps is that previous research (Taskonomy, best paper of cvpr18) testified that learning from related task is helpful for better representation learning. And we find that it is an important part to learn in the pretraining process.Thanks for your interests in ROMP.

[mkhoshle]

@Arthur151 Nice. Thanks so much.