Reproduce Performance Discussion

Thx for the nice job. However I downloaded the code and trained the model, but the results in the paper were not well reproduced.

Setting

Dataset: Cityscapes Train with train split, 2975 images. Evaluate with val split. Follow all details in this repo. Train models with different max_iterations (60000 as default setting in this repo.)

Results in paper

Result

model	Max Iter	mIoU
Resnet101-RCCA(R=2)	40000	75.85%
Resnet101-RCCA(R=2)	60000	76.81%
Resnet101-RCCA(R=2)	100000	76.36%
Resnet101-PSP	40000	76.92%
Resnet101-PSP	60000	76.85%
Resnet101-PSP	100000	76.90%

Env

pytorch 0.4.0 torchvision 0.2.1 4*TITAN XP

Is there any tricks in the implementation?

[EthanZhangYi]

@sydney0zq @speedinghzl @lxtGH @HqWei @mingminzhen I trained models with the latest code, and here are the results.

model	ID	mIoU
Resnet101-RCCA(R=2)	1	78.32%
Resnet101-RCCA(R=2)	2	78.89%
Resnet101-RCCA(R=2)	3	77.05%
Resnet101-PSP	1	77.95%
Resnet101-PSP	2	78.30%
Resnet101-PSP	3	78.47%

The result of PSPNet in paper is 78.5%, which is well reproduced. The result of CCNet in paper is 79.8%, which is NOT reproduced.

checklist:

• Reproduce the result of CCNet.
• Find the reason of the repruduction failure at the very start of this issue.
• The performance is not stable. (I guess the reason is that IoUs for classes of little samples such as bus, train are not stable, since the samples are heavily unbalanced )

Env

Python version : 3.6.3 Pytorch Version : 0.4.0 Cuda : 9.0 Cudnn : 7.0 Nccl: 2.1.15 GCC : 4.8.5 GPU : Titan XP

[EthanZhangYi]

@speedinghzl Thanks for your reply. Hoping you will release a model exactly trained with this repo. Thanks