Forwarding and tuning of TrackR-CNN

Currently I trained my algorithm on my own dataset. Next I would like to forward and tune the datasets. Nevertheless, I have some doubts about this. @ahnonay @pvoigtlaender

1. What’s the purpose of the forwarding process?
2. How do I tune my training dataset by using a validation set (images+instances)? Because in the ReadMe it’s not given how to use a separate validation set (e.g with the path KITTI_MOTS/data/testing/(instances+images)/(0000,0001,0002)) in order to tune the parameters of the forwarded output (created on the training dataset)?
3. If tuning can only be performed on the training dataset how can the MOTS metrics (MOTSA, MOTSP, sMOTSA) be calculated on the validation set?
4. After tuning, are the weights of the forwarding and tracking somehow updated in order to visualize new tracks?
5. How can I visualize the predicted masks and tracks on the validation dataset by using the output parameters of the tuned model?
6. Which variables in your code belong to the γ, β, δ, α in your paper? In order to adapt the ranges of these parameters within the segtrack_tune_experiment.py to get less id switches. Are these parameters only used for the Forwarding and Tracking step? Or also for the Training step?
7. Which of the above variables are highly affecting the ID switches? Currently I obtain MOTSA values between (-15, -80) when I’m tracking apples (smaller area compared to cars/pedestrians) annotated on 25FPS.

Scores(sMOTSA_car=-95.2, sMOTSA_ped=-inf, MOTSA_car=-85.9, MOTSA_ped=-inf, MOTSP_car=78.9, MOTSP_ped=inf, IDS_car=723, IDS_ped=0)

[('0006', 101), ('0007', 103)] False

Scores(sMOTSA_car=-69.9, sMOTSA_ped=-inf, MOTSA_car=-61.8, MOTSA_ped=-inf, MOTSP_car=79.6, MOTSP_ped=inf, IDS_car=985, IDS_ped=0)

[('0006', 101), ('0007', 103)] False

Thanks in advance!

[pvoigtlaender]

Hi,

1. Forwarding means evaluating the network on the given video sequences and creating the tracking output
2. Can’t you just give the tuning script the path to the validation set instead of the training set?
3. No, tuning can also be done on the validation set, you just need to invoke the tuning script with other parameters. You can calculate the MOTS metrics using the mots tools https://github.com/VisualComputingInstitute/mots_tools
4. The tuning process will determine the best set of hyperparameters, and these are then used to evaluate on the final dataset split (test set)
5. First run forwarding on the validation set to create the output, then use mots tools to create a visualization
6. All the parameters are here: https://github.com/VisualComputingInstitute/TrackR-CNN/blob/master/scripts/segtrack_tune_experiment.py#L90 Unfortunately I don’t have the exact correspondence now
7. reid_weight_car and mask_iou_weight_car should be quite important for id switches. In general one of the most important hyperparameters is the detection confidence threshold (detection_confidence_threshold_car) although it relates less to ID switches

[jong372]

Many thanks! This helps me a lot for implementing it correctly (and based on certain decisions).