Expected Performence

Thanks for your work to implement BMN on PyTorch!

May I ask, what is the model performance that you get?

On my side, the AUC is 64.5% and AR@100 is 73.5. Please find the details below.

I did not change any parameters before running the code. Minor typos were fixed.

+ python main.py --module Evaluation                                                                                                                                        
[INIT] Loaded annotations from validation subset.                                                                                                     
        Number of ground truth instances: 7293                                                                                                                              
        Number of proposals: 472628                                                                                                                   
        Fixed threshold for tiou score: [0.5  0.55 0.6  0.65 0.7  0.75 0.8  0.85 0.9  0.95]                                                                                 
[RESULTS] Performance on ActivityNet proposal task.                                                                                                   
        Area Under the AR vs AN curve: 64.50481283422461%                                                                                                                   
/home/xum/anaconda3/envs/GTAL/lib/python3.7/site-packages/matplotlib/figure.py:98: MatplotlibDeprecationWarning:                                      
Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version,                                                     a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior                               ensured, by passing a unique label to each axes instance.                                                                                                                   
  "Adding an axes using the same arguments as a previous axes "                                                                                       
AR@1 is          0.30287947346770877                                                                                                                                        
AR@5 is          0.43831070889894425                                                                                                                  
AR@10 is         0.5157136980666392                                                                                                                                         
AR@100 is        0.7352392705333882

[yklilfft]

Hello, I just read PaddlePaddle code the BMN author released, I found the details of model are not really same as paper. I tried to copy his hyperparameters in this pytorch version, but still cannot get performance like PaddlePaddle version( best AR@100 is about 74.5 ,AUC is nearly 65%). I notice that in his version, the BM map generation function is more complex, maybe it’s the reason.

If u are familiar with PaddlePaddle framework, you can try to learn about this code: https://github.com/PaddlePaddle/models/tree/develop/PaddleCV/PaddleVideo/models/bmn

[JJBOY]

nn.Conv3d(self.hidden_dim_1d, self.hidden_dim_3d, kernel_size=(self.num_sample, 1, 1))

the kernel size is 32, so whatever the stride is, the process is same.

Now I upload my pretrained model. You can find the link in the new README and download from Baidu Cloud.

My performace is here: train subset video numbers: 9649 validation subset video numbers: 4728 python main.py --mode inference BMN training loss(epoch 0): tem_loss: 1.176, pem class_loss: 0.361, pem reg_loss: 0.020, total_loss: 1.741 BMN training loss(epoch 0): tem_loss: 1.151, pem class_loss: 0.348, pem reg_loss: 0.019, total_loss: 1.686 BMN training loss(epoch 1): tem_loss: 1.119, pem class_loss: 0.335, pem reg_loss: 0.018, total_loss: 1.633 BMN training loss(epoch 1): tem_loss: 1.140, pem class_loss: 0.335, pem reg_loss: 0.017, total_loss: 1.647 BMN training loss(epoch 2): tem_loss: 1.097, pem class_loss: 0.325, pem reg_loss: 0.017, total_loss: 1.591 BMN training loss(epoch 2): tem_loss: 1.139, pem class_loss: 0.361, pem reg_loss: 0.019, total_loss: 1.694 BMN training loss(epoch 3): tem_loss: 1.082, pem class_loss: 0.317, pem reg_loss: 0.016, total_loss: 1.561 BMN training loss(epoch 3): tem_loss: 1.138, pem class_loss: 0.350, pem reg_loss: 0.019, total_loss: 1.679 BMN training loss(epoch 4): tem_loss: 1.070, pem class_loss: 0.310, pem reg_loss: 0.016, total_loss: 1.540 BMN training loss(epoch 4): tem_loss: 1.130, pem class_loss: 0.327, pem reg_loss: 0.016, total_loss: 1.618 BMN training loss(epoch 5): tem_loss: 1.057, pem class_loss: 0.303, pem reg_loss: 0.015, total_loss: 1.513 BMN training loss(epoch 5): tem_loss: 1.146, pem class_loss: 0.331, pem reg_loss: 0.016, total_loss: 1.640 BMN training loss(epoch 6): tem_loss: 1.018, pem class_loss: 0.279, pem reg_loss: 0.014, total_loss: 1.435 BMN training loss(epoch 6): tem_loss: 1.126, pem class_loss: 0.329, pem reg_loss: 0.016, total_loss: 1.611 BMN training loss(epoch 7): tem_loss: 0.998, pem class_loss: 0.271, pem reg_loss: 0.013, total_loss: 1.404 BMN training loss(epoch 7): tem_loss: 1.128, pem class_loss: 0.324, pem reg_loss: 0.015, total_loss: 1.607 BMN training loss(epoch 8): tem_loss: 0.991, pem class_loss: 0.264, pem reg_loss: 0.013, total_loss: 1.386 BMN training loss(epoch 8): tem_loss: 1.132, pem class_loss: 0.333, pem reg_loss: 0.016, total_loss: 1.622 (qinxin) zenghao@amax-22:~/BMN$ python main.py --mode inference /home/zenghao/anaconda3/envs/qinxin/lib/python3.6/site-packages/pandas/compat/_optional.py:106: UserWarning: Pandas requires version ‘1.2.1’ or newer of ‘bottleneck’ (version ‘1.2.0’ currently installed). warnings.warn(msg, UserWarning) validation subset video numbers: 4728 Post processing start Post processing finished [INIT] Loaded annotations from validation subset. Number of ground truth instances: 7293 Number of proposals: 472616 Fixed threshold for tiou score: [0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95] [RESULTS] Performance on ActivityNet proposal task. Area Under the AR vs AN curve: 67.4776498011792% /home/zenghao/anaconda3/lib/python3.6/site-packages/matplotlib-3.0.2-py3.6-linux-x86_64.egg/matplotlib/figure.py:98: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance. "Adding an axes using the same arguments as a previous axes " AR@1 is 0.3355957767722474 AR@5 is 0.497737556561086 AR@10 is 0.5695324283559577 AR@100 is 0.7539695598519128