question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Verification or VR@FAR10^6 score on megaface

See original GitHub issue

How do you get Verification or VR@FAR10^6 score on megaface? I run command like:

python run_experiment.py -s 1000000 \
    path_megaface_feature path_facescrub_feature _arcface.bin path_results/

And I got output with pretrained-R50E (cleaned)

Done matching! Score matrix size: 3530 1000000
Saving to /home/data/insight/megaface/raw_eval/r50/results_cm/otherFiles/facescrub_megaface_r50_112x112_1000000_1.bin
Computing test results with 1000000 images for set 1
Loaded 3530 probes spanning 80 classes
Loading from /home/data/insight/megaface/raw_eval/r50/results_cm/otherFiles/facescrub_facescrub_r50_112x112.bin
Probe score matrix size: 3530 3530
distractor score matrix size: 3530 1000000
Done loading. Time to compute some stats!
Finding top distractors!
Done sorting distractor scores
Making gallery!
Done Making Gallery!
Allocating ranks (1000080)
Rank 1: 0.965815

I carefully check all output log, and I couldn’t find anything seems to be Verification score. Is there any other command that can produce Verification or VR@FAR10^6 score? @nttstar @bruinxiong or anyone who is willing to offer help. Thank you.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:7

github_iconTop GitHub Comments

6reactions
nttstarcommented, Jun 13, 2018

In ‘roc’ block, you can choose the corresponding value near FAR1e-6. The 4th line in your case is ok which is 0.9695510268211365

1reaction
nttstarcommented, Jun 12, 2018

Check the generated cmc_*.json file.

Read more comments on GitHub >

github_iconTop Results From Across the Web

1 Million Faces for Recognition at Scale - MegaFace
Figure 1. The MegaFace challenge evaluates identification and verification as a function of increasing number of gallery distractors (going.
Read more >
MegaFace Benchmark (Face Verification) - Papers With Code
Rank Model Accuracy Year 1 Prodpoly 98.95% 2020 2 ElasticFace‑Arc 98.81% 2021 3 ArcFace + MS1MV2 + R100 + R 98.48% 2018
Read more >
Validate with MegaFace Challenge dataset #275 - GitHub
I think testing result on LFW is not enough because 2 reasons: LFW test ... face recognition, both Identification and Verification problem.
Read more >
The MegaFace Benchmark: 1 Million Faces for Recognition at ...
We present both identification and verification performance, ... We report results of state of the art and baseline algorithms.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found