Metrics general supplementary checking before `compute`
See original GitHub issueAs discussed here, the following assumption:
All metrics that has not been updated yet after last reset should raise NotComputableError when
compute
is called
holds for any metric.
Today we check metric specific variable _num_examples
if it’s equal zero and raise exception if it is the case. This behaviour can be replaced directly in Metric
by counting the number of update
calls between reset
and before compute
.
An example implementation is proposed by @zasdfgbnm :
class Metric:
def __init__(self, ...):
......
def wrapped_reset():
self._updated = False
self.reset()
self.reset = functools.wraps(self.reset)(wrapped_reset)
def wrapped_update(output):
self._updated = True
self.update(output)
self.update = functools.wraps(self.update)(wrapped_update)
def wrapped_compute():
if not self._updated:
raise NotComputableError('not updated before compute')
return self.compute()
self.compute = functools.wraps(self.compute)(wrapped_compute)
Let’s discuss here what we can do.
cc @zasdfgbnm
Issue Analytics
- State:
- Created 5 years ago
- Comments:19 (11 by maintainers)
Top Results From Across the Web
Supplementary Guidelines | OpenTelemetry
For each metric stream, two measurements are produced covering the same interval of time, which the SDK is expected to aggregate before producing...
Read more >Picard Metrics Definitions - GitHub Pages
ClusteredCrosscheckMetric: A Metric class to hold the result of clustered crosschecking fingerprints. CollectHiSeqXPfFailMetrics.PFFailDetailedMetric: a metric ...
Read more >Recruiting Metrics Cheat Sheet - LinkedIn Business
*Some companies use a benchmark of 3-4 interviewed candidates before a candidate is hired. Recruiting Metrics Cheatsheet. 5 talent.linkedin.com. The candidates ...
Read more >Supplementary information - Nature
General information. Supplementary Information (SI) is peer-reviewed material directly relevant to the conclusion of a paper that cannot be included in the ...
Read more >11 Most Important Website Metrics You Should Track - HostPapa
Metrics can also indicate issues with your website, a particular product web page or general performance. Your website's metrics tell a lot ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@vfdev-5 It is a little bit similar, but for this the operands are classes like
Precision
, and the result are also class. The idea is, ifA
andB
are classes for metrics, thenA + B
is another class for a new metric. Should be something like@vfdev-5 I opened a new issue for the metaclass problem: https://github.com/pytorch/ignite/issues/337