nan uncertainty estimates
See original GitHub issueFor certain types of overlapping distributions, the uncertainty estimate returned by BAR can benan
%matplotlib inline
import numpy as np
import pymbar
from matplotlib import pyplot as plt
for _ in range(10):
fwd = np.random.normal(0, 10.0, size=50000)
rev = np.random.normal(0, 175.0, size=50000)
print(pymbar.BAR(fwd, rev))
Returns
(0.03461197183239051, nan)
(0.01789748600329588, nan)
(-0.041729050167845116, 0.006057615238226341)
(-0.0008426040270812507, nan)
/home/yutong/venv37/lib/python3.7/site-packages/numpy/core/_methods.py:48: RuntimeWarning: overflow encountered in reduce
return umr_sum(a, axis, dtype, out, keepdims, initial, where)
(0.011851492025385824, nan)
(-0.011125827676117694, nan)
(-0.0011024556901020333, 0.006058115351639581)
(-0.14948909238990282, 0.006102804127491066)
(0.03077049043731961, 0.006088478465760845)
(-0.10509980343390524, nan)
The distributions look like:
Edit: G0 and G1 below is wrong, confused the distributions, there is probably no issue with the bias!
Futhermore, since the analytical partition functions are Z(sig)=1/(sig*sqrt(2pi))
, so using G=-ln(Z)
the analytical free energies are G0=3.221
and G1=6.083
, The estimator also appears to be biased.
Issue Analytics
- State:
- Created 2 years ago
- Comments:10 (5 by maintainers)
Top Results From Across the Web
Day 6, Keynote Lecture 3. Uncertainty estimation in ... - YouTube
Slides: https://github.com/bayesgroup/deepbayes-2019/blob/master/lectures/day6/2.
Read more >A Review of Uncertainty Quantification in Deep Learning - arXiv
This study reviews recent advances in UQ methods used in deep learning, investigates the application of these methods in reinforcement learning, ...
Read more >Robust uncertainty estimates with out-of-distribution pseudo ...
This paper considers the idea of generating pseudo-inputs (i.e. extra augmented samples) to improve OOD uncertainty estimates by training with an ...
Read more >A review of uncertainty quantification in deep learning
Uncertainty estimation and quantification have been studied extensively in DL and ... Nan and Ji [655], 2020, BSDS500 and three other standard benchmark ......
Read more >Parameter estimation in uncertain differential equations based ...
The parameter estimation of uncertain differential equation is a significant subject in uncertainty theory. This paper introduces three ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@jchodera - see the discussion above - the example was using U_1 and U_2, not U_1-U2 and U_2-U1, so the results were nonsensical because the data was not physical. Good question about what should be returned in those cases, though, and if they should be trapped (if possible? Might be hard to distinguish in many cases from numerically bad data).
I’ve just suggested we also add a method to compute the number of effective samples: #427
I’m guessing these cases are seeing a collapse in the number of effective samples, meaning the estimates will be unreliable.
We should add some protection to make sure we don’t return
nan
s, thought. And we may want to warn when there is a sample size collapse.