Sampling error in tests

🐞 Problem

I have recently started to get a weird error after running pytest.

Traceback

============================================================================== FAILURES ==============================================================================
___________________________________________ test_sample_closed_starts_and_ends_at_min_anomaly_if_in_range_and_no_max_given ___________________________________________

>   ???

tests/tests_twobody/test_sampling.py:53: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/tests_twobody/test_sampling.py:68: in test_sample_closed_starts_and_ends_at_min_anomaly_if_in_range_and_no_max_given
    assert_quantity_allclose(result[-1], min_nu, atol=1e-14 * u.rad)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

actual = <Quantity -3.99680289e-15 rad>, desired = <Quantity 6.2781633e-15 rad>, rtol = 1e-07, atol = <Quantity 1.e-14 rad>, kwargs = {}
np = <module 'numpy' from '/home/yash/poli_astro/poli/lib/python3.8/site-packages/numpy/__init__.py'>
_unquantify_allclose_arguments = <function _unquantify_allclose_arguments at 0x7f9a4d2caaf0>

    def assert_quantity_allclose(actual, desired, rtol=1.e-7, atol=None,
                                 **kwargs):
        """
        Raise an assertion if two objects are not equal up to desired tolerance.
    
        This is a :class:`~astropy.units.Quantity`-aware version of
        :func:`numpy.testing.assert_allclose`.
        """
        import numpy as np
        from astropy.units.quantity import _unquantify_allclose_arguments
>       np.testing.assert_allclose(*_unquantify_allclose_arguments(
            actual, desired, rtol, atol), **kwargs)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=1e-14
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 1.02749662e-14
E       Max relative difference: 1.63661977
E        x: array(-3.996803e-15)
E        y: array(6.278163e-15)

../poli/lib/python3.8/site-packages/astropy/tests/helper.py:491: AssertionError
----------------------------------------------------------------------------- Hypothesis -----------------------------------------------------------------------------
Falsifying example: test_sample_closed_starts_and_ends_at_min_anomaly_if_in_range_and_no_max_given(
    min_nu=<Quantity 6.2781633e-15 rad>, ecc=<Quantity 0.99161489>,
)

Ouput of pip freeze:

alabaster==0.7.12
anyio==2.0.2
apipkg==1.5
appdirs==1.4.4
argon2-cffi==20.1.0
astroid==2.4.2
astropy==4.2
astropy-sphinx-theme==1.1
astroquery==0.4.1
async-generator==1.10
attrs==20.3.0
Babel==2.9.0
backcall==0.2.0
beautifulsoup4==4.9.3
black==20.8b1
bleach==3.2.1
certifi==2020.12.5
cffi==1.14.4
chardet==4.0.0
click==7.1.2
cognitive-complexity==1.2.0
colorama==0.4.4
confuse==1.4.0
coverage==5.3.1
cryptography==3.3.1
cycler==0.10.0
czml3==0.5.4
decorator==4.4.2
defusedxml==0.6.0
distlib==0.3.1
docutils==0.16
entrypoints==0.3
execnet==1.8.0
filelock==3.0.12
flake8==3.8.4
flake8-cognitive-complexity==0.1.0
flake8-polyfill==1.0.2
flit==3.0.0
flit-core==3.0.0
future==0.18.2
html5lib==1.1
htmlmin==0.1.12
hypothesis==6.0.1
idna==2.10
ImageHash==4.2.0
imagesize==1.2.0
iniconfig==1.1.1
ipykernel==5.4.3
ipython==7.19.0
ipython-genutils==0.2.0
ipywidgets==7.6.3
isort==5.7.0
jedi==0.18.0
jeepney==0.6.0
Jinja2==2.11.2
joblib==1.0.0
jplephem==2.15
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.11
jupyter-core==4.7.0
jupyter-server==1.2.1
jupyterlab==3.0.4
jupyterlab-pygments==0.1.2
jupyterlab-server==2.1.2
jupyterlab-widgets==1.0.0
jupytext==1.9.1
keyring==21.8.0
kiwisolver==1.3.1
lazy-object-proxy==1.4.3
llvmlite==0.35.0
mando==0.6.4
markdown-it-py==0.6.1
MarkupSafe==1.1.1
matplotlib==3.3.3
mccabe==0.6.1
mdit-py-plugins==0.2.4
missingno==0.4.2
mistune==0.8.4
mypy==0.790
mypy-extensions==0.4.3
myst-parser==0.13.3
nbclassic==0.2.6
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nbsphinx==0.8.0
nest-asyncio==1.4.3
networkx==2.5
nodejs==0.1.1
notebook==6.1.6
NotFound==1.0.2
npm==0.1.1
numba==0.52.0
numpy==1.20.0
numpydoc==1.1.0
optional-django==0.1.0
packaging==20.8
pandas==1.2.0
pandoc==1.0.2
pandocfilters==1.4.3
parso==0.8.1
pathspec==0.8.1
patsy==0.5.1
pexpect==4.8.0
phik==0.11.0
pickleshare==0.7.5
Pillow==8.1.0
plotly==4.14.3
plotly-express==0.4.1
pluggy==0.13.1
ply==3.11
poliastro==0.15.dev0
prometheus-client==0.9.0
prompt-toolkit==3.0.10
ptyprocess==0.7.0
py==1.10.0
pycodestyle==2.6.0
pycparser==2.20
pyerfa==1.7.1.1
pyflakes==2.2.0
Pygments==2.7.4
pyparsing==2.4.7
pyrsistent==0.17.3
pytest==6.2.2
pytest-cov==2.5.1
pytest-doctestplus==0.9.0
pytest-mpl==0.12
pytest-mypy==0.8.0
pytest-remotedata==0.3.2
python-dateutil==2.8.1
pytoml==0.1.21
pytz==2020.5
PyWavelets==1.1.1
PyYAML==5.3.1
pyzmq==20.0.0
radon==4.3.2
regex==2020.11.13
requests==2.25.1
retrying==1.3.3
scipy==1.6.0
seaborn==0.11.1
SecretStorage==3.3.0
Send2Trash==1.5.0
six==1.15.0
sniffio==1.2.0
snowballstemmer==2.0.0
sortedcontainers==2.3.0
soupsieve==2.1
Sphinx==3.4.3
sphinx-astropy==1.3
sphinx-autoapi==1.5.1
sphinx-automodapi==0.13
sphinx-gallery==0.8.2
sphinx-hoverxref==0.5b1
sphinx-notfound-page==0.6
sphinx-rtd-theme==0.5.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
statsmodels==0.12.1
tangled-up-in-unicode==0.0.6
Tcl==0.2
terminado==0.9.2
testpath==0.4.4
toml==0.10.2
tornado==6.1
tox==3.21.1
tqdm==4.56.0
traitlets==5.0.5
typed-ast==1.4.2
typing-extensions==3.7.4.3
Unidecode==1.1.2
urllib3==1.26.2
virtualenv==20.3.1
visions==0.6.0
w3lib==1.22.0
wcwidth==0.2.5
webencodings==0.5.1
widgetsnbextension==3.5.1
wrapt==1.12.1

📋 Steps to solve the problem Investigate tests, specifically focusing on what eccentricities are being used to test them.

Notes

• The error traceback clearly states that we use falsifying examples for testing. Specifically, the problem seems to be in the eccentricity used: 0.99161489 (giving -3.996803e-15 as the final true anomaly) where instead it must have used 0.88680956 which would have yielded a correct final true anomaly (6.278163e-15). See the test here
• This mostly seems to be a problem in the testing part of the code and not in the source code implementation.
• I am currently investigating the cause and am not sure if this is only me or if it is really a bug.

Please let me know if this is really a problem.

Thanks!!

[Yash-10]

Ok, I tried running the tests again, and now I do not get any errors!

numba -s output:

__Time Stamp__
Report started (local time)                   : 2022-06-09 00:09:42.840996
UTC start time                                : 2022-06-08 18:39:42.841006
Running time (s)                              : 1.198525

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : icelake-client
CPU Count                                     : 4
Number of accessible CPUs                     : 4
List of accessible CPUs cores                 : 0-3
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit adx aes avx avx2
                                                avx512bitalg avx512bw avx512cd
                                                avx512dq avx512f avx512ifma
                                                avx512vbmi avx512vbmi2 avx512vl
                                                avx512vnni avx512vpopcntdq bmi
                                                bmi2 clflushopt cmov cx16 cx8 f16c
                                                fma fsgsbase fxsr gfni invpcid
                                                lzcnt mmx movbe pclmul pku popcnt
                                                prfchw rdpid rdrnd rdseed sahf sgx
                                                sha sse sse2 sse3 sse4.1 sse4.2
                                                ssse3 vaes vpclmulqdq xsave xsavec
                                                xsaveopt xsaves

Memory Total (MB)                             : 7735
Memory Available (MB)                         : 3056

__OS Information__
Platform Name                                 : Linux-5.13.0-44-generic-x86_64-with-glibc2.29
Platform Release                              : 5.13.0-44-generic
OS Name                                       : Linux
OS Version                                    : #49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022
OS Specific Version                           : ?
Libc Version                                  : glibc 2.31

__Python Information__
Python Compiler                               : GCC 9.4.0
Python Implementation                         : CPython
Python Version                                : 3.8.10
Python Locale                                 : en_IN.ISO8859-1

__Numba Toolchain Versions__
Numba Version                                 : 0.55.1
llvmlite Version                              : 0.38.0

__LLVM Information__
LLVM Version                                  : 11.1.0

__CUDA Information__
CUDA Device Initialized                       : False
CUDA Driver Version                           : ?
CUDA Runtime Version                          : ?
CUDA NVIDIA Bindings Available                : ?
CUDA NVIDIA Bindings In Use                   : ?
CUDA Detect Output:
None
CUDA Libraries Test Output:
None

__SVML Information__
SVML State, config.USING_SVML                 : False
SVML Library Loaded                           : False
llvmlite Using SVML Patched LLVM              : True
SVML Operational                              : False

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+-->Vendor: GNU
Workqueue Threading Layer Available           : True
+-->Workqueue imported successfully.

__Numba Environment Variable Information__
None found.

__Conda Information__
Conda not available.

__Installed Packages__
Package                       Version     Editable project location
----------------------------- ----------- -------------------------
alabaster                     0.7.12
astroid                       2.11.5
astropy                       5.0.3
astroquery                    0.4.5
asttokens                     2.0.5
attrs                         21.4.0
autoapi                       2.0.1
Babel                         2.9.1
backcall                      0.2.0
beautifulsoup4                4.10.0
bleach                        5.0.0
certifi                       2021.10.8
cffi                          1.15.0
charset-normalizer            2.0.12
cryptography                  36.0.1
cycler                        0.11.0
decorator                     5.1.1
defusedxml                    0.7.1
distlib                       0.3.4
docutils                      0.17.1
entrypoints                   0.4
executing                     0.8.3
fastjsonschema                2.15.3
filelock                      3.6.0
flit                          3.6.0
flit_core                     3.6.0
fonttools                     4.29.1
html5lib                      1.1
hypothesis                    6.47.0
idna                          3.3
imagesize                     1.3.0
importlib-metadata            4.11.1
importlib-resources           5.7.1
iniconfig                     1.1.1
ipython                       8.4.0
jedi                          0.18.1
jeepney                       0.7.1
Jinja2                        3.0.3
jplephem                      2.17
jsonschema                    4.6.0
jupyter-client                7.3.3
jupyter-core                  4.10.0
jupyterlab-pygments           0.2.2
jupytext                      1.13.8
keyring                       23.5.0
kiwisolver                    1.3.2
lazy-object-proxy             1.7.1
llvmlite                      0.38.0
markdown-it-py                2.1.0
MarkupSafe                    2.1.0
matplotlib                    3.5.1
matplotlib-inline             0.1.3
mdit-py-plugins               0.3.0
mdurl                         0.1.1
mistune                       0.8.4
myst-parser                   0.18.0
nbclient                      0.6.4
nbconvert                     6.5.0
nbformat                      5.4.0
nbsphinx                      0.8.9
nest-asyncio                  1.5.5
NotFound                      1.0.2
numba                         0.55.1
numpy                         1.21.5
packaging                     21.3
pandas                        1.4.1
pandocfilters                 1.5.0
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.0.1
pip                           22.0.3
pkg_resources                 0.0.0
platformdirs                  2.5.1
plotly                        5.6.0
pluggy                        1.0.0
poliastro                     0.17.dev0   /home/yash/poliastro
prompt-toolkit                3.0.29
ptyprocess                    0.7.0
pure-eval                     0.2.2
py                            1.11.0
pycparser                     2.21
pyerfa                        2.0.0.1
Pygments                      2.11.2
pyparsing                     3.0.7
pyrsistent                    0.18.1
pytest                        7.1.2
python-dateutil               2.8.2
pytz                          2021.3
pyvo                          1.3
PyYAML                        6.0
pyzmq                         23.1.0
requests                      2.27.1
scipy                         1.8.0
SecretStorage                 3.3.1
setuptools                    60.9.3
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.3.1
Sphinx                        4.4.0
sphinx-autoapi                1.8.4
sphinx-copybutton             0.5.0
sphinx-gallery                0.10.1
sphinx-github-role            0.1.0
sphinx-hoverxref              1.1.1
sphinx-notfound-page          0.8
sphinx-rtd-theme              1.0.0
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
stack-data                    0.2.0
tenacity                      8.0.1
tinycss2                      1.1.1
toml                          0.10.2
tomli                         2.0.1
tomli_w                       1.0.0
tornado                       6.1
tox                           3.24.5
traitlets                     5.2.2.post1
typing_extensions             4.2.0
Unidecode                     1.3.4
urllib3                       1.26.8
virtualenv                    20.13.1
wcwidth                       0.2.5
webencodings                  0.5.1
wheel                         0.37.1
wrapt                         1.14.1
zipp                          3.7.0

No errors reported.


__Warning log__
Warning (cuda): CUDA device intialisation problem. Message:Error at driver init: Call to cuInit results in CUDA_ERROR_NO_DEVICE (100)
Exception class: <class 'numba.cuda.cudadrv.error.CudaSupportError'>
Warning: Conda not available.
 Error was [Errno 2] No such file or directory: 'conda'

Warning (psutil): psutil cannot be imported. For more accuracy, consider installing it.

[astrojuanlu]

Oh wow, near-parabolic orbits there we go again… 😱

This is an excellent investigation, thanks for taking the time to dig into this. I’m so happy that we are using hypothesis to surface these sort of problems. Will give this some thought over the weekend, if anyone has more ideas feel free to share here.