[4.2rc1] test_timedelta_datetime_roundtrip emits hypothesis.errors.Flaky

Description

platform win32 -- Python 3.8.5, pytest-6.1.2, py-1.9.0, pluggy-0.13.1

Running tests with Astropy version 4.2rc1.
Running tests in C:\...\astropy.

Date: 2020-11-11T16:27:19

Platform: Windows-10-10.0.19041-SP0

Executable: C:\...\Miniconda3\envs\py38\python.exe

Full Python Version:
3.8.5 | packaged by conda-forge | (default, Jul 31 2020, 01:53:45) [MSC v.1916 64 bit (AMD64)]

encodings: sys: utf-8, locale: cp1252, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions:
Numpy: 1.19.1
Scipy: 1.5.0
Matplotlib: 3.3.1
h5py: 2.10.0
Pandas: 1.1.2
PyERFA: 1.7.0
Cython: 0.29.21
Scikit-image: not available
asdf: 2.7.1

Using Astropy options: remote_data: any.

rootdir: D:\...
plugins: asdf-2.7.1, hypothesis-5.29.4, arraydiff-0.3, astropy-header-0.1.2, cov-2.10.1, doctestplus-0.8.0,
filter-subpackage-0.1.1, forked-1.3.0, openfiles-0.5.0, remotedata-0.3.2, xdist-2.1.0

...
_____ test_timedelta_datetime_roundtrip[ut1] _____
scale = 'ut1'

>   ???

C:\...\astropy\time\tests\test_precision.py:609:
_ _ _ _ _ _ _ _ _
c:\...\hypothesis\core.py:637: in execute_once
    self.__flaky(
_ _ _ _ _ _ _ _ _ 
self = <hypothesis.core.StateForActualGivenExecution object at 0x00000205FCBF2CD0>
message = "Hypothesis test_timedelta_datetime_roundtrip(scale='ut1', days=0, day_frac=0.4955591007075132)
produces unreliable results: Falsified on the first call but did not on a subsequent one"

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis
test_timedelta_datetime_roundtrip(scale='ut1', days=0, day_frac=0.4955591007075132)
produces unreliable results: Falsified on the first call but did not on a subsequent one

c:\...\hypothesis\core.py:847: Flaky
--------- Hypothesis ---------
Highest target scores:
     4.78827e-07  (label=' (a-b).to_value(u.s), from TimeDelta')
     4.99866e-07  (label=' (b-a).to_value(u.s), from TimeDelta')

Falsifying example: test_timedelta_datetime_roundtrip(
    scale='ut1', days=0, day_frac=0.4955591007075132,
)
Unreliable test timings! On an initial run, this test took 250.62ms, which exceeded the deadline of 200.00ms,
but on a subsequent run it took 0.90 ms, which did not. If you expect this sort of variability in your test timings,
consider turning deadlines off for this test by setting deadline=None.

Might be related to #10724

[mhvk]

The timing difference is very strange. Could it be that it is the first test for which the IERS tables are being loaded? (which takes time; would be first test involving UT1). If so, the way to avoid it would be to have something like Time('2020-01-01').ut1 be executed in setup (not sure how to ensure that happens…)

[pllim]

I don’t see it on Windows with #11450 . If I see it again, I will open new issue. Thanks, everyone!