wandb in pytest causes OSError: [Errno 29] Illegal seek

• Weights and Biases version: 0.9.2
• Python version: 3.7.6
• Operating System: Linux

Description

Tried to use wanb in pytest here in this PR https://github.com/PyTorchLightning/pytorch-lightning/pull/2502 The error occurs locally and in the CI on Linux. On Mac, the CI is simply timing out.

What I Did

py.test -v pytorch_lightning/tests/loggers/test_wandb.py


_______________ ERROR at setup of test_auroc[pred1-target1-0.0] ________________

self = <contextlib._GeneratorContextManager object at 0x7fe46c19d190>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               OSError: [Errno 29] Illegal seek

/opt/hostedtoolcache/Python/3.7.7/x64/lib/python3.7/contextlib.py:119: OSError

[vanpelt]

It’s so odd this isn’t working. One thing to try would be to set wandb_logger._experiment = MagicMock() in your test_minimal. The other option is to install our next generation client with pip install wandb -U --pre and set os.environ["WANDB_MODE"] = "noop" in your test.

[avi-jit]

@avi-jit what error are you seeing when you attempt to use MagicMock?

The same: [Errno 29] Illegal seek

Here’s the traceback:

======================================== test session starts ========================================
platform linux -- Python 3.8.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /nas/home/thawani/MCS/num2vec/pycharm, inifile: pytest.ini
collected 1 item                                                                                    

test_.py FE                                                                                   [100%]

============================================== ERRORS ===============================================
_______________________________ ERROR at teardown of test_proto_read ________________________________

self = <contextlib._GeneratorContextManager object at 0x7f01dfe885e0>, type = None, value = None
traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               OSError: [Errno 29] Illegal seek

../../../anaconda3/envs/num_hf3/lib/python3.8/contextlib.py:120: OSError
============================================= FAILURES ==============================================
__________________________________________ test_proto_read __________________________________________

self = <contextlib._GeneratorContextManager object at 0x7f0254cd2ca0>, type = None, value = None
traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               OSError: [Errno 29] Illegal seek

../../../anaconda3/envs/num_hf3/lib/python3.8/contextlib.py:120: OSError
====================================== short test summary info ======================================
FAILED test_.py::test_proto_read - OSError: [Errno 29] Illegal seek
ERROR test_.py::test_proto_read - OSError: [Errno 29] Illegal seek
==================================== 1 failed, 1 error in 58.94s ====================================

Traceback (most recent call last):
  File "/nas/home/thawani/anaconda3/envs/num_hf3/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/config/__init__.py", line 124, in main
    ret = config.hook.pytest_cmdline_main(
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/main.py", line 240, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/main.py", line 235, in wrap_session
    config._ensure_unconfigure()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/config/__init__.py", line 829, in _ensure_unconfigure
    fin()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/capture.py", line 129, in stop_global_capturing
    self._global_capturing.pop_outerr_to_orig()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/capture.py", line 474, in pop_outerr_to_orig
    out, err = self.readouterr()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/capture.py", line 515, in readouterr
    out = self.out.snap()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/capture.py", line 626, in snap
    res = super().snap()
  File "/nas/home/thawani/anaconda3/envs/num_hf3/lib/python3.8/site-packages/_pytest/capture.py", line 585, in snap
    self.tmpfile.seek(0)
OSError: [Errno 29] Illegal seek

wandb: Waiting for W&B process to finish, PID 140579

Note that this fails when I call trainer.fit() but doesn’t fail if I merely instantiate a wandb_logger, or even a Trainer with the wandb_logger.