Output appears in "Captured stderr call" but is unavailable in capsys or capfd

Environment

Pytest version: 5.2.1 Operating system: Kubuntu 18.04.3 LTS Pip list:

Package            Version
------------------ -------
atomicwrites       1.3.0
attrs              19.3.0
importlib-metadata 0.23
more-itertools     7.2.0
packaging          19.2
pip                19.3.1
pluggy             0.13.0
py                 1.8.0
pyparsing          2.4.2
pytest             5.2.1
setuptools         41.4.0
six                1.12.0
wcwidth            0.1.7
wheel              0.33.6
zipp               0.6.0

Description

It seems that output can appear in “Captured stderr call” but be unavailable to capsys or capfd, which can be a bug? It’s definitely weird.

I came accross this issue while trying to test my logging setup. This is a minimal example I came up with (available at https://github.com/butla/experiments/tree/master/pytest_capture_log_error):

# a_file.py
import logging

logging.basicConfig(level=logging.INFO)

LOG_MESSAGE = "the message we'll look for in the test"

_log = logging.getLogger(__name__)


def bla():
    _log.info(LOG_MESSAGE)
    return 5

# test_file.py
import a_file

def test_a(capsys):
    assert a_file.bla() == 5
    assert a_file.LOG_MESSAGE in capsys.readouterr().err

The test fails and it shows

------------------------------------------ Captured stderr call ------------------------------------------
INFO:a_file:the message we'll look for in the test

but both capsys and capfd show out and err to be empty.

I would use caplog, but the messages I get from it don’t conform to the log configuration that I have set up in my original app, so I decided to just use stderr.

[asottile]

@RonnyPfannschmidt my reproduction above does not use logging – this is definitely something capfd should be able to handle. pytest almost-mockingly displays “Captured stdout” “wat” when it fails so pytest definitely could capture it properly!

[Kache]

I think I ran into the same issue, my repro:

# foo.py
print("hello")

Surprisingly:

from importlib import import_module

def test_prints_hello(capfd):
    import_module('foo')
    out, err = capfd.readouterr()

    assert out == ""  # will pass
    # assert out == "hello\n"  # and so will this one (expected), but only if above line is commented out

edit Although this is still happening in the particular project I used, I’m not able to reproduce this in an fresh & independent venv. Very strange…