pytest leaks memory and crashes jenkins slaves when this error occurs

Hi team, unfortunately I don’t have exact steps to recreate this, but I will include as much information as I have at this point and build upon it.

pytest version: 5.2.1

[gw11] linux Python 3.7.5 cwd: /home/jenkins/workspace/branch/rootdir
18:16:25  /home/jenkins/workspace/branch/rootdir/venv/lib/python3.7/site-packages/_pytest/pathlib.py:47: PytestWarning: (rm_rf) error removing /home/jenkins/workspace/branch/rootdir/.pytest_cache/v: [Errno 2] No such file or directory: 'v'
18:16:25    PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue))
18:16:25  /home/jenkins/workspace/branch/rootdir/venv/lib/python3.7/site-packages/_pytest/pathlib.py:47: PytestWarning: (rm_rf) error removing /home/jenkins/workspace/branch/rootdir/.pytest_cache/.gitignore: [Errno 2] No such file or directory: '.gitignore'
18:16:25    PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue))
18:16:34  collected 1947 items / 987 deselected / 960 selected

Not sure if this is caused by xdist? not sure if its also one of our plugins (internally loaded) but it looks like core pytest code here. Weird that it is looking for .gitignore etc to try and rm?

I will update this ticket as more information comes to light, sorry for lack of information, it happens perhaps 1 in 30 runs on jenkins.

When the above occurs, memory on the jenkins slave is consumed and consumed eventually reaching 100% and destroying the node, leaving the build hanging indefinitely

[blueyed]

@symonk It was addressed / improved in https://github.com/pytest-dev/pytest/commit/172b82875a2c3c1052938a300bc97d8cfb9688ab, but looks more like a red herring, i.e. it might indicate that something wrong is going on, but do not expect this to cause a hang - but maybe xdist gets confused through it somehow…?

As for clearing the cache you could also just try not using the option, but rm -rf … manually, and the see if it helps / works around this.

[symonk]

thanks, i dont think its too crazy - it is primarily integration and full end2end system testing that can take into the minutes for 1