pytest produces spurious error output in non-git directories/repos with git 2.14.2 when determining project name

Environment

OS: Debian 9.3 Stretch (amd64), with backports repository enabled Python version: 2.7.13 pytest-benchmark version: 3.1.1 git version: 2.14.2

Requirements file (pip freeze):

attrs==17.4.0
docutils==0.14
funcsigs==1.0.2
pathlib==1.0.1
pkg-resources==0.0.0
pluggy==0.6.0
py==1.5.2
py-cpuinfo==3.3.0
pytest==3.3.2
pytest-benchmark==3.1.1
six==1.11.0
statistics==1.0.3.5

Reproduction steps

1. Bring-up a VM, and install the following prerequisites on it (subsequent steps should be performed on the VM itself):

   • virutalenvwrapper
   • git, version 2.14.2 (2.13.x and upwards may work as well, but I have not tested this)

   If using Vagrant, this should be as simple as:

   mkdir -p ~/tmp/pytest-benchmark-error-output
   cd ~/tmp/pytest-benchmark-error-output
   vagrant init debian/stretch64
   vagrant up
   vagrant ssh -c 'echo "deb http://ftp.debian.org/debian stretch-backports main" | sudo tee /etc/apt/sources.list.d/backports.list'
   vagrant ssh -c 'sudo apt-get update && sudo apt-get -t stretch-backports -y install virtualenvwrapper git'
   

2. Create an empty directory where pytest will be run, and switch to it:

   mkdir /tmp/pytest-benchmark-error-output
   cd /tmp/pytest-benchmark-error-output
   

3. Set-up Python virtual environment and activate it:

   mkvirtualenv pytest-benchmark-error-output && pip install pytest-benchmark==3.1.1
   

4. Run pytest:

   pytest
   

Expected results

While running command from (4), output is as follows:

======== test session starts ========
platform linux2 -- Python 2.7.13, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
benchmark: 3.1.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /tmp/pytest-benchmark-error-output, inifile:
plugins: benchmark-3.1.1
collected 0 items                                                                                                                                                                                                 

======== no tests ran in 0.00 seconds ========

Actual results:

While running command from (4), output is as follows (note the fatal error at top):

fatal: --local can only be used inside a git repository
======== test session starts ========
platform linux2 -- Python 2.7.13, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
benchmark: 3.1.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /tmp/pytest-benchmark-error-output, inifile:
plugins: benchmark-3.1.1
collected 0 items                                                                                                                                                                                                 

======== no tests ran in 0.00 seconds ========

Additional notes

The error comes from the file src/pytest_benchmark/utils.py, function get_project_name_git. Technically speaking, the error can safely be ignored - it only seems that more recent git versions will be a bit more “whiny” about running the git config --local remote.origin.url command outside of a git repository.

I can see three possible solutions, and could provide a PR depending on what is more desirable:

1. Check if the directory we are in is a git repository or not. This is probably the safest thing to do long-term. Stack Overflow has a decent solution listed.

2. Mask the specific error output. This would have tendency to break in future.

3. Mask any error output. This would have tendency to hide more serious errors (like missing git binary or something similar).

[ionelmc]

Ok, we could have an config option for this.

[ml31415]

I don’t see a good reason, why pytest-benchmark should interact with any versioning system at all, if I don’t explicitly tell it to. Just run into this when I noticed that pytest-benchmark is calling for hg and producing error output, if I’m in a completely unversioned folder. I totally agree with @kiilerix that this is quite a violation from what I expect this tool to do. And a quite error prone and time waster when it’s doing such stuff, whenever a test is run.