incorrect case in test exclusion via marks causes collection crash

Im going to preface this by saying I stumbled across this and it can be arguably caused by user error. That said, I am reporting this due to the fashion in which pytest fails.

Issue

When deselecting tests by test mark, if NOT is used rather than not, pytest crashes with a lengthy stacktrace.

Action root@5de4510ee14c:/potato# pytest -m "NOT database" expected behavior In this case, i expected pytest to handle as if i used not, deselecting my database tests and running the rest of the suite

• I would also accept an argument error of some form, a graceful error would suffice.

actual behavior

 pytest -m "NOT (database)"
================================================= test session starts ==================================================
platform linux -- Python 3.6.6, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: /potato, inifile: pytest.ini
collecting ... INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 185, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 224, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 234, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 450, in perform_collect
INTERNALERROR>     session=self, config=self.config, items=items
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/mark/__init__.py", line 153, in pytest_collection_modifyitems
INTERNALERROR>     deselect_by_mark(items, config)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/mark/__init__.py", line 141, in deselect_by_mark
INTERNALERROR>     if matchmark(item, matchexpr):
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/mark/legacy.py", line 71, in matchmark
INTERNALERROR>     return eval(markexpr, {}, MarkMapping.from_item(colitem))
INTERNALERROR>   File "<string>", line 1, in <module>
INTERNALERROR> TypeError: 'bool' object is not callable

============================================= no tests ran in 0.02 seconds =============================================

• Include a detailed description of the bug or suggestion

pip list
Package        Version
-------------- -------
atomicwrites   1.2.1
attrs          18.2.0
more-itertools 5.0.0
pip            18.0
pluggy         0.8.0
py             1.7.0
pytest         4.0.2
setuptools     40.0.0
six            1.12.0
wheel          0.31.1

• pip list of the virtual environment you are using
• pytest and operating system versions pytest version 4.0.2, OS: Windows 10 x64, Linux (Docker python:3.6 container) Python versions: 3.7.1 x64, 3.6.6
• Minimal example if possible Below is a minimal setup that results in this crash

Example pytest.ini

[pytest]
markers =
    database: marks a test as a database test

example test file test_potato.py

from pytest import mark


@mark.database
def test_foo():
        assert 2+2 == 4

def test_bar():
        assert 4*4 == 16

example falsifying pytest invocation:

pytest -m "NOT (database)"

– its worth noting that pytest -m "not (database)" works as expected, and does not result in the above crash

[symonk]

@theunkn0wn1 yep completely; lets atleast make it have something user friendly for now, thanks for the report, i’ll have a look

[symonk]

@bluetech apologies i forgot a fixed # commit message; ok to close this? (or do we have a ‘pending release’ type label or something?)