Bandit broken via `stevedore` dependency with `importlib-metadata>=5`.

Describe the bug

Bandit depends on the stevedore package for extension management. That package, in turn, depends on importlib-metadata for the entrypoints mechanism that it uses. importlib-metadata 5.0.0 was recently released and apparently removed compatibility with the mechanism used by stevedore.

The following crash now happens, if importlib-metadata 5.0.0 is installed:

$ bandit --version
Traceback (most recent call last):
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/_cache.py", line 159, in _get_data_for_path
    with open(filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/me/.cache/python-entrypoints/19c8187c80a898f0d56eed9b81a44d91389b2a2953186081f4546ac88dcb6e00'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/me/src/python-lockup/stevedore-bug/bin/bandit", line 5, in <module>
    from bandit.cli.main import main
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/__init__.py", line 7, in <module>
    from bandit.core import config  # noqa
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/core/__init__.py", line 5, in <module>
    from bandit.core import config  # noqa
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/core/config.py", line 15, in <module>
    from bandit.core import extension_loader
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/core/extension_loader.py", line 109, in <module>
    MANAGER = Manager()
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/core/extension_loader.py", line 21, in __init__
    self.load_formatters(formatters_namespace)
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/bandit/core/extension_loader.py", line 29, in load_formatters
    verify_requirements=False,
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/extension.py", line 136, in __init__
    verify_requirements)
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/extension.py", line 218, in _load_plugins
    for ep in self.list_entry_points():
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/extension.py", line 207, in list_entry_points
    eps = list(_cache.get_group_all(self.namespace))
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/_cache.py", line 179, in get_group_all
    data = self._get_data_for_path(path)
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/_cache.py", line 162, in _get_data_for_path
    data = _build_cacheable_data(path)
  File "/home/me/src/python-lockup/stevedore-bug/lib/python3.7/site-packages/stevedore/_cache.py", line 110, in _build_cacheable_data
    for name, group_data in real_groups.items():
AttributeError: 'EntryPoints' object has no attribute 'items'

Reproduction steps

For crash:

1. ASDF_PYTHON_VERSION=3.7.14 python3 -m venv stevedore-bug # or your favorite way of creating a Python virtual environment
2. . stevedore-bug/bin/activate # Assuming a Bourne shell-like sourcing mechanism for your shell.
3. pip install bandit importlib-metadata
4. bandit --version # crashes

To prove that newest importlib-metadata is the culprit by constraining it:

1. ASDF_PYTHON_VERSION=3.7.14 python3 -m venv stevedore-workaround
2. . stevedore-workaround/bin/activate
3. pip install bandit 'importlib-metadata<5'
4. bandit --version # works

Expected behavior

Display of Bandit version:

bandit 1.7.4
  python version = 3.7.14 (default, Sep 12 2022, 16:42:27) [GCC 9.4.0]

Bandit version

1.7.4 (Default)

Python version

3.7

Additional context

Have not reported upstream to stevedore project yet since their issue tracker is not on Github. May do so later, depending on level of effort/hassle. Probably best workaround for now is to explicitly clamp the importlib-metadata version.

[mportesdev]

@kszmigiel Until a fix is released, you can pip install 'importlib-metadata<5;python_version<"3.8"' manually in the environment where bandit is installed.

[kszmigiel]

@emcd thank you for creating this issue, I was tearing my hair out since yesterday as I couldn’t find the reason behind this. Do you think updating the whole app to python 3.8 will help? Unfortunately I can’t wait for official fix, and can’t skip bandit check either.