Cannot audit requirements.txt containing URL dependencies

Bug description

Cannot produce an audit for a file containing URL dependencies.

Reproduction steps

requirements-lock.txt

de-core-news-sm @ https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.2.0/en_core_web_sm-3.2.0.tar.gz
httplib2==0.9.1

$ pip-audit -r requirements-lock.txt --no-deps --skip-editable
WARNING:pip_audit._cli:--no-deps is supported, but users are encouraged to fully hash their pinned dependencies
WARNING:pip_audit._cli:Consider using a tool like `pip-compile`: https://pip-tools.readthedocs.io/en/latest/#using-hashes
WARNING:pip_audit._cache:Warning: pip 20.0.2 doesn't support the `cache dir` subcommand, using /home/rooter/.pip-audit-cache instead
Traceback (most recent call last):
  File "/projects/dummy/.venv/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_cli.py", line 428, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_service/interface.py", line 148, in query_all
    for spec in specs:
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 120, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 310, in _collect_cached_deps
    for req, dep in self._collect_preresolved_deps(
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 260, in _collect_preresolved_deps
    raise RequirementSourceError(f"requirement {req.name} is not pinned: {str(req)}")
pip_audit._dependency_source.requirement.RequirementSourceError: requirement de-core-news-sm is not pinned: de-core-news-sm@ https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz from https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz (from RequirementLine(line_number=149, line='de-core-news-sm @ https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz', filename=PosixPath('requirements-lock.txt')))

Expected behavior

I would like it either be automatically skipped (maybe with an error code thrown), and/or have a CLI flag that allows ignoring them completely (without error code thrown at the end).

Screenshots and logs

pip-audit -r requirements.txt --no-deps --skip-editable -v
DEBUG:pip_audit._cli:parsed arguments: Namespace(cache_dir=None, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, dry_run=False, extra_index_urls=[], fix=False, format=<OutputFormatChoice.Columns: 'columns'>, ignore_vulns=[], index_url='https://pypi.org/simple', local=False, no_deps=True, output=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, paths=[], progress_spinner=<ProgressSpinnerChoice.On: 'on'>, project_path=None, require_hashes=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], skip_editable=True, strict=False, timeout=15, verbose=True, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>)
WARNING:pip_audit._cli:--no-deps is supported, but users are encouraged to fully hash their pinned dependencies
WARNING:pip_audit._cli:Consider using a tool like `pip-compile`: https://pip-tools.readthedocs.io/en/latest/#using-hashes
WARNING:pip_audit._cache:Warning: pip 20.0.2 doesn't support the `cache dir` subcommand, using /home/rooter/.pip-audit-cache instead
Traceback (most recent call last):
  File "/projects/dummy/.venv/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_cli.py", line 428, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_service/interface.py", line 148, in query_all
    for spec in specs:
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 120, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 310, in _collect_cached_deps
    for req, dep in self._collect_preresolved_deps(
  File "/projects/dummy/.venv/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 260, in _collect_preresolved_deps
    raise RequirementSourceError(f"requirement {req.name} is not pinned: {str(req)}")
pip_audit._dependency_source.requirement.RequirementSourceError: requirement de-core-news-sm is not pinned: de-core-news-sm@ https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz from https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz (from RequirementLine(line_number=1, line='de-core-news-sm @ https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.2.0/de_core_news_sm-3.2.0.tar.gz', filename=PosixPath('requirements.txt')))

Platform information

• OS name and version: Ubuntu 20.04.4 LTS
• pip-audit version (pip-audit -V): pip-audit 2.4.3
• Python version (python -V or python3 -V): Python 3.8.10
• pip version (pip -V or pip3 -V): pip 20.0.2

Additional context

Add any other context about the problem here.

[tetsuo-cpp]

(There’s a possible failure mode there where a user specifies package foo via a URL, but foo on PyPI is a completely different package that just happens to have the same name. If that happens, we may end up reporting incorrect vulnerabilities. But maybe that’s an acceptable tradeoff?)

I think that’s fine. We’ve already accepted that trade-off in other places.

For example, for VCS requirements or those on the actual filesystem, we just get the package name and version and check it against PyPI.

pip itself also has the same flavour of problem where packages with the same name on different indexes are assumed to be identical.

[tetsuo-cpp]

Thanks for reporting this @rooterkyberian. I’ll take a look at this shortly.