False positive: Typosquatting

Hi there, first off all: awesome tools you guys made! 🎉 Second, I encountered the following output when scanning a requirements.txt file:

Found 2 potentially malicious indicators in ruamel-yaml version 0.17.21

typosquatting: This package closely ressembles the following package names, and might be a typosquatting attempt: ruamel-yaml, ruamel-yaml

code-execution: found 1 source code matches
  * setup.py file executing code at ruamel.yaml-0.17.21/setup.py:955
        subprocess.check_output(cmd)

I do get why the second indicator is found, but the first one confuses me:

The package name (also installed on my machine) is ruamel.yaml. There is no package named ruamel-yaml in either my requirements nor on PyPi. Did something went from with the dots in the package name? Or is it because this package is listed in your typosquatting list as ruamel-yaml?

Thanks!

[QuinceyJames]

Hi! I don’t think that should be an issue since PEP 503 states:

This PEP references the concept of a “normalized” project name. As per PEP 426 the only valid characters in a name are the ASCII alphabet, ASCII numbers, ., -, and _. The name should be lowercased with all runs of the characters ., -, or _ replaced with a single - character.

Additionally:

Repositories MAY redirect unnormalized URLs to the canonical normalized URL (e.g. /Foobar/ may redirect to /foobar/), however clients MUST NOT rely on this redirection and MUST request the normalized URL.

You’ll notice PyPi does redirect all of these to the same site:

• https://pypi.org/project/pdfminer.six/
• https://pypi.org/project/pdfminer-six/
• https://pypi.org/project/pDfMiNeR.sIx/
• https://pypi.org/project/pdfminer.-_-.six/

But more importantly, PIP does not rely on this redirection and gets the normalized URL thanks to pip._internal.utils.packaging. These utilities can be reused by importing packaging and normalizing the name before running the typosquat algorithm.

[HugooB]

Great job @QuinceyJames , thanks for the detailed information!