find_on_path distribution finder can cause "maximum recursion depth exceeded in comparison"

A few days ago someone filed an issue on pip’s tracker that looks like a setuptools issue: https://github.com/pypa/pip/issues/6288

Basically, it looks like the issue is that find_on_path() (the “distribution finder” corresponding to pkgutil.ImpImporter and FileFinder) can get into a situation where it’s calling itself in an (infinite) recursive loop. Also, I believe this is happening in the find_on_path(only=False) case, rather than only=True.

[JVillella]

I ran into this problem today and found a solution for my case. My setup is probably different than the OP, but the errors appear to be the same.

I had a docker container w/ the following,

RUN cd /usr/local/lib/python3.7/dist-packages && python /src/setup.py develop

Purpose was so that the .egg-link doesn’t live under /src and I can mount a src volume. Not doing this would overwrite the .egg-link when mounting.

Then, once in the container, trying to run any pip install ... returns the find_path() recursion failure. The problem was that I was using dist-packages, should have been site-packages (the latter of which didn’t initially exist in the container). Here’s what fixed the problem,

ENV PYTHONPATH=/src
COPY . /src
RUN cd /usr/local/lib/python3.7 && \
    mkdir site-packages && \
    cd site-packages && \
    python /src/setup.py develop
WORKDIR /src

[jaraco]

I am not sure what a “reproducing this problem” actually means. I run into this error when doing 'pip install ’ with any package.

The problem is, when the maintainers here run pip install, we don’t encounter the issue… so we’re kinda stuck investigating. I’ve tried running pip install service-identity on my Windows machine and it completes just fine. This all suggests to me there’s another issue at play here. We need to figure out what that is. Maybe you have symlinks installed. Maybe you’re in a directory that has settings configured. Maybe you have environment variables set. Maybe you’re behind a weird firewall. Maybe you’re running on an unprivileged account. We need to eliminate all these variables and hone in on what are the factors that lead to the failure you’ve encountered. Now that we have two people encountering the issue, perhaps we could compare environments to ascertain what things might be in common.

cjerdonek has done some investigation trying to imagine what might be going wrong, but we don’t have a good way to confirm (or to ensure this doesn’t happen to someone else).

For next steps, I recommend trying another machine. If you’re on a work machine, find a personal machine (or vice versa) to try the same operation. Or try installing a clean version of Python (maybe try 3.7 instead of 3.6). Or find a friend and see if they encounter the same problem. Alternately, install a clean version of Windows into a VM and try that. Or even better, if you can reproduce the issue outside of Windows, that will vastly expand the audience of maintainers that will be able to help track down the issue.

Failing all of that, I recommend digging into the code when the failure occurs (add print statements or use pdb to break into the code and inspect the state. You’ll need to study the code enough to understand what it’s trying to do to know when the behavior diverges from that expectation.

Best of luck.