pip install fails for demjson3 on Windows

I experience sporadic failures during pip installs of offline packages on Windows only . It is really hard to reproduce and only occurs on some Windows machines, Linux not being affected at all:

Processing c:\users\user\desktop\proj\pypi\packages\dev\demjson3-3.0.5.tar.gz
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
         or: setup.py --help [cmd1 cmd2 ...]
         or: setup.py --help-commands
         or: setup.py cmd --help

      error: invalid command 'egg_info'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

pip and setuptools are up to date.

For some reason demjson3 does not behave well.

Looking at the code and #408 the library is only used for parsing JavaScript (not JSON) data structures. It tried replacing demson3 with json from the standard library or other librariers such as orjson or ujson, but due to not being valid JSON this fails.

I have come up with a solution based on the PyYAML package. It is just a direct replacement that retains decoding capability (e.g. unquoted properties) and also speeds up decoding: from 1.91s to 0.13s for 1000 runs → >= 10x improvement.

I could provide a PR including adapted json test.

What are your thoughts on this?

[WolfgangFahl]

@hroemer - thanks for clarification - yes please open up a different issue and pull request. Event if it’s only for adding you to the collaborators list. In this phase i think it’s very important to have lively discussion and stepwise progress.

[hroemer]

@WolfgangFahl I finally dug a little deeper in to the problem, I can now clearly reproduce it.

It is not a problem with demjson3 at all. My apologies for the fuss and false assumptions here.

The above problem is actually related to pip/setuptools in combination with a “missing” setuptools-aa.b.c.dist-info/ folder and sdist packages (where demjson3 just happens to be the first in alphabetical order on mind end) that causes the mentioned pip failure!?

So a potential exchange of one library with another is on a different plate and should be considered with more care. If you are still interested in the small changes (just imports, dependencies and function call) that would be required and on how PyYAML actually performs decoding (simple timeit revealed a 10x boost), I could provide a fork/PR.

Thanks!