sdist race condition and temp dir customization

Currently if two sdist commands are ran at the same time there is a race condition. One sdist command can be purging the temp files under {packagename}-{packageversion} in the setup.py directory while the other is zipping it, causing error messages like this

Creating tar archive
removing 'cosmo4d-0.0.0' (and everything under it)
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/setup.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_engine.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_pmesh.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests/test_map2d.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/tests'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/iotools.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/pmeshengine.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/__init__.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/version.py'
error removing cosmo4d-0.0.0: [Errno 2] No such file or directory: 'cosmo4d-0.0.0/cosmo4d/options.py'
Processing /tmp/VlnJRkP/cosmo4d-0.0.0.tar.gz
Skipping bdist_wheel for cosmo4d, due to binaries being disabled for it.

One possible solution to avoid the racing is to allow an argument --temproot to allow redirection of the temp directory to a different location. Then the caller to setup.py (or sdist itself) can use mktemp -d to avoid building sdist from the same temp dir; or at least to minimize the chances of racing.

[ghost]

Threads aren’t the only way to access and API concurrently (processes?). The proposed “global setuptools lock” would be file-based to prevent concurrent access by multiple processes as well. That said, this isn’t something that is going to even matter in setuptools because a frontend will be doing the locking.

The point is that running any distutils command concurrently for the same setup.py is not supported. In addition, there is no way that it could be supported because setuptools is simply wrapping other people’s scripts that aren’t thread-safe (which yes, includes “processes”).

[rainwoodman]

Thanks for the suggestions. I’ll do something along the lines. But this issue I’ll close as it is a wont fix.