Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`test_persistentdataset_dist` error on windows

See original GitHub issue

Describe the bug the error occurs randomly:

test_mp_dataset (tests.test_persistentdataset_dist.TestDistDataset) ... Process SpawnProcess-51:
persistent 1
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py", line 791, in move
    os.rename(src, real_dst)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp2ndb3qxg\\54c0443faf33e022375c4a3adc6afd11.pt' -> 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpovt2z4gf\\test\\54c0443faf33e022375c4a3adc6afd11.pt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "D:\a\MONAI\MONAI\tests\utils.py", line 396, in run_process
    raise e
  File "D:\a\MONAI\MONAI\tests\utils.py", line 387, in run_process
    func(*args, **kwargs)
  File "D:\a\MONAI\MONAI\tests\utils.py", line 557, in _call_original_func
    return f(*args, **kwargs)
  File "D:\a\MONAI\MONAI\tests\test_persistentdataset_dist.py", line 73, in test_mp_dataset
    self.assertEqual(list(ds1), list(ds))
  File "D:\a\MONAI\MONAI\monai\data\dataset.py", line 97, in __getitem__
    return self._transform(index)
  File "D:\a\MONAI\MONAI\monai\data\dataset.py", line 360, in _transform
    pre_random_item = self._cachecheck(self.data[index])
  File "D:\a\MONAI\MONAI\monai\data\dataset.py", line 354, in _cachecheck
    shutil.move(temp_hash_file, hashfile)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py", line 811, in move
    copy_function(src, real_dst)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py", line 435, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpovt2z4gf\\test\\54c0443faf33e022375c4a3adc6afd11.pt'
persistent 0
FAIL

— update — A similar issue was reported at bpo-29982 where the calling app tempfile.TemporaryDirectory to shutil should handle the exception. A python PR has been merged to address that issue https://github.com/python/cpython/pull/24793. Before that PR is officially released in python 3.10, we handle the exception with another try...except in https://github.com/Project-MONAI/MONAI/pull/3633.