Permissions error on /tmp/ir_dataset directory due to multiple users on the same server

Describe the bug When more than one users on the same server or device use the ir_datasets to fetch documents, then the permission denied error might be encountered if one of the users does not have write access to the already created directory

Affected dataset(s) This issue does not affect datasets

To Reproduce Steps to reproduce the behavior:

1. User A runs a script that imports some documents using ir_datasets
2. User B who is on the same system performs the same actions
3. User B is part of the Others group in the system and hence does not have write permissions to the already existing /tmp/ir_datasets directory
4. User B sees the following error: PermissionError: [Errno 13] Permission denied: '/tmp/ir_datasets/tmp3sn3tbic'

Expected behavior When multiple users are using the package on the same device, some additional checks would need to be in place to avoid permission errors. For example, the ir_directory directory that is created for tmp files could be prefaced by a username to avoid such conflicts.

Additional context This issue can be bypassed by utilizing the IR_DATASETS_TMP environment variable.

[seanmacavaney]

and when one hits an error, the process isn’t closed automatically

Gotcha – thanks! This is a bug, as it should close the file in this case so others can use it. I’ll look into fixing this.

Possibly due to incomplete download?

Yep, something went wrong with the download. It’s not safe to use this version because the contents could be different, or you may be missing some records.

[yuenherny]

Hi @seanmacavaney , thanks for the prompt response.

Nope, I guess the process is open because I tried to download multiple parts of the dataset - queries, scorreddocs, docs, qrels in sequence in my notebook - and when one hits an error, the process isn’t closed automatically.

Now that I managed to download (after restarting my laptop), I get another error:

[INFO] Please confirm you agree to the MSMARCO data usage agreement found at <http://www.msmarco.org/dataset.aspx>
[INFO] If you have a local copy of https://msmarco.blob.core.windows.net/msmarcoranking/collectionandqueries.tar.gz, you can symlink it here to avoid downloading it again: C:\Users\USER\.ir_datasets\downloads\31644046b18952c1386cd4564ba2ae69
[INFO] [starting] https://msmarco.blob.core.windows.net/msmarcoranking/collectionandqueries.tar.gz
[INFO] [finished] https://msmarco.blob.core.windows.net/msmarcoranking/collectionandqueries.tar.gz: [59:56] [954MB] [265kB/s]
[WARNING] Download failed: Expected md5 hash to be 31644046b18952c1386cd4564ba2ae69 but got 9a1336b80866927a64cd43a5d820f277

Possibly due to incomplete download?