question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Permission on local cached files should be configurable

See original GitHub issue

Is your feature request related to a problem? Please describe. If multiple users attempt to share the same models cache (e.g., on a company internal shared file system), anytime model files are downloaded, their permissions are 0600 (on linux) regardless of the umask. This is because file_download is using tempfile.NamedTemporaryFile (see https://github.com/huggingface/huggingface_hub/blob/60077db55b81f2add3fd3bf774d7dc0f7acc4857/src/huggingface_hub/file_download.py#L727 and https://stackoverflow.com/questions/10541760/can-i-set-the-umask-for-tempfile-namedtemporaryfile-in-python), so only the user who downloaded them can read those files. As a result, users can’t share a local cache, which is a waste of resources and time for users.

Describe the solution you’d like huggingface_hub should chmod the downloaded files (for example, using the current os.umask, or using a new configuration option) after the file is downloaded and before it is renamed so it can be shared across users.

Describe alternatives you’ve considered Separate caches per user is a workaround but is a massive waste of disk space and time. Each user would have to wait for new downloads even if the files have been downloaded by another user already.

Additional context Trying to use the HuggingFace diffusers/transformers library in an enterprise context.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
antochecommented, Dec 14, 2022

Hi, just confirming that we’ve switched to v0.11.1 and this works just as expected and is fixing a lot of headaches for me, thanks a lot!

1reaction
antochecommented, Oct 31, 2022

That sounds like a sensible solution. I’d love to contribute but I have quite a few hurdles in my way due to corporate security&legal limitations. I already have another change in the wings related to proxy/certificate settings that I’m trying get over the fence to you, I’m not sure when I’ll get that done. I’ll add this one on my plate, but if anyone else has some time to address it, feel free to go ahead.

Read more comments on GitHub >

github_iconTop Results From Across the Web

You don't have write access to . Please pick a different local ...
Please pick a different local cached files directory." I have this infuriating issue where Google Drive For desktop states that it does not...
Read more >
Cached Permissions - TIBCO Product Documentation
In order to speed the authorization process, the EMS server caches responses received from the Permissions Module in two pools, the allow cache...
Read more >
Plan and configure Cached Exchange Mode in Outlook 2016 ...
To configure Cached Exchange Mode settings by using the OCT​​ Click More settings. Click the Cached Mode tab. Click Configure Cached Exchange ...
Read more >
Advanced content caching settings on Mac - Apple Support
Key Description Default value AllowImports Allows import (upload) requests. Yes MetricsInterval 60 (seconds) Parents none
Read more >
How to Share a Cache Among Projects - DVC
Make sure that the directory has proper permissions, so that all your colleagues can write to it, and can read cached files written...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found