Permission on local cached files should be configurable
See original GitHub issueIs your feature request related to a problem? Please describe.
If multiple users attempt to share the same models cache (e.g., on a company internal shared file system), anytime model files are downloaded, their permissions are 0600 (on linux) regardless of the umask. This is because file_download is using tempfile.NamedTemporaryFile
(see https://github.com/huggingface/huggingface_hub/blob/60077db55b81f2add3fd3bf774d7dc0f7acc4857/src/huggingface_hub/file_download.py#L727 and https://stackoverflow.com/questions/10541760/can-i-set-the-umask-for-tempfile-namedtemporaryfile-in-python), so only the user who downloaded them can read those files. As a result, users can’t share a local cache, which is a waste of resources and time for users.
Describe the solution you’d like
huggingface_hub should chmod
the downloaded files (for example, using the current os.umask
, or using a new configuration option) after the file is downloaded and before it is renamed so it can be shared across users.
Describe alternatives you’ve considered Separate caches per user is a workaround but is a massive waste of disk space and time. Each user would have to wait for new downloads even if the files have been downloaded by another user already.
Additional context Trying to use the HuggingFace diffusers/transformers library in an enterprise context.
Issue Analytics
- State:
- Created a year ago
- Comments:5 (3 by maintainers)
Top GitHub Comments
Hi, just confirming that we’ve switched to v0.11.1 and this works just as expected and is fixing a lot of headaches for me, thanks a lot!
That sounds like a sensible solution. I’d love to contribute but I have quite a few hurdles in my way due to corporate security&legal limitations. I already have another change in the wings related to proxy/certificate settings that I’m trying get over the fence to you, I’m not sure when I’ll get that done. I’ll add this one on my plate, but if anyone else has some time to address it, feel free to go ahead.