question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Checkpoint handler save files with wrong permissions

See original GitHub issue

🐛 Bug description

Hi @vfdev-5 ,

We always run programs in dockers, usually it’s root user, and found an issue that: pure PyTorch program saves model(torch.save) and has the read permission for non-root users, but ignite Checkpoint handler saves model and only has read permission for root users. You can reproduce this issue by below 2 MONAI examples:

  1. https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/torch/unet_training_dict.py The saved model will be:
-rw-r--r-- 1 root root 19253862 Mar 26 02:08 best_metric_model_segmentation3d_dict.pth
  1. https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/ignite/unet_training_dict.py The saved model will be:
-rw------- 1 root root 57738580 Mar 26 01:54 net_checkpoint_50.pt

ignite version is 0.4.4, could you please help take a look at this issue? As we are close to MONAI and Clara next release, it’s an urgent bug for us.

Thanks in advance.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
vfdev-5commented, Mar 26, 2021

@Nic-Ma workaround now is to set atomic=False in ModelCheckpoint or DiskSaver.

I’ll fix the issue for atomic=True in a PR.

Thanks for reporting !

0reactions
vfdev-5commented, Mar 26, 2021

@Nic-Ma sure! Our #1880 should fix the issue and you could update MONAI code along with our new release in future.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Correcting the permissions on the folder with VHDS files ...
To do this, follow these steps for correcting the permissions on the folder with VHDS files & checkpoints for host level Hyper-V guest...
Read more >
Check Point SecurePlatform Command Reference
In SecurePlatform, executable operating system files are located in the following directories: ... Description: search device for bad blocks.
Read more >
Change file permissions for Checkpoint r80.30 so scpuser can ...
I found out how to change the permissions for the file. Using the admin account for the firewall I logged in via SSH...
Read more >
PermissionError: [Errno 13] Permission denied - Stack Overflow
This happens if you are trying to open a file, but your path is a folder. This can happen easily by mistake. To...
Read more >
File | Vector documentation
The directory used to persist file checkpoint positions. ... Please make sure the Vector project has write permissions to this dir. Examples.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found