AttributeError: 'FilePusher' object has no attribute 'rename_file'

wandb --version && python --version && uname:

• Weights and Biases version: wandb, version 0.9.1
• Python version: Python 3.6.9
• Operating System: Linux

Description

I am using pytorch_lightning and am trying to upload my saved models as they are written to the logging directory. Everything works as intended except that the first saved checkpoint produces an error and is not saved. I am using wandb.save() to save the models when they are written instead of at the end of training. However, the error also occurs when creating WandbLogger and just setting log_model=True. The error appears to be because of wandb and not pytorch_lightning.

What I Did

Logger setup:

wandb_logger = WandbLogger(
    project="testing-project",
)
args.logger = wandb_logger
import wandb
args.weights_save_path = wandb_logger.experiment.dir
checkpoint_glob = os.path.join(wandb_logger.experiment.dir, "*.ckpt")
wandb.save(checkpoint_glob)

Creating a logger like the below also causes the problem:

wandb_logger = WandbLogger(
    project="testing-project",
    log_model=True,
)

Error message:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/dist-packages/watchdog/observers/api.py", line 196, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/usr/local/lib/python3.6/dist-packages/watchdog/observers/api.py", line 369, in dispatch_events
    handler.dispatch(event)
  File "/usr/local/lib/python3.6/dist-packages/watchdog/events.py", line 452, in dispatch
    super(PatternMatchingEventHandler, self).dispatch(event)
  File "/usr/local/lib/python3.6/dist-packages/watchdog/events.py", line 336, in dispatch
    }[event.event_type](event)
  File "/usr/local/lib/python3.6/dist-packages/wandb/run_manager.py", line 712, in _on_file_moved
    self._file_pusher.rename_file(old_save_name, new_save_name, event.dest_path)
AttributeError: 'FilePusher' object has no attribute 'rename_file'

[vanpelt]

Thanks for reporting @HHousen we’re targeting a fix for the next release.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.63. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.