Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TimeStopping Callback fail in verbose mode

See original GitHub issue

System information

OS Platform and Distribution (Ubuntu 19.04): TensorFlow version and how it was installed (pip==2.1.0): TensorFlow-Addons version and how it was installed (pip==0.7.0): Python version: 3.7 Is GPU used? (yes):

Describe the bug

When the fit method stop before the timer, it raises with

AttributeError: 'TimeStopping' object has no attribute 'stopped_epoch'

Code to reproduce the issue

Use tqdm callback

Other info / logs

Proposed fix

...

    def on_epoch_end(self, epoch, logs={}):
        self.stopped_epoch = epoch
        if time.time() >= self.stopping_time:
            self.model.stop_training = True

...

    def on_train_end(self, logs=None):
        if self.verbose > 0:
            formatted_time = datetime.timedelta(seconds=self.seconds)
            msg = "Stopping at epoch {} after training for {}".format(
                self.stopped_epoch + 1, formatted_time
            )
            print(msg)
...

Issue Analytics

State:
Created 4 years ago
Comments:14 (14 by maintainers)

Top GitHub Comments

1reaction

failure-to-thrivecommented, Jan 31, 2020

#967 has merged and fixed the bug I believe. Do we want to keep this open for @failure-to-thrive s implementation or close and start a new one?

#967 is a quick workaround to avoid crashes. It does its job very well but now we should move forward to a comprehensive solution, because for example https://github.com/tensorflow/addons/pull/967#issuecomment-579628853. It’s still #964, I think we should stay here.

0reactions

shun-lincommented, Feb 2, 2020

Closing this issue as @failure-to-thrive’s PR has been merged, thanks!