Crash when metric is NaN

Using the FastaiV2 callback, when the metric becomes NaN, the error is:

sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: trial_intermediate_values.intermediate_value [SQL: INSERT INTO trial_intermediate_values (trial_id, step, intermediate_value) VALUES (?, ?, ?)] [parameters: (52, 20, nan)]

My workaround is (inside FastAIV2PruningCallback):

    def after_epoch(self) -> None:
        super().after_epoch()
        # self.idx is set by TrackTrackerCallback

        out = self.recorder.final_record[self.idx]
        if np.isnan(out):
            out = np.inf if self.trial.study.direction == optuna.study.StudyDirection.MINIMIZE else -np.inf
        self.trial.report(out, step=self.epoch)

        if self.trial.should_prune():
            raise CancelFitException()

[nzw0301]

Thank you for your clarification! Indeed, I can reproduce the error by using the following simpler code

import optuna


def objective(trial):
    for i in range(10):
        trial.report(float('nan'), i)

    return float('nan')

pruner = optuna.pruners.MedianPruner()
study = optuna.create_study(direction="maximize", pruner=pruner,  storage="sqlite:///example.db")
study.optimize(objective, n_trials=10)

This might be related storage issue of Optuna, so I transfer this issue to the main repository.

[nzw0301]

@lsc64 thank you for letting us know about it! Indeed, this issue is already known. Let me close this issue.