Restarting/redeploy WebJob can cause DailySchedule to run twice
See original GitHub issueUsing <package id="Microsoft.Azure.WebJobs.Extensions" version="2.0.0-beta1-10332" targetFramework="net461" />
I have a DailySchedule running each day at 1 AM.
internal class BackupScheduler : DailySchedule
{
public BackupScheduler() : base("01:00:00")
{
}
}
Sometimes the System.Timers.Timer will trigger the job before 1AM
The status file written to azure-webjobs-hosts\timers\{hostid\{functionId}\status}
Looks like this:
{"Last":"2016-07-28T00:59:59.8690904+00:00","Next":"2016-07-28T01:00:00"}
The “Next” value is wrongly calculated to only a few milliseconds after the “Last” run. This is not a problem if the process keeps running, because (as I understand) the status file is only read on startup.
But if the WebJobs is redeployed or simply restarted. The CheckPastDueAsync
in ScheduleMonitor
will find that the timer IS past due and should be started.
I have fixed it locally with a hack; But this should be fixed in a proper way.
internal class BackupScheduler : DailySchedule
{
public BackupScheduler() : base("01:00:00")
{
}
public override DateTime GetNextOccurrence(DateTime currentTime)
{
TimeSpan threshold = TimeSpan.FromSeconds(10);
DateTime nextOccurrence = base.GetNextOccurrence(currentTime);
if (currentTime + threshold <= nextOccurrence)
{
return nextOccurrence;
}
return nextOccurrence.AddDays(1);
}
}
Issue Analytics
- State:
- Created 7 years ago
- Reactions:3
- Comments:18 (6 by maintainers)
Top GitHub Comments
Yes! Thank you for the details @Thnslj, this helps enormously. I think I know what’s going on now – it’s possible for a timer to fire just before its ‘officially’ scheduled time (as we’ve written it in the blob). So if you’ve said you want it to run every day at midnight and we fire the timer just before midnight on the 1st, we’ll calculate that the next run should still be at midnight on the 1st – because that’s the next midnight that we see. It doesn’t affect anything if the site remains running but once it restarts, it consults the status blob and sees that it’s past due so it runs again.
I’ll look into a proper fix.
👍 All working for me now, awesome stuff!