monthly_partitioned_config is broken because of wrong cron scheduling
See original GitHub issueSummary
I’m a bit surprised this hasn’t been reported before, so maybe it’s on me.
Basically, when using a monthly_partitioned_config and running with any key, I get the following error:
croniter.croniter.CroniterBadCronError: [0 0 0 * *] is not acceptable, out of range
inspecting the source code the reason is the method get_cron_schedule has 1 as a default value for the argument day_of_week. This method receives a monthly ScheduleType as only argument when running a monthly partitioned job. Therefore the output cron schedule is “0 0 0 * *”, which isn’t valid for the croniter library that dagster uses.
The fix to this would be as simple as changing the default value of that argument to 1. I can open a PR if you’re fine with it.
Reproduction
from dagster import job, monthly_partitioned_config
from datetime import datetime
@monthly_partitioned_config(start_date=datetime(2021, 1, 1))
def config(start: datetime, _end: datetime):
pass
@job(config=config)
def some_job():
pass
if __name__ == "__main__":
previous_month = "2021-03-01"
some_job.execute_in_process(partition_key=previous_month)
Additional Info about Your Environment
dagster==0.13.14
Message from the maintainers:
Impacted by this bug? Give it a 👍. We factor engagement into prioritization.
Issue Analytics
- State:
- Created 2 years ago
- Reactions:1
- Comments:5 (2 by maintainers)
Top GitHub Comments
This was fixed in 0.13.15, yeah. Either upgrading dagster or downgrading croniter on that version will fix this.
I’m using version
0.13.8
and just ran into the same issue. Did we get a fix for this? I’ll downgrade croniter in the meantime.