question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

monthly_partitioned_config is broken because of wrong cron scheduling

See original GitHub issue

Summary

I’m a bit surprised this hasn’t been reported before, so maybe it’s on me.

Basically, when using a monthly_partitioned_config and running with any key, I get the following error:

croniter.croniter.CroniterBadCronError: [0 0 0 * *] is not acceptable, out of range

inspecting the source code the reason is the method get_cron_schedule has 1 as a default value for the argument day_of_week. This method receives a monthly ScheduleType as only argument when running a monthly partitioned job. Therefore the output cron schedule is “0 0 0 * *”, which isn’t valid for the croniter library that dagster uses.

The fix to this would be as simple as changing the default value of that argument to 1. I can open a PR if you’re fine with it.

Reproduction

from dagster import job, monthly_partitioned_config
from datetime import datetime


@monthly_partitioned_config(start_date=datetime(2021, 1, 1))
def config(start: datetime, _end: datetime):
    pass


@job(config=config)
def some_job():
    pass

if __name__ == "__main__":
    previous_month = "2021-03-01"
    some_job.execute_in_process(partition_key=previous_month)

Additional Info about Your Environment

dagster==0.13.14


Message from the maintainers:

Impacted by this bug? Give it a 👍. We factor engagement into prioritization.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
gibsondancommented, Feb 17, 2022

This was fixed in 0.13.15, yeah. Either upgrading dagster or downgrading croniter on that version will fix this.

0reactions
Tiri1992commented, Feb 17, 2022

I’m using version 0.13.8 and just ran into the same issue. Did we get a fix for this? I’ll downgrade croniter in the meantime.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Managing PostgreSQL partitions with the pg_partman ...
The pg_cron scheduler automatically executes SQL statements, functions, and procedures defined in your databases. The following example uses the events table ...
Read more >
Cron job troubleshooting guide
If you're adding a new cron job and it is not working. ... Invalid permissions can cause your cron jobs to fail in...
Read more >
Top Reasons Your Cron Job Isn't Running
This article walks through the most common reasons why your cron job may not be working as expected: schedule errors, environmental changes, ...
Read more >
Cron - GitLab Docs
Cron scheduling uses a series of five numbers, separated by spaces: ... Run once a month at midnight of the first day of...
Read more >
How I use cron in Linux
The cron service can schedule tasks on a repetitive basis, such as daily, weekly, or monthly. In this article, I'll introduce the cron...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found