Sub-second Timedelta?

Great project. I just switched an app over from Schedule.

One question, and I may be asking Rocketry to do something not in its nature. Is there an option to use Timedeltas lower than one second? ms and milliseconds are in the source and parsing them doesn’t seem to explode Task assignment. However, they don’t fire as often as expected.

In the example below I’ve fiddled with cycle_sleep down to 0.005 and None, but still only get 2-3 per second.

Is this supported? No problem if not, I’ll have my tasks do it internally.

app = Rocketry(config={"cycle_sleep": None})

@app.task("every 100 ms")
def do_things():
    print(str(datetime.now()) + " hello")

if __name__ == "__main__":
    app.run()

2022-08-31 15:16:12.324561 hello
2022-08-31 15:16:12.713914 hello
2022-08-31 15:16:13.102267 hello
2022-08-31 15:16:13.488618 hello
2022-08-31 15:16:13.874968 hello
2022-08-31 15:16:14.261318 hello
2022-08-31 15:16:14.647670 hello
2022-08-31 15:16:15.035021 hello
2022-08-31 15:16:15.422372 hello
2022-08-31 15:16:15.812726 hello
2022-08-31 15:16:16.207084 hello
2022-08-31 15:16:16.599441 hello
2022-08-31 15:16:16.993798 hello

[Miksus]

Actually, maybe the random condition could work like this:

@app.task(randomly(0.2, period=minutely))
def do_things():
    ...

In this case, it’s with 20% chance true every minute. Note that minutely is a time period and not a condition in this example.

Illustration how this would work:

• Time is 22:32:00
• The randomly is re-evaluated. Let’s say that we are lucky and it happens to be True
• The condition is true for the next minute till the period ends (from 22:32:00 to 22:33:00
• Then at 22:33:00 it is re-evaluated and this time it was False
• The condition will stay false the next minute

In this case the every (or timespan) could be fixed to the first check of the condition. And perhaps if the period is not given, it’s determined from the task so we could support both ways.

[Miksus]

Thanks for the idea! I think the second option can be first implemented and then thought whether the first is useful and not too confusing.

However, not sure whether this:

@app.task(randomly(0.2, period=minutely))
def do_things():
    ...

should be true over the period if its state ended up to be true or be bounded with the task run (like daily checks the task has not run today). The former means that the condition could be easily and arbitrarily combined with anything but the latter is easier to use.

---

About this:

One other question, and I’m sure I’m missing it. I’d still like to be able to hand tasks initialization values, or at least a non-global object reference. I’m looking at SimpleArg, ex:

I think we could import SimpleArg. The reason is that it’s mostly used in testing as I was not entirely sure how useful it is usually. In most cases, you should be able to pass the value itself as-is. It’s actually just an argument wrapper for any Python object.

The answer to your question is that there is an argument parameters in the task init so this should work:

@app.task("every 300 ms", execution="thread", parameters={"item": SimpleArg("Foo")})
def do_things(item):
    ...

Or alternatively just:

@app.task("every 300 ms", execution="thread", parameters={"item": "Foo"})
def do_things(item):
    ...

In these cases the argument item will have the value Foo. You can also set that later after the initiation if you wish so (in another task using the Task argument or Session).

---

And I think I understand what you are after: sort of creating states in the task. At least you could pass a mutable object as the parameter and modify that if you don’t use execution="process". Alternatively, if the task itself represents sort of a state (like fetch data) you could just use rocketry.args.Return.

Maybe there could be room for this sort of an argument. Maybe a Namespace which materializes to a simple namespace object.

---

Also as you seem to be interested in the args/parameter mechanics more. Those are quite undocumented as well (as the library is so vast that it’s a full time job to write documentations) but I inherently it was simple: just pass this dict as parameters to this task (function). But then multiprocessing complicated the issue and I needed a way to “materialize” the arguments. Not everything is pickled so I needed a way to “stage” the value of a parameter to make it in a form that can be pickled and then when the process has started, the argument is “materialized” (turned to the actual value). This is relevant for some use cases:

• Pass database connection to a task as connections cannot be pickled (staging turns it to connection string and materialization to actual connection)
• Large files: instead of slowing the scheduler down by read the argument value (the file), the materialization in the child process can do it.

---

There are also some aspects that I’m not sure whether they fit the current idea of the library. Currently, all of the session parameters are passed to tasks that have their names in the argument signature and not sure if that’s maybe too flexible. I can show this later what I mean.

But it seems docs are indeed slightly off with the SimpleArg. Thanks for spotting!