question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The task will cache?

See original GitHub issue

Expected Behavior

Changes as database data changes

Current Behavior

If only one thread is opened, the data will never change. When I open 20 threads, the data of the thread after the database data changes is updated.

Steps to Reproduce

  1. I use the Flask-apscheduler.
  2. Use SQLAlchemy to get database data.
  3. Create a task that runs every second to get database data.
  4. Chang the data in database.

Context (Environment)

Win10 1903 python3.7 apscheduler3.6.3

Detailed Description

task code

def check_node_health():
    for node in Node.query.all():
        if node.records:
            last_time = datetime.datetime.min
            print(node.records)
            for record in Record.query.filter_by(chip_id=node.chip_id).all():
                last_time = max(last_time, record.create_datetime)
            print(last_time)
            print((datetime.datetime.now() - last_time).seconds)

config code

SCHEDULER_API_ENABLED = True
JOBS = [
    {
        'id': 'node_health_checker',
        'func': check_node_health,
        'args': '',
        'trigger': 'interval',
        'seconds': 1
    }
]
SCHEDULER_EXECUTORS = {
    'default': ThreadPoolExecutor(20),
}

批注 2020-02-04 233511

As the picture shows:

The last line of data is what I added at runtime Duplicate tasks run by the thread, showing updated data,but some are not. So I suspect that the thread executing the task will cache the data.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
agronholmcommented, Feb 5, 2020

Your problem does not seem to have anything to do with scheduling. APScheduler does not set up or operate the database connection you’re using, so why have you reported this as an APScheduler problem? For the record, I think your problem stems from transaction isolation. You might not be closing transactions properly.

0reactions
0x587commented, Feb 5, 2020

Thanks for your guidance, I found the problem.Thanks!!!

Read more comments on GitHub >

github_iconTop Results From Across the Web

c# - When to cache Tasks?
I have trouble understanding how this actually helps more than just storing the results. When a method is marked with the async modifier, ......
Read more >
How to cache Task objects for improving performance
Task caching is an approach towards caching in which instead of caching the results of the execution of a Task, you cache the...
Read more >
Cache@2 - Cache v2 task
Improve build performance by using this task to cache files, ... The variable to set to true when the cache is restored (i.e....
Read more >
Caching and Persisting Data
Prefect provides a few ways to work with cached data between tasks or flows. In-memory caching of task inputs is automatically applied by...
Read more >
Caching
Task caching is useful when a user knows that many executions with the same inputs may occur. For example, consider the following scenarios:....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found