Performance issue with task["depends"]

Looking up the dependencies of some tasks lakes much longer than loading the tasks. This causes some noticeable lag in the new version of vit, a taskmanager front end. See https://github.com/scottkosty/vit/issues/216 for the related issue.

I’ve wrote a script to illustrate the problem: tasklib.zip It creates one base task and 40 tasks that depend on it. In the first loop it just iterates over the tasks and converts them to their string representation. The second loop retrieves the id of the depends.

The result looks like this on my machine:

loop1 time: 0.015522241592407227  tasks: 41
loop2 time: 0.33356642723083496  tasks: 41

Retrieving the id of the depends takes about 20x as long as loading the tasks. I suppose this is because of LazyUUIDTaskSet which looks up the base task by its UUID 40 times. Is there an easy way to improve this?

[thomasrebele]

Yes, the LRU cache fixes the performance problem. Thank you for adding it!

[robgolding]

@thomasrebele @thehunmonkgroup I’ve released version 2.1.1 which includes a performance enhancement that should fix this issue. Please give it a go!