Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Improve performance for heavy users
See original GitHub issueIn our project, we heavily use (I can hear you think “abuse”) Quartz.NET. We actually use Quartz.NET more as a thread scheduler than a job scheduler. We have a lot of jobs, with multiple instances for a given job and short intervals between executions. We have no need for persistence, so we only use RAMJobStore.
We’re running on .NET Framework 4.8, but planning for a migration to .NET 5+. One step in that direction was to upgrade Quartz.NET from version 2.x to version 3.x. We did a POC for this, and learned that the upgrade came with a cost. I suspect part of this cost comes from the async machinery, but will need a little more time to dig into this. The async story has improved a lot on .NET (Core), but we cannot upgrade to .NET (Core) in a single pass.
This issue is NOT to complain about slower performance. We’re extremely grateful for all the work you have been doing, so let me just say: Thanks!!
I hope to use this issue to:
- raise awareness of the impact that an upgrade from 2.x to 3.x (and probably 4.x) has for those projects making heave use of Quartz.NET.
- inform and learn about areas in Quartz.NET that could use some “love”.
- help you (and ourselves) to improve performance and reduce allocations, where possible.
RAMJobStore
When I learned of the reduced performance (and higher CPU usage) of Quartz.NET, I did a quick scan of the (Quartz.NET) code base to look for opportunities to improve performance.
I identified improvements to RAMJobStore (for which I’ll submit a PR) that can be categorized as:
- Where possible do stuff outside of the instance lock, hereby moving the needle (just a little) for concurrency.
- Avoid virtual calls as these are a lot more costly than direct calls.
- Do not use
RetrieveJobInternal(JobKey jobKey)to only check if a job exists as that method comes with an extra cost due to the cloning of IJobDetail. - Prefer for loops over foreach to avoid cost of allocating and disposing an enumerator.
- Avoid using ConcurrentDictionary for a dictionary that is only ever used while owning the instance lock.
I do not expects these changes to resolve all our issues. Baby steps …
Future
Once my work on RAMJobStore is done, I’ll scan for other areas where improvements can be realised. If there are areas that are known to be less optimal (time is a limited resource for everyone), I’d be happy to take a stab at improving these.
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
To see where we’re at right now, I used an improved version of SchedulerBenchmark (which I’ll soon create a PR for) against the tip of the v4 branch and the commit before I started improving Quartz.NET performance (with high-precision SimpleTriggerImpl changes on top).
Before
After
Observations:
Next steps:
Some of these changes should have an even greater impact on performance, concurrency and memory allocations.
Thanks for the fast feedback!!
I do not plan to introduce any breaking changes, even though - as you mentioned - changing some methods to return concrete types, or take concrete types as arguments might surely be beneficial.
Changing from IEnumerable<T> to List<T>, for example, may be great for perf but from a consumer point of view you’re moving from something that has the clear intent of being read-only to something which is clearly not.
Let’s start with small harmless changes for 4.x, and take it from there. I’ll also try to find time to write benchmarks that we can run against 2.x and 4.x.
The change I proposed to replace the last occurrence of ConcurrentDictionary<TKey,TValue> with Dictionary<TKey,TValue> has already dropped dead in the water:
ConcurrentDictionary<TKey,TValue>with a call toDictionary<TKey,TValue>.TryGetValue(TKey key, out TValue value)andDictionary<TKey,TValue>.Remove(TKey key).We do miss a few opportunities where we could delay or even avoid acquiring the instance-level lock.