high cpu use

We see long periods of 100% cpu when scheduling jobs or a lot of cron schedules happen at the same time.

We could try to group similar cron jobs on our side but I wonder if this could be optimised on the side of the library.

Some test results below

setup all tests run on my macbook 2,7 GHz Intel Core i7 since node can only use one core ‘as fast as possible’ means at 100% single core

results creating 10000 empty cron jobs takes ~ 20 seconds 90 mb memory creating 20000 empty cron jobs takes ~ 130 seconds 130 mb memory creating 30000 empty cron jobs takes ~ 394 seconds 190 mb memory

running 10000 jobs scheduled at the same time takes 2 minutes even if they only print a line to the console (cpu peaks)

note the non-linear growth in time needed to create jobs (this is also a problem for us as we need to rebuild cron jobs on a restart)

for reference creating 30000 settimeouts takes 0.171 seconds running 30000 timeouts takes less than a second

const schedule = require('node-schedule')
const util = require('util')
const number = 30000
util.log(`Scheduling ${number} crons`)
const start = new Date().getTime()
for (let i = 0; i < number; i++) {
  const nr = i
  console.log(`create ${nr}`)
  // setTimeout(function(){
  //   console.log(`run ${nr}`)
  // }, 6000)
  schedule.scheduleJob('00 01 23 * * *', function(){
    console.log(`run ${nr}`)
  })
}
const end = new Date().getTime()
util.log(`Ok in ${end-start} ms`)

let p = new Date().getTime()
setInterval(function () {
  p = p + 1000
  const difference = new Date().getTime() - p
  util.log("Difference between expected & actual: " + difference)
}, 1000)

[santigimeno]

BTW, fixed in 1.2.1.

[santigimeno]

With https://github.com/node-schedule/node-schedule/pull/335, the 30000 jobs take ~ 9700ms whereas with current master take ~ 322045ms.