[Possibly fixed in 1.2-beta] Exception in queued task

Hey, don’t know what causes this issue, but our logs are full of such messages:

Exception in queued task: Error: Could not find element with id HmenhKFtHnhTMHZSJ to change
    at [object Object]._.extend.changed (packages/ddp-server/livedata_server.js:188:13)
    at [object Object]._.extend.changed (packages/ddp-server/livedata_server.js:403:10)
    at [object Object]._.extend.changed (packages/ddp-server/livedata_server.js:1262:19)
    at [object Object].subscriptionProto.(anonymous function) [as changed] (packages/meteorhacks_kadira.js:2861:30)
    at Object.cursor.observeChanges.changed (packages/mongo/collection.js:344:11)
    at Object.changed (packages/cultofcoders:redis-oplog/lib/mongo/extendObserveChanges.js:87:26)
    at packages/cultofcoders:redis-oplog/lib/cache/PublicationEntry.js:88:13
    at Array.forEach (native)
    at PublicationEntry.send (packages/cultofcoders:redis-oplog/lib/cache/PublicationEntry.js:87:24)
    at ObservableCollection.send (packages/cultofcoders:redis-oplog/lib/cache/ObservableCollection.js:138:9)

Because there are no timestamps in our PM2 logs, I can’t see when they did happen. I’ll report if I get such messages again with the current 1.2 config.

[dnish]

@theodorDiaconu Nope, it does. We had a big issue with the bulk removes, they “killed” one of our app instances most of the time we’ve started the job. The instance started to eat a lot of CPU and the loop delay increased up to 25ms, while the app began to lag on the user side. I guess this happened because of the remove oplog notifications (100k docs per day). I hope that redis-oplog solves this issue, because we are now able to control what Meteor should computate and what could be ignored. We’ve fired the cronjob today and there were no problems.

We want to try it also in one of our other application which handles 1,5k concurrent connections on a single VPS. There we currently using socket.io to get notifications about a converting process, but this solution has downsides (f.e. if the user looses connections, session state can’t be established again). Because we send a lot of notifications, we’ve decided us against Meteors live data system. But we want to check if redis-oplog could handle the load.

if you use publish-composite I made a package that boosts the speed of optimistic-ui dramatically

Yep, we’ve already replaced the original one with your version. Works perfectly 😃

[theodorDiaconu]

@dnish this is very cool. I’m glad. I’m gonna close this issue, unless it happens again, and if it does, I will try to catch wrap the error, so we have an idea of what collection it is and what publication and maybe dump the store so we see exactly whatsup