How to control the speed of performing a job

I need to make a request to an API, but the API limits the speed of requests to 10 per sec. when the redis queue A has 1000 jobs called jobB to perform, how to control the speed of performing the job, or performing it at a constant speed, e.g. 10 jobs per second.

  const jobs = {
    jobB: {
      plugins: ["JobLock"],
      pluginOptions: {
        JobLock: {}
      },
      perform: async (a, b) => {
        // request a api
      }
    }
  };

[juneapp]

Hi @evershy can you provide me with your final RateLimit version? Thanks a lot and best regards Henning

[evantahler]

At a quick glance that looks a good stat! Be sure to write some tests for it.

There’s a race condition you can get into where you read the key (it has a value of 9) and by the time you write back to it, another worker has started the 10th job before this worker did. You can use watch to prevent this - https://redis.io/topics/transactions#optimistic-locking-using-check-and-set

Here’s an example of the node-resque scheduler using watch to precent a command if someone else changes the key https://github.com/actionhero/node-resque/blob/master/src/core/scheduler.ts#L268-L280