Failure to lock

First of all, let me congratulate you for the way you handle your issues, your answers are complete and thoughtful.

Now, to my issue: I’m seeing a relatively small (1-3% out of around 100.000 tries) of LockError when trying to lock a resource. It exceeds the 10 attempts to lock.

What are the possible causes for a lock to fail? From what I could gather, I am not attempting to lock the same resource twice (there are no other calls to lock a resource with the same tag LOCK:STATUS:PAYLOAD:${id})

Is there something I could do about it? I know it’s hard without sharing any more code.

Thanks in advance.

[mike-marcacci]

You’re super welcome; and glad to help! As far as open source projects are concerned, the issues that get filed here tend to be amazingly high-quality and well thought out. That makes it easy to respond with similar thoughtfulness 😃

[mike-marcacci]

Hi @bzuker - my apologies for the delay!

That’s a great question, with a couple different possible answers.

Let’s say you have redlock configured for 3 attempts instead of 10. One possibility is simply that the lock was acquired for the entire duration of your attempts. This can happen a couple different ways.

In the simplest case, one actor (A) acquires the lock and holds it for the entire time that a second actor (B) attempts to acquire a lock for the same resource:

   acquire        release
A     |              |
---------------------------
B       |    |    |
        1    2    3 

Another possibility is that if the resource is under reasonable contention, that multiple different actors hold the lock for the duration that B is attempting to acquire the lock:

  acquire   release
A     |        |
---------------------------
        |    |     |
B       1    2     3 
        |    |     |
---------------------------
C                |       |
            acquire    release

The first scenario can be solved by increasing the retryCount, or by increasing the retryDelay between attempts. However, increasing the delay would make the second scenario more likely, since it increases the opportunity for C to sneak in and acquire the lock first.

Another totally different scenario is only applicable in a high-availability setup. In this scenario, under very high contention, different actors can partially acquire locks on different redis servers, leading to a scenario where nobody fully acquires the lock, and everyone has to retry. However, if they all retry at the same time, we might simply repeat this scenario. To prevent this, it’s important that the retryJitter is high enough to avoid this kind of resonance.

At the end of the day, the most corse setting to play with is going to be retryCount: if you’d rather wait than error, turn this up (you can go as high as Infinity, also representable as -1 if your configs need to be JSON serializable); if you would rather stop the operation, though, then you can turn it down as low as 0.

Hope that helps!