Lock became infinite?
See original GitHub issueHi.
I have a lock defined as:
lock = redis.Lock(
"servicebook.tasks.emit_servicebook_notifications",
expire=240,
auto_renewal=True
)
I’m acquiring it that way:
if lock.acquire(blocking=False):
try:
...
finally:
lock.release()
Today our customer reported us an issue. Our background task, which relies on redis lock, was not executing. I’ve checked the key and I’ve found that:
127.0.0.1:6379> GET "lock:servicebook.tasks.emit_servicebook_notifications"
"GxwErhZvp2QmLvacH/pVgjGj"
127.0.0.1:6379> TTL "lock:servicebook.tasks.emit_servicebook_notifications"
(integer) -1
The TTL was set to infinite, so lock was always held.
Before I switched to lock.acquire(blocking=False)
we had a code ran with blocking=True
. Our background tasks are managed by RQ and sometimes they were killed due to timeout. To avoid timeouts I changed blocking=True to blocking=False. There were no other changes, upgrades nor manual changes to Redis db.
I wonder how it is possible that lock automatically changed its expiration time to infinite. There is a risk that this would happen again. Any thoughts? Feel free to ask anything If I can help somehow.
NOTE: We have several machines running same app servers (failover & load balancing).
Issue Analytics
- State:
- Created 3 years ago
- Comments:6 (3 by maintainers)
Top GitHub Comments
Just released v3.6.0. I hope it may fix the problem, or help you with the more strict option validation.
Ok so I did some testing locally. It turns out one could do “lock.expire(-1)”. I think negative values should be disallowed. I’ll implement a fix.