xreadgroup throws an exception if message was trimmed when reading pending messages
See original GitHub issueVersion: 3.0.1
Platform: Python 3.6.6 on Debian 9 and Redis 5.0.2
Description: If messages get deleted from a STREAM via XADD with MAXLEN or XTRIM in a stream where there is an existing consumer group after it’s consumed but before being acked, when you consume the pending messages using XREADGROUP with id 0, redis returns a (nil) as the payload of the messages instead of field key/value pairs. This causes xreadgroup to throw an exception:
[2019-01-10 12:00:10,206] [16] [ERROR] Error reading messages
Traceback (most recent call last):
File "/code/app/daemons/__init__.py", line 38, in run
stream_messages = self.redis_client.xreadgroup("consumer-group", "consumer", {"stream": "0"}, count=1000)
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2185, in xreadgroup
return self.execute_command('XREADGROUP', *pieces)
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 755, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 774, in parse_response
return self.response_callbacks[command_name](response, **options)
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 275, in parse_xread
return [[nativestr(r[0]), parse_stream_list(r[1])] for r in response]
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 275, in <listcomp>
return [[nativestr(r[0]), parse_stream_list(r[1])] for r in response]
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 246, in parse_stream_list
return [(r[0], pairs_to_dict(r[1])) for r in response]
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 246, in <listcomp>
return [(r[0], pairs_to_dict(r[1])) for r in response]
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 195, in pairs_to_dict
it = iter(response)
TypeError: 'NoneType' object is not iterable
You can see this is the expected behaviour in Redis’s source code: https://github.com/antirez/redis/blob/5.0.2/src/t_stream.c#L1031
Issue Analytics
- State:
- Created 5 years ago
- Comments:7 (6 by maintainers)
Top Results From Across the Web
xreadgroup throws an exception if message was trimmed ...
Version: 3.0.1 Platform: Python 3.6.6 on Debian 9 and Redis 5.0.2 Description: If messages get deleted from a STREAM via XADD with MAXLEN...
Read more >Redis stream XReadGroup not reading new messages even if ...
I am currently facing a problem that if I keep the application (involving this code) idle for 10-12 hours, XReadGroup is not able...
Read more >Redis Commands — redis-py dev documentation
In this mode invalidation messages are reported for all the prefixes specified, regardless of the keys requested by the connection. optin when broadcasting...
Read more >Spring Data Redis
When you read with messages via a Consumer Group , the server will remember that a given message was delivered and add it...
Read more >Database.Redis - Hackage - Haskell.org
Redis commands behave differently when issued in- or outside of a transaction. ... Throws an exception if the connection to the Redis server...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
relate to https://github.com/antirez/redis/issues/5718
Merged. Thanks!