question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

xreadgroup throws an exception if message was trimmed when reading pending messages

See original GitHub issue

Version: 3.0.1

Platform: Python 3.6.6 on Debian 9 and Redis 5.0.2

Description: If messages get deleted from a STREAM via XADD with MAXLEN or XTRIM in a stream where there is an existing consumer group after it’s consumed but before being acked, when you consume the pending messages using XREADGROUP with id 0, redis returns a (nil) as the payload of the messages instead of field key/value pairs. This causes xreadgroup to throw an exception:

[2019-01-10 12:00:10,206] [16] [ERROR] Error reading messages
Traceback (most recent call last):
  File "/code/app/daemons/__init__.py", line 38, in run
    stream_messages = self.redis_client.xreadgroup("consumer-group", "consumer", {"stream": "0"}, count=1000)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2185, in xreadgroup
    return self.execute_command('XREADGROUP', *pieces)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 755, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 774, in parse_response
    return self.response_callbacks[command_name](response, **options)
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 275, in parse_xread
    return [[nativestr(r[0]), parse_stream_list(r[1])] for r in response]
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 275, in <listcomp>
    return [[nativestr(r[0]), parse_stream_list(r[1])] for r in response]
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 246, in parse_stream_list
    return [(r[0], pairs_to_dict(r[1])) for r in response]
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 246, in <listcomp>
    return [(r[0], pairs_to_dict(r[1])) for r in response]
  File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 195, in pairs_to_dict
    it = iter(response)
TypeError: 'NoneType' object is not iterable

You can see this is the expected behaviour in Redis’s source code: https://github.com/antirez/redis/blob/5.0.2/src/t_stream.c#L1031

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
soloestoycommented, Jan 11, 2019
0reactions
andymccurdycommented, Jan 27, 2019

Merged. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

xreadgroup throws an exception if message was trimmed ...
Version: 3.0.1 Platform: Python 3.6.6 on Debian 9 and Redis 5.0.2 Description: If messages get deleted from a STREAM via XADD with MAXLEN...
Read more >
Redis stream XReadGroup not reading new messages even if ...
I am currently facing a problem that if I keep the application (involving this code) idle for 10-12 hours, XReadGroup is not able...
Read more >
Redis Commands — redis-py dev documentation
In this mode invalidation messages are reported for all the prefixes specified, regardless of the keys requested by the connection. optin when broadcasting...
Read more >
Spring Data Redis
When you read with messages via a Consumer Group , the server will remember that a given message was delivered and add it...
Read more >
Database.Redis - Hackage - Haskell.org
Redis commands behave differently when issued in- or outside of a transaction. ... Throws an exception if the connection to the Redis server...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found