KeyError on self._in_use_connections.remove(connection)

Version: 2.10.5 Platform: Python 2.7, Ubuntu This is a very rare occurrence on a very high traffic server but I receive a key error occasionally from here: self._in_use_connections.remove(connection) https://github.com/andymccurdy/redis-py/blob/2.10.5/redis/connection.py#L913 I haven’t been able to reproduce this in testing or root out the cause.

Theories: self._available_connections is a list so you could append in the same connection more then once. Then when you try to self._in_use_connections.add(connection) https://github.com/andymccurdy/redis-py/blob/2.10.5/redis/connection.py#L898 You get no error for using the same connection. Until you try to release _in_use_connections is a set so were the add() multiple times desen’t raise an error remove() will.

[Yanght115]

 def reset(self):
        self.pid = os.getpid()
        self._created_connections = 0
        self._available_connections = []
        self._in_use_connections = set()
        self._check_lock = threading.Lock()

    def _checkpid(self):
        if self.pid != os.getpid():
            with self._check_lock:
                if self.pid == os.getpid():
                    # another thread already did the work while we waited
                    # on the lock.
                    return
                self.reset()

    def get_connection(self, command_name, *keys, **options):
        "Get a connection from the pool"
        self._checkpid()
        try:
            connection = self._available_connections.pop()
        except IndexError:
            connection = self.make_connection()

I guess there’s one case, thread A and B, A get lock and then call reset(), when A set pid with self.pid = os.getpid() but not self._in_use_connections = set(), B enter the _checkpid() and fail to meet the condition if self.pid != os.getpid(), so B return and get connection from old _available_connections

[andymccurdy]

I’m not a fan of simply moving the .pid assignment below the other statements. It might work more than the released version but it’s still not thread safe.

I just created a new PR #1270 that implements what I believe is a fully thread-safe pool. Can anyone please give it a spin and see if it fixes your issue? Thanks!