question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Fix `RecentlyUsedContainer` regression

See original GitHub issue

#2115 added typing to the _collections module. Unfortunately, it also introduced a regression that breaks test_urllib3_pool_connection_closed in the requests test suite, as noticed in https://github.com/urllib3/urllib3/pull/2316/checks?check_run_id=3049608329. __setitem__ is implemented like this in 1.26.x:

https://github.com/urllib3/urllib3/blob/ba95e9eac73452d3bccfb5413b00d9a4fe3e4c31/src/urllib3/_collections.py#L61-L74

And it’s now implemented like this in main:

https://github.com/urllib3/urllib3/blob/baba8c2b4aabda8907a777039baa3f8758e4333a/src/urllib3/_collections.py#L133-L155

When len(self._container) == self._maxsize == 0, self._container[key] = value is now called after self._container.popitem(last=False), which is why that popitem call raises an exception.

Applying this hacky diff fixes the requests test:

diff --git a/src/urllib3/_collections.py b/src/urllib3/_collections.py
index 32d5330e..798ba432 100644
--- a/src/urllib3/_collections.py
+++ b/src/urllib3/_collections.py
@@ -139,15 +139,15 @@ class RecentlyUsedContainer(Generic[_KT, _VT], MutableMapping[_KT, _VT]):
                 # size of the pool. Because accessing a key should move it to
                 # the end of the eviction line, we pop it out first.
                 evicted_item = key, self._container.pop(key)
+                self._container[key] = value
             except KeyError:
+                self._container[key] = value
                 if len(self._container) >= self._maxsize:
                     # If we didn't evict an existing value, and we've hit our maximum
                     # size, then we have to evict the least recently used item from
                     # the beginning of the container.
                     evicted_item = self._container.popitem(last=False)
 
-            # Finally, insert the new value.
-            self._container[key] = value
 
         # After releasing the lock on the pool, dispose of any evicted value.
         if evicted_item is not None and self.dispose_func:

@haikuginger What are your thoughts on this? Is it possible to revert to the original logic?

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:2
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
haikugingercommented, Jul 12, 2021

@SethMichaelLarson ah good point; I misunderstood. That approach looks sound to me.

0reactions
pquentincommented, Jul 13, 2021
Read more comments on GitHub >

github_iconTop Results From Across the Web

urllib3/CHANGES.rst at main - GitHub
Fixed thread-safety issue where accessing a PoolManager with many distinct origins would cause connection pools to be closed while requests are in progress...
Read more >
urllib3 [python-library] - Occam :: Details
Fixed regression in 1.21 that threw exceptions when users passed the socket_options flag ... Made RecentlyUsedContainer (and consequently PoolManager) more ...
Read more >
mozilla-central: changeset 607774 ...
(Pull #1154) - -* Fixed regression in 1.21 that threw exceptions when users passed the - ``socket_options`` flag to the ``PoolManager``.
Read more >
HP TRIM Software Problem and Enhancement Report
Fixed. TRIM will check for the existence of a temporary table before ... The drop-down list shows the list of containers and the...
Read more >
Toolbar Registry Hacks • sketchUcation • 1
fixes a toolbar resizing regression on PC as the restore() call # does not seem to work as the script is first loading....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found