Issue found when trying to use vcrpy with requests and concurrent.futures

Whilst writing a test for some new work I was doing, I encountered an issue where vcr was somehow passing a now deprecated parameter to urllib3, or causing requests to do so. I have reproduced a minimal example below that causes the same issue, and is doing things in much the same way that I was. I have also included a version of the script that doesn’t explode, where there is no vcr involved.

For details sake, here is the python version that the scripts were running under.

Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

And here is the pip freeze (I only did pip install vcrpy requests):

certifi==2017.4.17
chardet==3.0.3
idna==2.5
multidict==2.1.6
PyYAML==3.12
requests==2.17.3
six==1.10.0
urllib3==1.21.1
vcrpy==1.11.1
wrapt==1.10.10
yarl==0.10.2

Version One: Breaks with the bad stack trace (below)

import concurrent.futures
import vcr
import requests

testing_vcr = vcr.VCR(
    cassette_library_dir='cassettes',
    record_mode='once'
)

URLS = [
    'http://www.google.com',
    'http://www.otago.ac.nz'
]

futures = []


def load_url(url, timeout):
    response = requests.get(url, timeout=timeout)
    return response.text


with testing_vcr.use_cassette('urls.yaml'):
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        futures = [executor.submit(load_url, url, 60) for url in URLS]
    done, failed = concurrent.futures.wait(futures)
    for url in done:
        print(url.result())

Version Two: Does not break

import concurrent.futures
import requests

URLS = [
    'http://www.google.com',
    'http://www.otago.ac.nz'
]

futures = []


def load_url(url, timeout):
    response = requests.get(url, timeout=timeout)
    return response.text


with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    futures = [executor.submit(load_url, url, 60) for url in URLS]
done, failed = concurrent.futures.wait(futures)
for url in done:
    print(url.result())

The stacktrace produced:

Traceback (most recent call last):
  File "vcr-futures-issue.py", line 61, in <module>
    print(url.result())
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "vcr-futures-issue.py", line 50, in load_url
    response = requests.get(url, timeout=timeout)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/requests/sessions.py", line 513, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/requests/sessions.py", line 623, in send
    r = adapter.send(request, **kwargs)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/urllib3/connectionpool.py", line 588, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/vcr/patch.py", line 200, in patched_get_conn
    connection = get_conn(pool, timeout)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/urllib3/connectionpool.py", line 250, in _get_conn
    return conn or self._new_conn()
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/vcr/patch.py", line 221, in patched_new_conn
    new_connection = new_conn(pool)
  File "/home/dylanjenkinson/projects/python/venvs/vcrbug/lib/python3.5/site-packages/urllib3/connectionpool.py", line 211, in _new_conn
    strict=self.strict, **self.conn_kw)
TypeError: __init__() got an unexpected keyword argument 'strict'

Hopefully this is enough info, and if there is anything else that I can provide, please let me know.

Cheers, Dylan

[Kartstig]

I am able to repo with threads. Not sure if that’s helpful, but I’m using TheadPoolExecutor.map:

def _fetch_all_direct_deps(action, idents):
    with ThreadPoolExecutor(max_workers=12) as executor:
        deps = executor.map(lambda ident: _fetch_direct_deps(action, ident), idents)
        return [ result for result in deps ]

[kevin1024]

You’re right, we probably should do a new release.