force_reset makes vcr non thread-safe

I tried to use vcr to write tests for an old program that uses threadpools internally. It turned out that this doesn’t work and I believe it’s due to the use of force_reset when the unpatched base classes are instantiated. The reset is necessary since httplib.HTTPSConnection refers to httplib.HTTPConnection by name.

The following program shows unexpected errors on my machine:

import sys, urllib2

from concurrent import futures
from vcr import use_cassette
from vcr.errors import CannotOverwriteExistingCassetteException

@use_cassette('not-actually-used.yaml', record_mode='none')
def load_all(*urls):
    print 'Submitting URLs...',
    with futures.ThreadPoolExecutor(max_workers=2) as executor:
        fs = [executor.submit(urllib2.urlopen, url) for url in urls]
        print 'done!'
        for f in futures.as_completed(fs):
            try:
                result = f.result()
                print 'Unexpected result:', result
                sys.exit(1)
            except CannotOverwriteExistingCassetteException as e:
                pass  # Success!
            except Exception as e:
                print 'Unexpected error:', e
                sys.exit(1)


if __name__ == '__main__':
    load_all('https://httpbin.org/get?q=testing+vcrpy+with',
             'https://httpbin.org/get?q=https+and+threads')

The error shown is

unbound method __init__() must be called with VCRHTTPConnectionnot-actually-used.yaml
instance as first argument (got HTTPSConnection instance instead)

and is triggered when the unpatched httplib.HTTPSConnection.__init__ calls the patched httplib.HTTPConnection.__init__.

I see the error about 50% of the time, so I normally run the script in a little loop (Zsh syntax):

errors=0; for i in {1..20}; do python test.py; ((errors += $?)); echo; echo "*** Successes: $((i - errors))/$i, $((100 * (i - errors) / i))%"; done

[IvanMalison]

It is still relevant. Reproducing on vcrpy==4.1.1. @neozenith, can you pls reopen the issue?

@selevit I’m not really that involved in the maintenance of vcrpy anymore, but this is the one area of code I would say I am probably still the resident expert in, and I really don’t think that vcrpy is EVER going to be multi threaded (at least with its current design).

You’re not really reproducing anything, because vcrpy DOES NOT SUPPORT multi threading. The only way it would be able to do so would basically be to put a global lock around everything, at which point you have to ask what the point of multi-threading is in the first place.

[selevit]

A lot of changes have happened to VCRpy since this ticket was opened. As this ticket has become stale and we have a lot of old tickets that need to be groomed, I’m closing this for now to make it easier to see what is still relevant.

However if it is still needed, please feel free to re-open or create a new ticket.

Thanks! 🙏

It is still relevant. Reproducing on vcrpy==4.1.1. @neozenith, can you pls reopen the issue?