Something breaks my cache file, then exceptions are encountered

I haven’t yet worked out what triggers this, but from time to time my my sqlite cache appears to break, after which all calls to requests.get fail with the exception below. Deleting the cache file obviously fixes the problem but that’s not ideal! Have you seen this before?

Traceback (most recent call last):
  File "tool/main.py", line 188, in <module>
    main()
  File "tool/main.py", line 62, in main
    for xls_url in get_excel_urls(download_url(INDEX_URL)):
  File "tool/main.py", line 101, in download_url
    response = requests.get(url)
  File "/home/venv/local/lib/python2.7/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/home/venv/local/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/venv/local/lib/python2.7/site-packages/requests_cache/core.py", line 111, in request
    hooks, stream, verify, cert)
  File "/home/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/venv/local/lib/python2.7/site-packages/requests_cache/core.py", line 89, in send
    response, timestamp = self.cache.get_response_and_time(cache_key)
  File "/home/venv/local/lib/python2.7/site-packages/requests_cache/backends/base.py", line 64, in get_response_and_time
    if key not in self.responses:
  File "/home/venv/lib/python2.7/_abcoll.py", line 348, in __contains__
    self[key]
  File "/home/venv/local/lib/python2.7/site-packages/requests_cache/backends/storage/dbdict.py", line 171, in __getitem__
    return pickle.loads(bytes(super(DbPickleDict, self).__getitem__(key)))
  File "/home/venv/lib/python2.7/copy_reg.py", line 50, in _reconstructor
    obj = base.__new__(cls, state)
TypeError: ('dict.__new__(CaseInsensitiveDict): CaseInsensitiveDict is not a subtype of dict', <function _reconstructor at 0x7fe49d3039b0>, (<class 'requests.structures.CaseInsensitiveDict'>, <type 'dict'>, {'Accept-Encoding': 'gzip, deflate, compress', 'Accept': '*/*', 'User-Agent': 'python-requests/1.2.0 CPython/2.7.4 Linux/3.8.0-23-generic'}))

[liyanchang]

Just poking around a bit, it seems that requests has rewritten CaseInsensitiveDict in https://github.com/kennethreitz/requests/commit/f7596c75dce4e87ab83bdf74e8f120a4b1a5ff03#diff-2bdbe7e19f5215e8c319573cdd114f01

As you can see, before that change, isinstance(CaseInsensitiveDict(), dict) was true and now, isinstance(CaseInsensitiveDict(), dict) is false

For those of you keeping track of versions, it changed between 1.2.0 and 1.2.1

Knowing this, reproduction is pretty straightforward: Use requests 1.2.0, create a cache, then try to use it with requests 1.2.1.

As caches are supposed to be temporary and that trying to migrate the caches are fraught with risk, I recommend that:

1. The issue be closed with a note on how to clear your cache. OR, if you’re feeling nice:
2. That DbPickleDict be modified to try/catch the type error and clear out the cache when reached.

[JWCook]

For reference: See also #102.