TypeError: HTTPHeaderDict is not a subtype of dict

It appears that some time in the last couple of months something changed within the Python/Requests/Requests-Cache stack which caused this error to be thrown for all of my apps which use requests-cache and which had caches which were created before the change.

Deleting all the caches caused the problems to go away, but cleaner handling would be nice. If the cache can’t be made robust to changes to the underlying stack, perhaps at least a better error message could be given.

At a glance it appears that the problem is likely related to this PR & commit for urllib3 which changed the superclass of HTTPHeaderDict from dict to MutableMapping: https://github.com/shazow/urllib3/pull/679 https://github.com/shazow/urllib3/commit/64adf9f9df6a4e76f921a13ec81744af7438c4b2 but it’s not clear to me if the breaking change that they made was to a public API and whether it was avoidable, so I’m just going to dump my report here and let the two projects hash it out.

  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests/api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests_cache/core.py", line 128, in request
    **kwargs
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests_cache/core.py", line 99, in send
    response, timestamp = self.cache.get_response_and_time(cache_key)
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests_cache/backends/base.py", line 69, in get_response_and_time
    if key not in self.responses:
  File "/Users/tfmorris/anaconda/lib/python2.7/_abcoll.py", line 388, in __contains__
    self[key]
  File "/Users/tfmorris/anaconda/lib/python2.7/site-packages/requests_cache/backends/storage/dbdict.py", line 163, in __getitem__
    return pickle.loads(bytes(super(DbPickleDict, self).__getitem__(key)))
  File "/Users/tfmorris/anaconda/lib/python2.7/copy_reg.py", line 50, in _reconstructor
    obj = base.__new__(cls, state)
TypeError: ('dict.__new__(HTTPHeaderDict): HTTPHeaderDict is not a subtype of dict', <function _reconstructor at 0x10073da28>, (<class 'requests.packages.urllib3._collections.HTTPHeaderDict'>, <type 'dict'>, {'access-control-max-age': ('Access-Control-Max-Age', '86400'), 'transfer-encoding': ('Transfer-Encoding', 'chunked'), 'access-control-allow-method': ('Access-Control-Allow-Method', 'GET, OPTIONS'), 'server': ('Server', 'nginx/1.1.19'), 'x-ol-stats': ('X-OL-Stats', '"IB 1 0.005 TT 0 0.005"'), 'connection': ('Connection', 'keep-alive'), 'date': ('Date', 'Thu, 03 Sep 2015 23:36:25 GMT'), 'access-control-allow-origin': ('Access-Control-Allow-Origin', '*'), 'content-type': ('Content-Type', 'application/json')}))

[reclosedev]

Yes caches can be broken in case of some internal requests/urllib3 change. It probably can be handled by adding versioning to cache, but it will complicate things.

Did removal of old database helped?

[JWCook]

For reference: See also #102.

This is much less of an issue now, due to better error handling and the option for custom request matching.