ResponseNotRead when attempting to access response data through respx.calls

I’m working on mocking an api client I’ve written using httpx which manipulates the data before returning it, and I can’t seem to get the raw content from the request (before the manipulation). I’ve reconstructed what the client does, with cookies, headers, etc, to demonstrate the issue still happens with httpbin.org.

As you can see, the request.content is read (it’s none, so an empty byte-string is expected), but the response is readable from httpx, but not respx.

Due to the api client construction, I don’t immediately have access to the raw request (i.e. httpx.get(...) or client.get(...)), and I was hoping the mock would store the content in a retrievable format when passing through.

Note: Although not demonstrated here, this also happens with text and json().

Am I doing something wrong or are my expectations misplaced?

Versions:

• respx (07ca910)
• httpx (0.7.6)

Demo:

class TestGetContent:
    @respx.mock
    def test_get_content(self, mc_client):
        respx.get("https://httpbin.org/json", pass_through=True)
        base_url = httpx.URL("https://httpbin.org")
        cookies = httpx.Cookies()
        cookies.set(
            name="a_cookie", value="value", domain="httpbin.org",
        )
        headers = httpx.Headers(
            {"User-Agent": "my-python-client/0.0.7.dev0", "X-Auth": "auth_value"}
        )
        timeout = httpx.config.TimeoutConfig(timeout=300)
        c = httpx.Client(base_url=base_url, cookies=cookies, headers=headers, timeout=timeout)
        response = c.get("json")
        print(f"response.text={response.text}\n===")
        print(f"response.json()={response.json()}\n===")
        print(f"response.content={response.content}\n===")
        for req, resp in respx.calls:
            print(f"respx request.content={req.content}\n---")
            print(f"respx response.content={resp.content}\n---")

PyTest Result:

>           print(f"respx response.content={resp.content}\n---")

tests/test_content.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [200 OK]>

    @property
    def content(self) -> bytes:
        if not hasattr(self, "_content"):
            if hasattr(self, "_raw_content"):
                raw_content = self._raw_content  # type: ignore
                content = self.decoder.decode(raw_content)
                content += self.decoder.flush()
                self._content = content
            else:
>               raise ResponseNotRead()
E               httpx.exceptions.ResponseNotRead

../../../Library/Caches/pypoetry/virtualenvs/test-py3.7/lib/python3.7/site-packages/httpx/models.py:816: ResponseNotRead
--------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------
response.text={
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}

===
response.json()={'slideshow': {'author': 'Yours Truly', 'date': 'date of publication', 'slides': [{'title': 'Wake up to WonderWidgets!', 'type': 'all'}, {'items': ['Why <em>WonderWidgets</em> are great', 'Who <em>buys</em> WonderWidgets'], 'title': 'Overview', 'type': 'all'}], 'title': 'Sample Slide Show'}}
===
response.content=b'{\n  "slideshow": {\n    "author": "Yours Truly", \n    "date": "date of publication", \n    "slides": [\n      {\n        "title": "Wake up to WonderWidgets!", \n        "type": "all"\n      }, \n      {\n        "items": [\n          "Why <em>WonderWidgets</em> are great", \n          "Who <em>buys</em> WonderWidgets"\n        ], \n        "title": "Overview", \n        "type": "all"\n      }\n    ], \n    "title": "Sample Slide Show"\n  }\n}\n'
===
respx request.content=b''
---

[StephenBrown2]

Yup, looks like it does! Thanks for the quick work on that.

[lundberg]

I’ve finally found the cause of this issue!

RESPX patches BaseClient._get_response which both sync and async clients use. Problem is that in this stage, the response is still async due to internal dispatch logic in HTTPX.

The PR #23 should solve this, by earlier individual patching in both sync and async clients.

Thanks for raising this issue @StephenBrown2 ! Please try the PR in your use case, before I merge.