Regression of ASGI mocking after `respx == 0.17.1` and `httpx == 0.19.0`

We run code alike this:

import httpx
import pytest
import respx
from respx.mocks import HTTPCoreMocker


@pytest.mark.asyncio
async def test_asgi():
    try:
        HTTPCoreMocker.add_targets(
            "httpx._transports.asgi.ASGITransport",
            "httpx._transports.wsgi.WSGITransport",
        )
        async with respx.mock:
            async with httpx.AsyncClient(app="fake-asgi") as client:
                url = "https://foo.bar/"
                jzon = {"status": "ok"}
                headers = {"X-Foo": "bar"}
                request = respx.get(url) % dict(
                    status_code=202, headers=headers, json=jzon
                )
                response = await client.get(url)
                assert request.called is True
                assert response.status_code == 202
                assert response.headers == httpx.Headers(
                    {
                        "Content-Type": "application/json",
                        "Content-Length": "16",
                        **headers,
                    }
                )
                assert response.json() == {"status": "ok"}
    finally:
        HTTPCoreMocker.remove_targets(
            "httpx._transports.asgi.ASGITransport",
            "httpx._transports.wsgi.WSGITransport",
        )

It works perfectly fine using respx == 0.17.1 and httpx == 0.19.0, as can be seen by:

$> pytest test.py
=================== test session starts ===================
platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0
plugins: asyncio-0.19.0, respx-0.17.1, anyio-3.6.1
asyncio: mode=strict
collected 1 item                                          

test.py .                                           [100%]

==================== 1 passed in 0.00s ====================

However upgrading httpx == 0.20.0 yields:

.../lib/python3.10/site-packages/respx/mocks.py:179: in amock
    request = cls.to_httpx_request(**kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'respx.mocks.HTTPCoreMocker'>, kwargs = {'request': <Request('GET', 'https://foo.bar/')>}

    @classmethod
    def to_httpx_request(cls, **kwargs):
        """
        Create a `HTTPX` request from transport request args.
        """
        request = (
>           kwargs["method"],
            kwargs["url"],
            kwargs.get("headers"),
            kwargs.get("stream"),
        )
E       KeyError: 'method'

.../lib/python3.10/site-packages/respx/mocks.py:288: KeyError

While trying to upgrade respx == 0.18.0 yields a package resolution error:

  SolverProblemError

  Because respx (0.18.0) depends on httpx (>=0.20.0)
   and respxbug depends on httpx (^0.19.0), respx is forbidden.
  So, because respxbug depends on respx (0.18.0), version solving failed.

Upgrading both yields an error alike the one for just upgrading respx.

Running with respx == 0.19.2 and httpx == 0.23.0 (the newest version at the time for writing), yields:

.../lib/python3.10/site-packages/respx/mocks.py:186: in amock
    request = cls.to_httpx_request(**kwargs))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'respx.mocks.HTTPCoreMocker'>, kwargs = {'request': <Request('GET', 'https://foo.bar/')>}, request = <Request('GET', 'https://foo.bar/')>

    @classmethod
    def to_httpx_request(cls, **kwargs):
        """
        Create a `HTTPX` request from transport request arg.
        """
        request = kwargs["request"]
        raw_url = (
            request.url.scheme,
            request.url.host,
            request.url.port,
>           request.url.target,
        )
E       AttributeError: 'URL' object has no attribute 'target'

.../lib/python3.10/site-packages/respx/mocks.py:302: AttributeError

I have attempted to debug the issue, and it seems there’s a difference in the incoming request.url object type during ASGI and non-ASGI mocking (see the example below).

With ASGI mocking:

(Pdb) pp request.url
URL('https://foo.bar/')
(Pdb) pp type(request.url)
<class 'httpx.URL'>

Without ASGI mocking (i.e. ordinary mocking):

(Pdb) pp request.url
URL(scheme=b'https', host=b'foo.bar', port=None, target=b'/')
(Pdb) type(request.url)
<class 'httpcore.URL'>

The non-ASGI mocking case was produced with the exact same code as above, but by changing:

            async with httpx.AsyncClient(app="fake-asgi") as client:

to:

            async with httpx.AsyncClient() as client:

I have not studied the flow of respx mock well enough to know how to implement an appropriate fix, but the problem seems to arise from the fact that .target is not a documented member on httpxs URL object, however it does have similar code to the above, here, which utilizes .raw_path instead of .target.

So maybe the code ought to branch on the incoming URL type and provide url.raw if the type is a httpx.URL, unless it can be passed on directly?

Finally it seems like the test validating that ASGI mocking works was removed in this commit: 47c0b935176e081a3aa7886aed8b8ed31c0e9457, while the core functionality was added in this PR: #131 (along with said test).

As the code stands now it only seems to test that the "httpx._transports.asgi.ASGITransport" can be added and removed using HTTPCoreMocker.add_targets and HTTPCoreMocker.remove_targets not that mocking with the ASGITransport actually works.

[lundberg]

Thanks @Skeen for the issue and research!

It’s true that the ASGI mocking lacks logic testing, will have to look at this next week 👍

[lundberg]

@Skeen, I’ll close this issue. Please re-open if my answer didn’t solve your problem.