aiohttp: replaying doesn't work with two exact requests in replay mode "once"

Hi again maintainers of vcrpy 😃

I found this interesting problem today where in my test, I make 2 HTTP identical requests (one in the test and one in app code — perhaps this is a terrible idea and let me know if it is) and both are recorded, but when trying to replay the test it fails on the second request being made.

After debugging this for a while, it looks like what happens is we update play_counts for both instances of the responses being played for the first request, even though we should only track one being sent back. The code in question is here:

https://github.com/kevin1024/vcrpy/blob/baadf913ef3ed67fa599c3af18d5f9fcb005cf41/vcr/cassette.py#L267-L270

So play_states will be Counter({0: 1, 1: 1}), which feels incorrect to me since requests and responses should be one-to-one. Does that sound reasonable?

When looking up request in self: https://github.com/kevin1024/vcrpy/blob/baadf913ef3ed67fa599c3af18d5f9fcb005cf41/vcr/cassette.py#L260

It fails because of

https://github.com/kevin1024/vcrpy/blob/baadf913ef3ed67fa599c3af18d5f9fcb005cf41/vcr/cassette.py#L355-L360

I think what’s happening is something specific to aiohttp specifically around these lines:

https://github.com/kevin1024/vcrpy/blob/baadf913ef3ed67fa599c3af18d5f9fcb005cf41/vcr/stubs/aiohttp_stubs/__init__.py#L77-L80

Perhaps it shouldn’t be a while? It seems that it’s necessary for this bug that was recently resolved: https://github.com/kevin1024/vcrpy/pull/456.

It’s been a while since I’ve looked at this codebase, so I’m not sure what would be best to remedy this and I haven’t spent time thinking about a solution since I only just now figured out the problem. @lamenezes @arthurHamon2 it seems like y’all are familiar with this — what do you think we should do here? Or am I doing something completely wrong here and this is working as intended? How can I help, if this is a legitimate bug?

Here’s my cassette data:

interactions:
- request:
    body: '{"query": "\n    query {\n        coachAssignments {\n            uuid\n        }\n    }\n    "}'
    headers:
      Content-Type:
      - application/json
      authorization:
      - DUMMY
      x-datadog-parent-id:
      - DUMMY
      x-datadog-trace-id:
      - DUMMY
    method: POST
    uri: http://localhost:8000/graphql
  response:
    body:
      string: '{"data":{"coachAssignments":[{"uuid":"59829c6c-c904-49ea-8157-1d1ac6a2412a"}]}}'
    headers:
      Connection: keep-alive
      Content-Length: '79'
      Content-Type: application/json
      Date: Wed, 28 Aug 2019 23:00:18 GMT
      Server: nginx
      Set-Cookie: csrftoken=redacted;
        expires=Wed, 26 Aug 2020 23:00:18 GMT; Max-Age=31449600; Path=/; SameSite=Lax
      Vary: Cookie, Origin
      X-Frame-Options: SAMEORIGIN
    status:
      code: 200
      message: OK
    url: http://localhost:8000/graphql
- request:
    body: '{"query": "\n    query {\n        coachAssignments {\n            uuid\n        }\n    }\n    "}'
    headers:
      Content-Type:
      - application/json
      authorization:
      - DUMMY
      x-datadog-parent-id:
      - DUMMY
      x-datadog-trace-id:
      - DUMMY
    method: POST
    uri: http://localhost:8000/graphql
  response:
    body:
      string: '{"data":{"coachAssignments":[{"uuid":"59829c6c-c904-49ea-8157-1d1ac6a2412a"}]}}'
    headers:
      Connection: keep-alive
      Content-Length: '79'
      Content-Type: application/json
      Date: Wed, 28 Aug 2019 23:00:18 GMT
      Server: nginx
      Set-Cookie: csrftoken=redacted;
        expires=Wed, 26 Aug 2020 23:00:18 GMT; Max-Age=31449600; Path=/; SameSite=Lax
      Vary: Cookie, Origin
      X-Frame-Options: SAMEORIGIN
    status:
      code: 200
      message: OK
    url: http://localhost:8000/graphql
version: 1

[nickdirienzo]

Hey y’all. I finally found time and energy to resolving this issue via #495.

@cjh79, @vEpiphyte: if you have time and are interested in seeing if my fork resolves your issue that would be awesome. No worries if not! There are test cases that should resemble the behavior we both identified.

[vEpiphyte]

Hi! I ran across this issue today when writing a test which hit the same endpoint twice in a row. The test worked fine when capturing data; but failed upon replay. I’ve got a minimal test for vcr’s aiohttp integration tests that could be used to assist with debugging this behavior.

The test itself runs but the happy-path cases are currently commented out.

def test_double_requests(tmpdir):
    url = "https://httpbin.org/get"

    with vcr.use_cassette(str(tmpdir.join("text.yaml"))):
        _, response_text1 = get(url, output="text")
        _, response_text2 = get(url, output="text")

    with vcr.use_cassette(str(tmpdir.join("text.yaml"))) as cassette:
        resp, cassette_response_text = get(url, output="text")
        assert resp.status == 200
        assert cassette_response_text == response_text1
        # This check fails because we increment the play_count
        # to 2 in the while loop!
        # assert cassette.play_count == 1

        # Now make the second test to url
        resp, cassette_response_text = get(url, output="text")

        # This check fails because we have a 599, not a 200 returned.
        # asset resp.status == 200
        assert resp.status == 599  # <----- sad path check :(

        # This check fails because we get a error message back, not the
        # actual response text
        # assert cassette_response_text == response_text2

        sad_path_message = 'No match for the request <Request (GET) https://httpbin.org/get> was found'
        assert cassette_response_text.startswith(sad_path_message)  # <--- Sad path check :(

        # The final play_count is correct
        assert cassette.play_count == 2