Correct way to retry response on 503 in script?

Env: mitmdump 0.18.2 (trying to migrate a script from 0.16)

Hello!

I’m trying to understand how I might use replay_request or something else to add proxy-level retries on certain errors, such as when a flaky backend throws a 503 error.

I’ve tried this:

$ mitmdump -p 8080 --no-http2 -s myscript.py

myscript.py:

from mitmproxy import ctx

def response(flow):
    # Retry once if the backend responded with 503:
    if flow.response.status_code == 503:
        f = ctx.master.duplicate_flow(flow)
        ctx.master.replay_request(f, block=True)

…but this seems to deadlock, I think because I don’t understand the way the main thread processes flows (or how exactly to replace the current flow.response).

Can you point me in the right direction here?

[fotinakis]

Here’s what ended up working, feel free to let me know if something looks off here.

Notably: it seems that the response hook fires for every replayed request (which is the opposite of what I read you talk about on the forums), so I had to add a _num_retries tracker to prevent infinite recursion.

I also had to do flow.response = new_flow.response, which I’m not sure if that’s correct way to override the top-level response.

@concurrent
def response(flow):
  if flow.response.status_code == 503:
    retry_number = getattr(flow, '_num_retries', 0)

    if retry_number < MAX_REQUEST_RETRIES:
      new_flow = master.duplicate_flow(flow)
      setattr(new_flow, '_num_retries', retry_number + 1)
      master.replay_request(new_flow, block=True)
      flow.response = new_flow.response
    else:
      syslog.syslog(syslog.LOG_ERR, 'Permanent 503 from: %s' % flow.request.url)

Thanks for the help!

[maximusfox]

Here’s what ended up working, feel free to let me know if something looks off here.

Notably: it seems that the response hook fires for every replayed request (which is the opposite of what I read you talk about on the forums), so I had to add a _num_retries tracker to prevent infinite recursion.

I also had to do flow.response = new_flow.response, which I’m not sure if that’s correct way to override the top-level response.

@concurrent
def response(flow):
  if flow.response.status_code == 503:
    retry_number = getattr(flow, '_num_retries', 0)

    if retry_number < MAX_REQUEST_RETRIES:
      new_flow = master.duplicate_flow(flow)
      setattr(new_flow, '_num_retries', retry_number + 1)
      master.replay_request(new_flow, block=True)
      flow.response = new_flow.response
    else:
      syslog.syslog(syslog.LOG_ERR, 'Permanent 503 from: %s' % flow.request.url)

Thanks for the help!

How to make script like this on 5.x version?