Connection Reset by Peer

Summary

While making an HTTPS GET request to fema.gov I get a Connection Reset Error, this is confirmed on multiple machines not to happen in Python 2 but does happen on Python 3, both are using the same version of requests. Very bizarrely (in this case) the failure occurs when trying to read the 3704 byte. (discovered by using lower level functions). Below is the URL:

https://hazards.fema.gov/gis/nfhl/rest/services/public/NFHL/MapServer/16/query?where=&text=&objectIds=&time=&geometry=-9240000%2C4400000%2C-9230000%2C4410000&geometryType=esriGeometryEnvelope&inSR=102100&spatialRel=esriSpatialRelIntersects&relationParam=&outFields=&returnGeometry=true&maxAllowableOffset=&geometryPrecision=&outSR=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&returnDistinctValues=false&returnTrueCurves=false&resultOffset=&resultRecordCount=&f=pjson

Things I have tried:

• User-Agent strings
• Disabling verify
• Using machines with different IP addresses
• Matching all libraries except openssl_version to the working Python 2.7 environment

Reproduction Steps

python
import requests
url ="https://hazards.fema.gov/gis/nfhl/rest/services/public/NFHL/MapServer/16/query?where=&text=&objectIds=&time=&geometry=-9240000%2C4400000%2C-9230000%2C4410000&geometryType=esriGeometryEnvelope&inSR=102100&spatialRel=esriSpatialRelIntersects&relationParam=&outFields=&returnGeometry=true&maxAllowableOffset=&geometryPrecision=&outSR=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&returnDistinctValues=false&returnTrueCurves=false&resultOffset=&resultRecordCount=&f=pjson"
response=requests.get(url)

System Information

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.2.2"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010008f",
    "version": "18.0.0"
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}

As noted above the request works fine in Python 2.7

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.14"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1000207f",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}

I have tried to get the versions of my Python 3 to match but even doing this the request still fails in Python 3.

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}

[etothepii]

Hi Ian,

Thank you for you help, it is very much appreciated.

As discussed above I have satisfied the immediate problem by running curl in a sub process but that seems like a horrible solution.

I really appreciate your help and I agree with you that it works on some other setups. As evidenced by it working on Python 3. To me this suggests that it is most likely not the server that is at fault (as would be the normal response to a Connection Reset by Peer). Doesn’t it seem so very odd to you that I am able to make the request and get a 200 response using lower level functions but get the connection reset if trying to read past char 3704, despite the message clearly being 15000 chars long?

Please accept my apologies for the confusion above - with diagnostics. I was trying to show that I had already tried to align my non-working Python 3 system with a working Python 2 system, by showing what I had changed it to. As far as I can see I have managed to sync everything up between the working between them except for the system SSL version which is 1000207f (working) rather than 1010006f (non-working). Everything else is the same according to my version of Beyond Compare (image attached).

Any help or suggestions for further avenues of inquiry would be very welcome.

Kind regards,

James

[nateprewitt]

Thanks for checking back in @etothepii. Since it doesn’t look like there is more information to work with, and this seems to be a local configuration issue, I’m going to close this out.

Please feel free to reopen if you have more information later. Thanks!