Time out on large layers

This parcel layer has 1.846 million features.

esri2geojson http://gis2.cookcountyil.gov/arcgis/rest/services/cookVwrDynmcCondo/MapServer/44 condos.geojson
cli.esridump - INFO - Built 1846 requests using OID where clause method
Traceback (most recent call last):
  File "/usr/local/bin/esri2geojson", line 9, in <module>
    load_entry_point('esridump==1.1.1', 'console_scripts', 'esri2geojson')()
  File "/Library/Python/2.7/site-packages/esridump/cli.py", line 88, in main
    feature = feature_iter.next()
  File "/Library/Python/2.7/site-packages/esridump/dumper.py", line 360, in __iter__
    raise EsriDownloadError("Could not connect to URL", e)
esridump.errors.EsriDownloadError: ('Could not connect to URL', ReadTimeout(ReadTimeoutError("HTTPConnectionPool(host='gis2.cookcountyil.gov', port=80): Read timed out. (read timeout=30)",),))

It cuts out at 226 MB after 1 minute. I’m guessing the final version will be nearly 1 GB.

[iandees]

This is more of a problem that the server is getting overwhelmed than something wrong with the code. We could maybe add a retry and backoff timer so we don’t hit the server quite as quickly as we do now.

[andrewharvey]

@stevevance If the layer has an ID field, then I’ve found forcing esri2geojson to query by ID range avoids timeouts, you can now do this with --paginate-oid, from https://github.com/openaddresses/pyesridump/commit/a4c68db6025264e323bb73b9d901be6050ae6f7f. Could you try that and see if it makes a difference?