question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

arcgis.gis.ContentManager.advanced_search returns duplicate results for high values of max_items

See original GitHub issue

Describe the bug Content searches using arcgis.gis.ContentManager.advanced_search return additional, duplicate results in batches of 100 if max_items is set greater than 10,000 and there are fewer results than the max_items value. At max_items=10001 the result count will be incremented by 100. At max_items=10101 the result count is increased 200.

To Reproduce Steps to reproduce the behavior:

# you may need to set to an orgid you have access to
from arcgis import GIS

gis = GIS(profile='umntc_agol_admin')
print(f"Connected to {gis.properties.urlKey}.{gis.properties.customBaseUrl}",
      f"as {gis.properties.user.username} (role: {gis.properties.user.role})")

orgId = "8df8p0NlLFEShl0r"
maxItems = [300, 400, 9999, 10000, 10001, 10050,
            10100, 10101, 100000, 100001]

print("max items,result count")
for mi in maxItems:
    response = gis.content.advanced_search(
        query = f"orgid:{orgId} and type:'Feature Service'",
        max_items = mi
    )
    print(f"{mi},{len(response['results'])}")

error:

# no error presented; output follows
Connected to umntc.maps.arcgis.com as wiringa_nosso2 (role: org_admin)
max items,result count
300,300
400,373
9999,373
10000,373
10001,473
10050,473
10100,473
10101,573
100000,90373
100001,90473

Expected behavior Don’t duplicate search results.

Platform (please complete the following information):

  • OS: Windows 10 64-bit
  • Browser n/a
  • Python API Version 1.6.2
  • Python version 3.6.9
  • AGOL Version 7.3 (also seen prior to the 2019-10-01 AGOL update)

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
knoopumcommented, Nov 5, 2021

For anyone else interested, you can ask Esri Support to attach you to: #ENH-000144313 [Enhancement] When attempting to use the ArcGIS REST API (or the ArcGIS API for Python on top of it) in a situation where more than 10,000 items exist, the API can only access the first 10,000 items.

2reactions
achapkowskicommented, Jan 21, 2020

No, it is being looked at. It turns out not to be a Python API issue, rather a REST API issue. The max result for any query has been set to a max of 10,000 records regardless of the number of records in the query. This means there is a hard stop at 10,000. We will have logic inplace to prevent going over 10,000, and I will pass along this issue to the REST API folks.

Read more comments on GitHub >

github_iconTop Results From Across the Web

arcgis.gis module | ArcGIS API for Python
The GIS provides a mapping widget that can be used in the Jupyter Notebook environment for visualizing GIS content as well as the...
Read more >
gis.ContentManager.search will not do exact match.
Solved: Running this code in Jupyter notebook gives this output. from arcgis.gis import GIS from config import Config portal = GIS(Config.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found