Reindex helper does not respect size limit

Not sure if this is possible, but I haven’t found a way to apply a maximum number of documents to reindex using the reindex helper. Everything I’ve tried just reindexes everything.

Using a size limit with search works as expected, but reindex doesn’t seem to respect this. I can use curl with the same query for reindexing and it works as expected. Is there something I’m missing by any chance?

Currently using with the 2.4.1 release.

[honzakral]

yes, the limit is being ignored because reindex uses scan search to get all the documents, you’d have to craft a query that only matches N documents in your index.

But if you look into the implementation of reindex you see how easy it is, it is literally feeding search results into bulk, which you can do on your own with the limited search API call:

data = es_production.search(index='i', body={"size": 1000, "query": {...}})
bulk(es_sandbox, data['hits']['hits'])

[Battleroid]

Ah ok. Thank you.

On April 17, 2017 6:22:57 PM EDT, “Honza Král” [e-mail] wrote:

yes, the limit is being ignored because reindex uses scan search to get all the documents, you’d have to craft a query that only matches N documents in your index.

But if you look into the implementation of reindex you see how easy it is, it is literally feeding search results into bulk, which you can do on your own with the limited search API call:

data = es_production.search(index='i', body={"size": 1000, "query":
{...}})
bulk(es_sandbox, data['hits']['hits'])

– Casey Weed