question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow for `scan` with aggregations

See original GitHub issue

In 5.0 elasticsearch allows a search request with aggregations when using scan/scroll which we should expose.

This has been moved here from elasticsearch-py - https://github.com/elastic/elasticsearch-py/issues/530

Issue Analytics

  • State:open
  • Created 7 years ago
  • Reactions:1
  • Comments:11 (3 by maintainers)

github_iconTop GitHub Comments

11reactions
macdjordcommented, Feb 7, 2017

To be clear - this is not about using a scrolled query to fetch the aggregations bit by bit. This is about having a query with both aggregations and hits, and you want to use a scrolled query for the hits while still seeing the aggregations.

Currently, there’s no way to do this: Search.execute() doesn’t do scrolled queries, while Search.scan() only return an iterator over the hits with no way to access the aggregation results.

My proposal is to add a new method, Search.execute_scan(), which returns a Response object like Search.execute(), but the Response.hits property, instead of being a static list, is a Search.scan()-style iterator.

1reaction
honzakralcommented, Jul 24, 2019

Just to clarify this is not scanning through results of aggregation, just returning an aggregates first and then scanning through the documents.

To “scan” over aggregations you can use the composite aggregation as shown here - https://github.com/elastic/elasticsearch-dsl-py/blob/master/examples/composite_agg.py

Read more comments on GitHub >

github_iconTop Results From Across the Web

Secondary aggregation not allowed in scans - Hahn-Tech, LLC
You have found that secondary aggregation periods are not supported for custom scans. The only way to work around this limitation is to...
Read more >
Intro to Rx - Aggregation - Introduction to Rx
Scan. While Aggregate allows us to get a final value for sequences that will complete, sometimes this is not what we need. If...
Read more >
Scan through result in ES-DSL aggregation - Elastic Discuss
I wanted to see how many unique link that a user has posted for every user and save something likeuser_id, number_of_post to a...
Read more >
Scanning and aggregation of serialization codes | SICK
The Multi Code Reader quality control system scans and aggregates a high number of serialization codes: The system automatically records codes from up...
Read more >
Defining an aggregate reallocation scan - NetApp
If reallocation has been enabled on your storage system, you can initiate an aggregate reallocation scan to optimize the location of physical blocks...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found