Return aggregation and subaggregation values only.

I’d really like to be able to perform a arbitrarily nested aggregation with mixed agg types and be able to return only the values associated with each named aggregation (and any nested sub-aggregation). I know the returned model mirrors elasticseach defaults, but the default model is deeply hostile to [simply] parsing anything beyond a first or second order aggregation.

Example:

match_all = s.query()
match_all.aggs.bucket("example1", "terms", field="field")
match_all.aggs.bucket("example2", "avg", field="field2")
match_all.aggs["example1"].bucket("example3", "cardinality", field="field3")

This is fantastic, I have a simple query dsl where I can easily pull aggregate stats on my data (I use elasticsearch for text analysis in a data science context).

The problem arises when I get the result.

>> results.aggregations.to_dict()
{u'example1': {u'buckets': [{u'doc_count': 53586,
    u'example3': {u'value': 1},
    u'key': u'example 1 agg value'},
   {u'doc_count': 18278, u'example3': {u'value': 1}, u'key': u'1'},
   {u'doc_count': 17309, u'example3': {u'value': 1}, u'key': u'2'},
   {u'doc_count': 10230, u'example3': {u'value': 1}, u'key': u'3'},
   {u'doc_count': 9328, u'example3': {u'value': 1}, u'key': u'4'},
   {u'doc_count': 9090, u'example3': {u'value': 1}, u'key': u'5'},
   {u'doc_count': 7528, u'example3': {u'value': 1}, u'key': u'6'},
   {u'doc_count': 6409, u'example3': {u'value': 1}, u'key': u'7'},
   {u'doc_count': 5757, u'example3': {u'value': 1}, u'key': u'8'},
   {u'doc_count': 4896, u'example3': {u'value': 1}, u'key': u'9'}],
  u'doc_count_error_upper_bound': 0,
  u'sum_other_doc_count': 38543},
 u'example2': {u'value': 45}}

What I really want to be able to get using an option of some sort is this.

{
example1: [{example3: 1, ...} ...],
example2: 45
}

Thoughts? Exists? Possible? SOL (i.e. write a parser)?

[honzakral]

I improved the aggregation response in the aggs branch where each response is wrapped in an object similar to hits. This also allowed us to deserialize the bucket keys (so key for a date histogram bucket is an actual datetime object now) and should allow for easier extendability - simply create your own class with your methods etc.

It also contains other improvements like being able to iterate over the results, yielding you buckets.

please let me know what you think! Thank you!

[OAuthBringer]

This is great and timely. I fell off of doing elasticsearch things for a while, but have orbited back to it. Arbitrary nested aggregations are again a business need. Thanks for the update.