# Reduce-style aggregators

**URL:** https://discuss.elastic.co/t/reduce-style-aggregators/15973
**Category:** Elasticsearch
**Created:** [February 22, 2014, 11:42pm UTC](https://discuss.elastic.co/t/reduce-style-aggregators/15973 "2014-02-22T23:42:37Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![mccraig\_mccraig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mccraig_mccraig/32/1780_2.png) [@mccraig\_mccraig](https://discuss.elastic.co/u/mccraig_mccraig)
#### Post date: [February 22, 2014, 11:42pm UTC](https://discuss.elastic.co/t/reduce-style-aggregators/15973/1 "2014-02-22T23:42:37Z")

</div>

i keep finding that i want to do more generic metric aggregations than are  
currently available

e.g. to do a like behaviour so given docs:

{name: "foo", location: "london", revenue: 100}  
{name: "foo", location: "paris", revenue: 500}  
{name: "bar", location: "sydney", revenue: 15}  
{name: "bar", location: "new york", revenue: 23}

i can get a result like this (collect the locations for a name, and sum the  
revenues) :

{name: "foo", location: ["london","paris"], revenue: 600}  
{name: "bar", location: ["sydney", "new york"], revenue: 38}

i.e. the kinds of things you can do with sql group-by, postgresql window  
functions or cascading/cascalog aggregators  
[[https://github.com/nathanmarz/cascalog/wiki/Guide-to-custom-operations#wiki-aggregators](https://github.com/nathanmarz/cascalog/wiki/Guide-to-custom-operations#wiki-aggregators)  
]

it seems that the es1.0 aggregations framework could straightforwardly  
support this type of aggregation operations as e.g. a reduce of the  
documents in a bucket

are there any such reduce-style aggregation operations on the roadmap ?

:c

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9f361d38-b9de-4a7f-bb4e-244b3be6743b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9f361d38-b9de-4a7f-bb4e-244b3be6743b%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![James\_Cook\_2](https://avatars.discourse-cdn.com/v4/letter/j/b19c9b/32.png) [@James\_Cook\_2](https://discuss.elastic.co/u/James_Cook_2)
#### Post date: [August 9, 2014, 5:44pm UTC](https://discuss.elastic.co/t/reduce-style-aggregators/15973/2 "2014-08-09T17:44:48Z")

</div>

There seems to be some reluctance by ES team to provide scrip table aggregators, or perhaps it's on a roadmap and just taking a long time. Kimchi has stated that he would like to identify these use cases and roll them into built-in aggregations so everyone can benefit. I think the range of these use cases is too broad for a specific set of implementations. The aggregations types included are fine for simple stats and bucketing, but there are probably hundreds of scenarios that require custom aggregation.

I'll probably look into creating a custom aggregator for my use case.

Note: a lot of the custom aggregators mentioned in this forum are general implementations of the reduce clause in a map reduce statement.

My use case is the implementation of an Item Response Theory algorithm on a filtered result set. The idea is that a list of student responses to a question can be incrementally processed to result in a proficiency value. The filter (map) results are restricted to a timeframe and sorted, and the aggregation (reduce) step will incrementally inject each students score into an algorithm that progressively converges on the student's ability magnitude to answer those questions.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4b8725e6-c73b-4da7-abeb-330c1e0d2406%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4b8725e6-c73b-4da7-abeb-330c1e0d2406%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)
#### Post date: [August 9, 2014, 6:55pm UTC](https://discuss.elastic.co/t/reduce-style-aggregators/15973/3 "2014-08-09T18:55:52Z")

</div>

You will be able to do this soon. See:

> <https://github.com/elastic/elasticsearch/pull/7075>
>
> A metrics aggregation which runs specified scripts at the init, collect, combine…, and reduce phases
> 
> Closes #5923

Thanks,  
Matt Weber  
On Aug 9, 2014 10:44 AM, "James Cook" [djamescook@gmail.com](mailto:djamescook@gmail.com) wrote:

> There seems to be some reluctance by ES team to provide scrip table  
> aggregators, or perhaps it's on a roadmap and just taking a long time.  
> Kimchi has stated that he would like to identify these use cases and roll  
> them into built-in aggregations so everyone can benefit. I think the range  
> of these use cases is too broad for a specific set of implementations. The  
> aggregations types included are fine for simple stats and bucketing, but  
> there are probably hundreds of scenarios that require custom aggregation.
> 
> I'll probably look into creating a custom aggregator for my use case.
> 
> Note: a lot of the custom aggregators mentioned in this forum are general  
> implementations of the reduce clause in a map reduce statement.
> 
> My use case is the implementation of an Item Response Theory algorithm on  
> a filtered result set. The idea is that a list of student responses to a  
> question can be incrementally processed to result in a proficiency value.  
> The filter (map) results are restricted to a timeframe and sorted, and the  
> aggregation (reduce) step will incrementally inject each students score  
> into an algorithm that progressively converges on the student's ability  
> magnitude to answer those questions.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/4b8725e6-c73b-4da7-abeb-330c1e0d2406%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4b8725e6-c73b-4da7-abeb-330c1e0d2406%40googlegroups.com)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoAwv2m1-hLHGWnY6wF%3DDEQwDBqau\_KN%2BMQTcgs%2BZooE0A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoAwv2m1-hLHGWnY6wF%3DDEQwDBqau_KN%2BMQTcgs%2BZooE0A%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:09am UTC](https://discuss.elastic.co/t/reduce-style-aggregators/15973/4 "2017-07-06T01:09:47Z")

</div>


