it seems that the es1.0 aggregations framework could straightforwardly
support this type of aggregation operations as e.g. a reduce of the
documents in a bucket
are there any such reduce-style aggregation operations on the roadmap ?
There seems to be some reluctance by ES team to provide scrip table aggregators, or perhaps it's on a roadmap and just taking a long time. Kimchi has stated that he would like to identify these use cases and roll them into built-in aggregations so everyone can benefit. I think the range of these use cases is too broad for a specific set of implementations. The aggregations types included are fine for simple stats and bucketing, but there are probably hundreds of scenarios that require custom aggregation.
I'll probably look into creating a custom aggregator for my use case.
Note: a lot of the custom aggregators mentioned in this forum are general implementations of the reduce clause in a map reduce statement.
My use case is the implementation of an Item Response Theory algorithm on a filtered result set. The idea is that a list of student responses to a question can be incrementally processed to result in a proficiency value. The filter (map) results are restricted to a timeframe and sorted, and the aggregation (reduce) step will incrementally inject each students score into an algorithm that progressively converges on the student's ability magnitude to answer those questions.
Thanks,
Matt Weber
On Aug 9, 2014 10:44 AM, "James Cook" djamescook@gmail.com wrote:
There seems to be some reluctance by ES team to provide scrip table
aggregators, or perhaps it's on a roadmap and just taking a long time.
Kimchi has stated that he would like to identify these use cases and roll
them into built-in aggregations so everyone can benefit. I think the range
of these use cases is too broad for a specific set of implementations. The
aggregations types included are fine for simple stats and bucketing, but
there are probably hundreds of scenarios that require custom aggregation.
I'll probably look into creating a custom aggregator for my use case.
Note: a lot of the custom aggregators mentioned in this forum are general
implementations of the reduce clause in a map reduce statement.
My use case is the implementation of an Item Response Theory algorithm on
a filtered result set. The idea is that a list of student responses to a
question can be incrementally processed to result in a proficiency value.
The filter (map) results are restricted to a timeframe and sorted, and the
aggregation (reduce) step will incrementally inject each students score
into an algorithm that progressively converges on the student's ability
magnitude to answer those questions.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.