Script to return array for scripted metric aggregation from combine

Hi ,

For scripted metric aggregation
http://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html
,
in the example shown in the documentation , the combine script returns a
single number.

Instead here , can i pass an array or hash ? I tried doing it , though it
did not return any error , i am not able to access those values from reduce
script. In reduce script per shard i am getting an instance when converted
to string read as 'Script2$_run_closure1@52ef3bd9'

Kindly let me know , if this can be accomplished in any way.
Thanks
Vineeth

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5krMeqn%2Bmh-r2kp0kYBrF9%2Ba%3Dc09CEtSBdVzT8rSvq1AQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

1 Like

Vineeth,

You can return any standard groovy object (by this i mean primitives,
strings, arrays or maps) from the combine script and it will be passed to
the reduce script. Below is a sense recreation script for a more complex
example which counts the number of occurances of each word in a field
(basically a crude version of the terms aggregation). Please note that
these scripts are for test purposes and should not be used in aa production
environment, not least because they are written in Groovy and require
dynamic scripting to be enabled.

DELETE test

POST /test/doc/1
{
"l": 10,
"s": "ten"
}
POST /test/doc/2
{
"l": 4,
"s": "four"
}
POST /test/doc/3
{
"l": 10,
"s": "ten"
}
POST /test/doc/4
{
"l": 7,
"s": "seven"
}
POST /test/doc/5
{
"l": 10,
"s": "ten"
}
POST /test/doc/6
{
"l": 4,
"s": "four"
}
POST /test/doc/7
{
"l": 6,
"s": "six"
}
POST /test/doc/8
{
"l": 6,
"s": "six"
}

Output of combine script on each shard is a map with a key for every word

and values for the number of occurances of that word
GET /test/_search?search_type=count
{
"aggs": {
"scripted_terms": {
"scripted_metric": {
"init_script": "_agg['words'] = ",
"map_script": "word = doc['s']; _agg.words.add(word.value)",
"combine_script": "combined = [:]; for (word in _agg.words) { if
(combined[word]) { combined[word] += 1 } else { combined[word] = 1 } };
return combined"
}
}
}
}

Reduce script uses the map from each shard and adds together the values

for common keys to produce a final map as output
GET /test/_search?search_type=count
{
"aggs": {
"scripted_terms": {
"scripted_metric": {
"init_script": "_agg['words'] = ",
"map_script": "word = doc['s']; _agg.words.add(word.value)",
"combine_script": "combined = [:]; for (word in _agg.words) { if
(combined[word]) { combined[word] += 1 } else { combined[word] = 1 } };
return combined",
"reduce_script": "reduced = [:]; for (a in _aggs) { for (entry in
a) { word = entry.key; if (reduced[word]) { reduced[word] += entry.value }
else { reduced[word] = entry.value } } }; return reduced"
}
}
}
}

Hope this helps,

Colin

On Tuesday, April 21, 2015 at 4:31:21 PM UTC+1, vineeth mohan wrote:

Hi ,

For scripted metric aggregation
http://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html ,
in the example shown in the documentation , the combine script returns a
single number.

Instead here , can i pass an array or hash ? I tried doing it , though it
did not return any error , i am not able to access those values from reduce
script. In reduce script per shard i am getting an instance when converted
to string read as 'Script2$_run_closure1@52ef3bd9'

Kindly let me know , if this can be accomplished in any way.
Thanks
Vineeth

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/57d34e94-203f-4a4f-83ef-2e89f6ab6328%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1 Like

Hello Colin ,

You are the man :).
Seems i have a lot to learn in groovy.

Thanks a ton man , it really helped me.

Thanks
Vineeth

On Tue, Apr 21, 2015 at 9:39 PM, Colin Goodheart-Smithe <
colings86@elastic.co> wrote:

Vineeth,

You can return any standard groovy object (by this i mean primitives,
strings, arrays or maps) from the combine script and it will be passed to
the reduce script. Below is a sense recreation script for a more complex
example which counts the number of occurances of each word in a field
(basically a crude version of the terms aggregation). Please note that
these scripts are for test purposes and should not be used in aa production
environment, not least because they are written in Groovy and require
dynamic scripting to be enabled.

DELETE test

POST /test/doc/1
{
"l": 10,
"s": "ten"
}
POST /test/doc/2
{
"l": 4,
"s": "four"
}
POST /test/doc/3
{
"l": 10,
"s": "ten"
}
POST /test/doc/4
{
"l": 7,
"s": "seven"
}
POST /test/doc/5
{
"l": 10,
"s": "ten"
}
POST /test/doc/6
{
"l": 4,
"s": "four"
}
POST /test/doc/7
{
"l": 6,
"s": "six"
}
POST /test/doc/8
{
"l": 6,
"s": "six"
}

Output of combine script on each shard is a map with a key for every

word and values for the number of occurances of that word
GET /test/_search?search_type=count
{
"aggs": {
"scripted_terms": {
"scripted_metric": {
"init_script": "_agg['words'] = ",
"map_script": "word = doc['s']; _agg.words.add(word.value)",
"combine_script": "combined = [:]; for (word in _agg.words) { if
(combined[word]) { combined[word] += 1 } else { combined[word] = 1 } };
return combined"
}
}
}
}

Reduce script uses the map from each shard and adds together the values

for common keys to produce a final map as output
GET /test/_search?search_type=count
{
"aggs": {
"scripted_terms": {
"scripted_metric": {
"init_script": "_agg['words'] = ",
"map_script": "word = doc['s']; _agg.words.add(word.value)",
"combine_script": "combined = [:]; for (word in _agg.words) { if
(combined[word]) { combined[word] += 1 } else { combined[word] = 1 } };
return combined",
"reduce_script": "reduced = [:]; for (a in _aggs) { for (entry in
a) { word = entry.key; if (reduced[word]) { reduced[word] += entry.value }
else { reduced[word] = entry.value } } }; return reduced"
}
}
}
}

Hope this helps,

Colin

On Tuesday, April 21, 2015 at 4:31:21 PM UTC+1, vineeth mohan wrote:

Hi ,

For scripted metric aggregation
http://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html ,
in the example shown in the documentation , the combine script returns a
single number.

Instead here , can i pass an array or hash ? I tried doing it , though it
did not return any error , i am not able to access those values from reduce
script. In reduce script per shard i am getting an instance when converted
to string read as 'Script2$_run_closure1@52ef3bd9'

Kindly let me know , if this can be accomplished in any way.
Thanks
Vineeth

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/57d34e94-203f-4a4f-83ef-2e89f6ab6328%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/57d34e94-203f-4a4f-83ef-2e89f6ab6328%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5ndSHyhXX0rMnjMs9K3nQWMJ6cZH3_mSx-gsKX0q15D1g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.