Aggregating using ingest Node

Elastic Stack Elasticsearch
1

Hi all,

I was curious if it is possible to perform a "count/terms" aggregation in an ingest node/pipeline using a script processor, or if there is any plan to add in support for a processor like that in the future?

To clarify, we have a index that we send raw data to for later querying. However, we have been reading more into ingest nodes, and are curious if anyone has been able to perform the aggregations in an ingest node with a script processor that would allow for the count aggregations to be performed before the data is actually indexed.

Here is the query we used to collect and count term counts:

GET index-name/_search?size=0
{
"aggs": {
"QueryName": {
"terms": {
"script": "doc['FieldName'].values"
}
}
}
}

Thank you.

2

Like doing lookups in elasticsearch?

No it won't happen. See:

There is an elasticsearch logstash plugin but sadly it does not support adding aggs: https://www.elastic.co/guide/en/logstash/5.4/plugins-filters-elasticsearch.html

1 Like
3

David,

Thanks for the quick reply, and for the github link as well - appreciate the help!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.