Create new aggregation index periodically

Elastic Stack Elasticsearch
1

Hi,

I'm trying to create periodically a new index based on aggregation, something like:

"size": 0,
"aggs": {
"group_by_xpto": {
"terms": {
"field": "xpto.keyword",
"size": 10000
},
"aggs": {
"latest_position": {
"max": {
"field": "PositionX"
}
},
"include_source": {
"top_hits": {
"size": 1,
"_source": {
"includes": [
"Field1","Field2","Field3"
]
}
}
}
}
}
}
}

I tried several approaches, Rollup jobs, Transformations, but none seems to support the type of aggregation I need. The last attempt was using Logstash with a schedule input and output, but it seems that aggregations are not supported, https://github.com/logstash-plugins/logstash-input-elasticsearch/issues/58 .
Can someone suggest a solution or the best approach ?

thanks

2

You can aggregate in Logstash filter. Might be worth checking out to see if it can work for this use case.

3

Thanks for your suggestion, I read the documentation for the filter, it seems a bit confusing, but I'll give it a try. If you have a link with some more working examples, please let me know.

thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.