Aggregate data after month to get smaller index

Elastic Stack Beats
1

hi all,
I would like to have a method to aggregate informations collected using Metricbeat.
Currently I have the system module enabled that collects details every 30 seconds. this situation is really good if I want to monitor information, for instance, of the current day or yesterday.

If I would like to store informations older than 6 months, what should be the best approach? I mean, if I would like to see the CPU trend of the last year, I think that my granularity should not be so deeper (30 seconds) but maybe hourly.
obviously, using Kibana and choosing the time range of 6 months it will adapts the dashboard but in my index there still will be a lot of documents (catched every 30 seconds). so, is there any operation that could I perform to make the index more tiny?

2

It sounds like you’re looking for Elasticsearch’s Rollups feature.

https://www.elastic.co/guide/en/elasticsearch/reference/current/xpack-rollup.html

1 Like
3

really thank you for the answer that addressed me in the correct direction.
I have just one doubt: how can I roll up every field that exists in my original index? Should I specify all of them in my rollup job?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.