How to generate multiple time granular indexes?

[ziyang]

I want to write monitoring data in ES. At present, there are 4 indicators, each indicator is 5 minutes granularity, every 5 minutes has more than 10,000 data, about 3 million data a day.

The requirements are:

1. How should ES be designed based on time index? Can I index at a granularity of 1 day, 7 days, 30 days?
2. The data granularity is 5 minutes, but when querying, I want to return according to different granularities. For example, querying for 1 day is 5 minutes granularity, querying for 7 days is hourly granularity, and querying for 30 days is granularity of days.

Thanks god.

[spinscale (Alexander Reelsen)]

Hey,

regarding different granularities. If you want to reduce your data, you can use rollup. However if space is not an issue, you can also query your 5 minute granularity data and query the last 30 days in a per-day granularity.

regarding time based indices: Take a look at index lifecycle management, especially to rollover, which allows you to rollover after time, size or number of documents.

hope this helps as a start.

[ziyang]

OK，thanks a lot, but a problem is my elasticsearch version is 5.6.3, so rollup can't use, can I use other method in this version?

[spinscale (Alexander Reelsen)]

yes, you need to upgrade in order to use any of those. you may want to take a look at curator that also works with older versions as it is an external process.