How to generate multiple time granular indexes?

I want to write monitoring data in ES. At present, there are 4 indicators, each indicator is 5 minutes granularity, every 5 minutes has more than 10,000 data, about 3 million data a day.

The requirements are:

  1. How should ES be designed based on time index? Can I index at a granularity of 1 day, 7 days, 30 days?
  2. The data granularity is 5 minutes, but when querying, I want to return according to different granularities. For example, querying for 1 day is 5 minutes granularity, querying for 7 days is hourly granularity, and querying for 30 days is granularity of days.

Thanks god.


regarding different granularities. If you want to reduce your data, you can use rollup. However if space is not an issue, you can also query your 5 minute granularity data and query the last 30 days in a per-day granularity.

regarding time based indices: Take a look at index lifecycle management, especially to rollover, which allows you to rollover after time, size or number of documents.

hope this helps as a start.

OK,thanks a lot, but a problem is my elasticsearch version is 5.6.3, so rollup can't use, can I use other method in this version?

yes, you need to upgrade in order to use any of those. you may want to take a look at curator that also works with older versions as it is an external process.