I would like to ensure similar size of indices and at the same time I need ensure different retention policy for some indices. Could you please give me some tip how to do that?
SIMILAR SIZE OF INDICES
I would like to have indices with similar size (say 20 gb). For this purpose I am using
rollover API in Curator. It works well - thank for explanation in Elastic Forum
DIFFERENT RETENTION POLICY
Some indices I can store only for 7 days and some for 2 months.
Is it possible to use
field_stats for difference retention policy of indices? I did not find some options how to do this using Curator.
If we have indices which are rolloved without option "max_age",
indicies can be created like this:
index-aaa-2018.01.01-00001 (25 gb) index-aaa-2018.01.01-00002 (25 gb) index-aaa-2018.01.02-00001 (25 gb) index-aaa-2018.01.20-00001 (25 gb) index-aaa-2018.01.20-00002 (25 gb)
All logs between dates 2018.01.02 - 2018.01.20 will be stored to index
index-aaa-2018.01.02-00001, because conditions of size or number of document were met on day 2018.01.20.
Now If I use curator for deleting indices by name (index prefix) and say we can delete indices index-aaa older than 3 days and today is 2018.01.20, it means index
index-aaa-2018.01.17 and older will be deleted.
It means that index
index-aaa-2018.01.02-00001 will be deleted too but it contains data which we can preserve (data from days 2018.01.18, 2018.01.19).
What is the best way If we can have indices with similar size with preservation different retention policy?