Hello
I would like to ensure similar size of indices and at the same time I need ensure different retention policy for some indices. Could you please give me some tip how to do that?
SIMILAR SIZE OF INDICES
I would like to have indices with similar size (say 20 gb). For this purpose I am using rollover API in Curator. It works well - thank for explanation in Elastic Forum
DIFFERENT RETENTION POLICY
Some indices I can store only for 7 days and some for 2 months.
FIELD_STATS
Is it possible to use field_stats for difference retention policy of indices? I did not find some options how to do this using Curator.
ONE EXAMPLE
If we have indices which are rolloved without option "max_age",
indicies can be created like this:
index-aaa-2018.01.01-00001 (25 gb)
index-aaa-2018.01.01-00002 (25 gb)
index-aaa-2018.01.02-00001 (25 gb)
index-aaa-2018.01.20-00001 (25 gb)
index-aaa-2018.01.20-00002 (25 gb)
All logs between dates 2018.01.02 - 2018.01.20 will be stored to index index-aaa-2018.01.02-00001, because conditions of size or number of document were met on day 2018.01.20.
Now If I use curator for deleting indices by name (index prefix) and say we can delete indices index-aaa older than 3 days and today is 2018.01.20, it means index index-aaa-2018.01.17 and older will be deleted.
It means that index index-aaa-2018.01.02-00001 will be deleted too but it contains data which we can preserve (data from days 2018.01.18, 2018.01.19).
What is the best way If we can have indices with similar size with preservation different retention policy?
Regads,
Vašek