Query about lifecycle management

Elastic Stack Elasticsearch
1

I have a years worth of daily indexes that I am cleaning up by reindexing. Reading the "FM" I find that there appears to be a much better way to manage indexes other then blindly creating a new index each day (I was wondering why I was running out of shards!). So I am looking to adding lifecyle options to the index template.

My first instinct is to get the new data going into the revamped index stream and then reindex the old data into the new stream while cleaning it up and getting it all consistent, but I have doubts...

My question is "How does ES work out the age of an index for moving it between hot and warm/cold"?

Does it work on index create time or does it use the time field in the index?

I am adding some cold nodes to the cluster and I want all my "old" indexes stored there but if these indexes are newly created via a reindex will this work?

suggestions as how best to proceed solicited :slight_smile:

Russell

2

This one.

You'd be better of moving new data to ILM, then just removing old data as it ages out.

3

Thanks Mark
My question is how does ES determine the age of the index -- by its contents (which your answer implies) or by the creation time of the index?

My initial instinct was to do what you suggest but then I wondered if ES would think the indexes with old data were new because they were created yesterday

4

I believe Curator has an option to base age on the data the index holds, but this is as far as I know not possible with ILM. Age is therefore as far as I recall based on index creation date.

5

I am not sure how you read that given what I quoted and how I responded? But as it seems a little unclear - it uses the index creation time.