I am indexing my data using ILM Policy and Indexing Template. So, the index moves from one Phase to another(Hot, Warm , Cold and Delete) after the completion of assigned time period for that Phase.
Now, I got a requirement where in I need to index older data.
Can I directly create index into warm phase with the older data? will these index go though the policy and move to cold and delete phase on completion of the assigned time period.
Can I have index-template to have name of index starting from index-000009 so that I can create index with older data into the ward phase naming as index-000008 and so forth?
Welcome to our community!
If you index the older data into the ILM policy then Elasticsearch assumes the data that you are ingesting is current, and so it will treat the index the same way it does for others in the policy.
The best option would be to index the data into individual indices, then attach those to the read alias. You will need to manage these yourself however.
Thanks a lot for answering !!
For the older data , I can create indices on my own and make them read only.
But then the transmission from WARM phase to COLD Phase and then to the DELETE Phase needs to be taken care by myself . So basically I will not be able to take the advantage of ILM for older data.
The Older data is huge, its about 90% of the current data. So, that would mean that I would have to manage 90% data myself and ILM manages 10 % data.
Is there some other approach to this ?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.