ILM Stages ES7

[cbille0]

Good evening (or morning, or afternoon),

I wanted to know if I could get some clarification on the four phases of the Index Management Lifecycle: Hot, Warm, Cold, and Delete. The documentation for warm phase says the index is no longer being updated, but still being queried. If I have logs being written to an index daily (basically a new index is created each day), I would want my ILM policy to move the index created yesterday to a warm phase, as it is longer being updated, but still being queried for visualizations and what not. As the indexes get older, would it be more efficient to move them over to a cold phase, as they get queried less often?

If you have some good articles on how others have implemented ILM policies, I would enjoy the read as I am always looking for more ways to improve my ES cluster.

Thank you,
Chris

[Christian_Dahlqvist]

While the stages are fixed in ILM, what they mean may vary from deployment to deployment. For large use cases the different phase transitions often mean doing some operations on the index and moving it to a separate set of hardware, e.g. hot/warm or hot/warm/cold architectures. For smaller deployments where there is only one type of node a state transition may simply mean that you close the index for writing and forcemerge it down to a single segment to optimize it for long term storage. If the indices stay on the same hosts and you do not freeze them there is no need to add a cold phase.

[cbille0]

Hello @Christian_Dahlqvist,

Thank your for your response. Given our deployment architecture, we probably will not use a hot/warm/cold ILM policy, but it is good to know for the future in case our deployment architecture changes.

Could you go into more detail about forcemerge? I am not familiar with this concept.

Thank you,
Chris