Move existing index from hot node to cold node

Elastic Stack Elasticsearch
1

Hello all,

I currently have created an mapped a new ILM polciy for my indices right now. But there are some older indices which are mapped to the older ILM policy. This index size is very big and currently I need to move this index to cold phase as my hot node is filling fast. I have changed the index settings as

"allocation": {
"include": {
"_tier_preference": "data_cold"

But still the index is in the hot node. I need to move this to cold phase as soon as possible as my hot nodes are filling.

image
image871×324 27 KB

image
image452×168 7.79 KB

In addition to this issue, I have created the new ILM policy to move indices from hot to cold after 3 days. But when I check, the indices are like 7 days older but still in the hot phase. When I check whether it is transfered to cold it is still in hot.

image
image798×275 24.3 KB

If you could help me on this matter it would be great.

Thank you.

2

Hi,

A few initial thoughts:

  • Once you create a new policy, you can specify that it should be applied to all new indexes, or you can also manually apply the policy to an existing index:
    Manage existing indices | Elasticsearch Guide [8.17] | Elastic

  • Secondly, I can see in the first screenshot that the logs policy has the default value of "max_age" = 30days which means that it is meant to migrate your index from hot once it reaches 30 days old - is this the current policy you want to apply or the previous one?

To get some more context - which version of Elastic are you running on?
There is a troubleshooting section on our docs explaining that:
The _tier_preference setting might conflict with other allocation settings. This conflict might prevent the shard from allocating. A conflict might occur when a cluster has not yet been completely migrated to data tiers.

This is due to a change applied since 7.10 as explained here:

Let me know if this helps!