Moving Index/shards from Hot Tier to Warm Tier using custom Attributes within zones using ILM

Elastic Stack Elasticsearch
1

I usually move an index to warm tier using ILM and the default _tier_preference if I have the nodes defined as data_warm.

Lets say I have multiple indices with varied number of shards. index1 with 5 shards and index 2 with 6 shards with 1 replicas. spread over 3 zones. (nodes.attr.az: us-east-1a, us-east-1b, us-east-1c). There is zone awareness enabled.
image

I have created a custom attribute "node-type": "warm" for all of my warm nodes. I would like to move my indices from hot nodes to "node-type" : "warm". But also, I want to move shards in hot tier us-east-1a move to warm node-type in us-east-1a, same way to us-east-1b and us-east-1c. Is there any way to achieve this with ILMs.

2

No, that is not possible to control. I believe new replicas are always copied from the current primary.

3

Copying shards within zone between tiers would save lot of cost caused by Data Transfer.
So, no way of doing that. Hmm.

4

Not new replicas, once the shards are already allocated and in hot after rollover and after a certain period of time specified in ILM, the index gets moved to warm tier, In this scenario, I am looking to move shards within the zone while they move to warm nodes.

5

When you "move" shards, I believe the mechanism behind the scenes is that new replicas are allocated based on the primary shard and the existing one then removed once the new replica is online. The end result is equivalent to a move though so it makes sense describing it as a move to a new tier.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.