Hot, warm, cold tiering with availability zones

Elastic Orchestration Elastic Cloud Enterprise (ECE)
1

Hello,
I'm trying to understand ILM behavior with ECE availability zones.
If I distribute a cluster across 2 or more availability zones, will a replica on zone 2 from a primary on zone 1 eventually move from the hot to warm tier on zone 2, when the ILM condition is reached, if the policy has been created?

2

Hi,

In our environment we have 3 Zones (3 HOT nodes on each zone & 2 WARM nodes on 2 zone).
So as per my understanding it is not like Shards move from Hot to Warm tier zone wise, it can be any zone as it depends on the number of Shards on each node. It divides equally on the 3/2 nodes WARM in our scenario.

Thanks!

3

Hello,
Thank you very much for your reply! Just to be sure that I understand:
If I have a primary shard on zone 1 with a replica shard on zone 2, the replica shard on zone 2 can be moved to the warm tier on zone 2 once ILM conditions are met. Am I understanding that correctly?
Thank you again!

4

Hi @jelliott1

Simply Put, with default routing, there is no guarantee that a primary or replica shard will stay in the same zone as it moves between data tiers.

The shard may or may not end up in the same zone.

Can I ask what your concern is or what you are trying to accomplish?

5

Hi @stephenb
Thank you for your quick reply to my question!
I'm attempting to size storage for the warm tier in a 2-zone ECE cluster. The third zone is being used as a tiebreaker. I need to make sure the warm tier has enough storage to endure a zone failure. My first thought was to size the warm tier for the replicas from the other site plus hot primaries from data being ingested at each zone. That's assuming incoming data is balanced across both zones.

6

Sorry, I am not clear about what you are trying to solve.

Warm needs to have space for primary and replicas...

Not sure what you mean by other sites etc...

You DO need to use routing if you have multiple warm nodes in each zone to ensure the primary and replica are in different zones... I *think this happens automatically if you have set up ECE properly across AZs

BTW you have ECE so by definition, you have Support so you can ask that.

7

@stephenb
Thanks for clarifying that!