ILM increasing number of replicas - suboptimal behavior?

Elastic Stack Elasticsearch
1

Hello nice elastic people.
I am currently toying with ILM (7.x, not yet upgraded to 8.0) and ran into a surprising behavior.

I have configured ILM to increase the number of replicas when transitioning from hot to cold.
However, at least when triggering the ILM transition manually by Move to lifecycle step API | Elasticsearch Guide [8.0] | Elastic, it seems to behave unexpectedly.

I start with an index on the data_hot nodes (replicas: 0).
Then I manually trigger ILM transition.
ILM configures the desired number of replicas (1).
ES starts replicating the index onto another data_hot node.
ILM waits for the replication to finish.
ILM configures _tier_preference and indices start to migrate to data_cold(s).

Why does it work like that?
Why is the replica created on the hot nodes just to be migrated away immediately?

What did I miss?
What is preventing this new replica from being put directly onto data_cold(s)?

2

(post deleted by author)

3

I think that's how I'd expect it to work today, but I could see why you would want these two things to happen together. Please open a bug report.

4

Thank you for the reply. Actually, I was sure there must be a good reason why it works the way it does. I'd still like to find out why and when it is beneficial for it to work like this. Can you please at least point me in the right direction?

I assume bug reports are opened via github. Ok, I'll remember it in case I ever own a Microsoft account. Sorry about that.

5

It's not a very good reason I think. The number of replicas is set in the allocate action whereas the move to the next tier is part of the migrate action and ILM runs its actions in sequence whereas it seems preferable to consolidate these two actions in your case. It sounds totally fixable tho, it's just that nobody has reported it yet.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.