Modify index.routing.allocation.total_shards_per_node by ILM when going to WARM

Elastic Stack Elasticsearch
1

I have set "index.routing.allocation.total_shards_per_node" :1 in index template
for index with 6 primary shards. Because it is index intensive ETL and having shards on different nodes gives best performance results.
However I have only 4 warm nodes.
When index age moves shards to warm nodes it moves only 4 shards and 2 remains on hot nodes.
Is it possible to set "index.routing.allocation.total_shards_per_node" : null in ILM policy ?
It there a different approach how to achieve this (other than adding 2 warm nodes )?

thank you

2

It isn't today but it will be in 7.16:

2 Likes
3

great thank you.
I suppose I will use today as a workaround the"index.routing.allocation.total_shards_per_node" : 2
it will not perform as good as 1shard per node but it will shift all shards to warm nodes and resolve my problem.

4

Sounds reasonable. Elasticsearch should be trying to spread the shards of the index out across the nodes anyway, but doesn't guarantee to do so without this setting.

If there's a period of time between rolling over and moving the data to the warm nodes (e.g. you retain N days of completed indices on the hot nodes) then you can write yourself a script to remove index.routing.allocation.total_shards_per_node once the index has rolled over. That's something that I've seen other folks do to work around this.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.