We have 2 warm data nodes in our cluster, and 6 shards configured for each index. When an index migrates from hot to warm, the shards are evenly distributed across both warm nodes. However we also have a shrink action confiigured in the warm phase. But before that can happen we have to relocate half the shards again since they all need to be on the same node. Is there a way to migrate from hot to warm such that all the shards for an index go to the same node?
Welcome to our community!
Elasticsearch and ILM should handle the moving of shards for you for the shrink action. Is that not happening?
It is, it just seems wasteful to copy half the shards twice.
You could probably setup a custom policy that dropped the replicas, moved the shards to a single node, shrunk, enabled wider allocation and then add replicas back in if you wanted.
Yeah, that doesn't sound hard but wanted to check if I was reinventing the wheel
You kinda are, but it's a reasonable question to ask