Move all my shards from data-master nodes to new Data-only nodes?

Elastic Stack Elasticsearch
1

Hi I am new to ES,

I have cluster of master-data-clients nodes, I am adding 3 data nodes (only data nodes and not master)
and I wanna migrate all my shards to these new data nodes, so I can destroy my older data-master nodes, is there a query that can do that for me ?

the query should move all my data from old data-master shards to new only data nodes. let say I had 5 data-master nodes and I wanna move them to 3 new only data nodes.

2

I believe you can close your index first or better to switch off ES, then move Linux folders where you ES data are located. Keep in mind a shard is just a Lucene index.

3

Use the routing API.

>         curl -XPUT "http://any-master-node-in-cluster:9200/_cluster/settings" -d '{
>   "transient" : {
>       "cluster.routing.allocation.exclude._ip" : "IP1, IP2, IP3, etc."
>   }
> }'

Where IP1, IP2 and so on are addresses of the nodes you want to migrate shards away from.

4

This. I've used this when removing nodes from the cluster. I've used it when we had to replace the disks and we wanted to bring three nodes down at a time. Just remember this request is honored asynchronously. You can check in the _cat APIs for how many shards a node is hosting.

5

@tinle @nik9000
I tried this but it did move the data to one of the excluded ips after I shutdown one of the nodes, any idea why? I even trying shutting down one node or whole cluster.
I tried both include and exclude _ip ...none of them force the data go to the desired places. how can I troubleshoot ?
my ES version is 1.7

6

after I set the include it wont move the shards and when I try to manually move a shard to that desired host it tells me
NO(node does not match global include filters)

but its totally wrong, both ip and host match the filters

7

Sounds like you set exclude to the ips you wanted it to move to rather than from? I can't really tell though. I've certainly used this in the past and it worked.

Manually asking elaticsearch to move shards tends to be futile because it'll start moving shards back to the node that lost the shard in an effort to balance the nodes again.