Stop elastic shard store on old elastic node

Elastic Stack Elasticsearch
1

Hi All,

I want to migrate existing elastic nodes to the new nodes using the join cluster method, So I don't have to backup-restore the data and kibana feature configuration. The details are below:

  1. I have 3 existing elastic nodes, and it will become old nodes.
  2. I also have 3 new elastic nodes.

I am joining all nodes, so there are 6 nodes in cluster with shards configuration 1 primary and 1 replica.

Is it possible to stop storing the data to 3 old elastic nodes, so next time forward the data only store in 3 new nodes, but the data on 3 old elastic nodes not removed or not migrated to the new nodes?

Thanks.

Best Regards,
Julian Fazri

2

Yes, it is possible to use IP addresses to exclude nodes instead of using node attributes. You can do this by using the Elasticsearch _settings API. Here is an example:

curl -X PUT "localhost:9200/_settings" -H 'Content-Type: application/json' -d'
{
  "index.routing.allocation.exclude._ip": "192.168.1.1,192.168.1.2,192.168.1.3"
}
'

Regards

1 Like
3

When you remove the 3 old nodes make sure you do not just drop all of them as 4 master eligible nodes are required to elect a master in a cluster with 6 master eligible nodes. Make sure to remove the nodes using the voting config exclusions API before you remove them.

4

Hi @yago82,

I have tried this, what happen is the shards was moved from excluded IPs and there are no shards on excluded IPs.

Is that indicates the data was moved? We don't want that.

Thanks.

5

I see I misunderstood. You intend to keep all nodes and not remove them. To control on which nodes specific indies are stored you can use shard allocation filtering. Label the old nodes one way and the new ones differently and then assign new indices to the new nodes through index template settings aligned with the shard allocation filtering.

6

Hi,

you can try this, at index level:

Regards

7

Hi @Christian_Dahlqvist and @yago82 ,

If I use this, should I applied one by one to each index? Should I do this for system indices and data stream indices also?

8

It has to be appliend to all indices that you wish to control the location of.

9

I have around 1000 indices, should I set it one by one? Or is there any ways like if the data after specific time, it will stored at new nodes?

10

You can automatically apply this to all new indices by index template for example

11

How about the old indices? I have try for index level shard allocation, the result is my shard on the old elastic nodes were moved. I am using the integration from elastic agent so index made by datastream.

Is it possible liks before today the data still stored one old nodes and not moved, but from today forward the data stored on new elastic nodes?