Methodology for merge data nodes to big one

[INS (Borys)]

Hi I need to change architecture of my cluster, thus I need to know Your's idea/approaches to do this.

So currently my cluster consist on 3 bare metal server with sdd and hdd disk.
But I need to increase memory of these nodes therefore it's needed to merge the data nodes.
Can You keep an eye on below schema of my cluster

One node is releated to one physical disk ---> this is the currently configuration.

As fist it will be changed file system to merge the disk but what should I do with data??? Do we have any dedicated plugin?

[leandrojmp (Leandro Pereira)]

Can you provide more context about your infrastructure?

You have 3 bare metals and the nodes you shared in this image are virtual machines running on those bare metals?

For example, if you need to remove the nodes data hot1_1, data hot2_1 and data hot3_1 to create a new node named data hot123_1 you will need to first remove all the data from these 3 nodes and allocate the data in the remaining data hot nodes, when there is no shard on those nodes you can remove them from the cluster and add the new node.

This documentation.

This is the first thing you need to do, you can't merge the disks while the nodes are running and have data, if you do that you will lose data, you need to remove the nodes from allocation, remove them from cluster, create your new node, add the new node to the cluster, wait for the rebalance to finish and them repeat this for all your other nodes.

You also need to have enough available space in the other nodes to allocate the shards.

[INS (Borys)]

@leandrojmp

For example, if you need to remove the nodes data hot1_1, data hot2_1 and data hot3_1 to create a new node named data hot123_1 you will need to first remove all the data from these 3 nodes

so theses nodes are docker containers, How can I exclude this for ex 3 nodes so as to smush the data for allocation to the other two sites.

PUT _cluster/settings
{
  "persistent" : {
    "cluster.routing.allocation.exclude._name" : "data hot1_1, data hot2_1, data hot3_1"
  }
}

Does elasticsearch in 8.1 support multidisk ? It means that on nodes can covers a few dir for store data from one node

[leandrojmp (Leandro Pereira)]

@INS

How can I exclude this for ex 3 nodes so as to smush the data for allocation to the other two sites.

I do not use docker, but the logic is the same you need to exclude from allocation the nodes you want to remove, you can exclude them by name or id as well, it is in the documentation.

@INS

Does elasticsearch in 8.1 support multidisk ? It means that on nodes can covers a few dir for store data from one node

This is multipath, it is still supported but it is deprecated and will be removed in the future you should avoid start using it now.