Scaling elasticsearch

Elastic Stack Elasticsearch
1

Hello,

My cluster is composed of 3 nodes (Master node, hot node and warm noeaud) i want to scal my cluster by adding a new elastic node which will be dedicated only for data replication. The replicas shard of all the cluster nodes will be sotred only in this replica node. Is it possible to implement this scenario? if yes how?

Thanks,

2

There's nothing native to Elasticsearch that will automatically handle this for you sorry.

1 Like
3

What are you looking to achieve by scaling out the cluster? If you are looking for high availability and resiliency you should look to ensure that you have at least 3 master eligible nodes in the cluster and at least one replica shard configured. Replicas shards in Elasticsearch do roughly the same amount of work as the primary shards in that they index all data locally and serve queries. If you want to add replicas for added resiliency you should therefore add one hot and one warm node. If you had a copy of all shards in the cluster on a single additional node it would most likely get overloaded and cause performance and stability problems.

4

Hello,

Thanks Warkolm and Christian_Dahlqvist for the reply.

After some research I think the Index shard allocation can help me to implement this scenario.

Best regards,

5

I suspect you can set this up by adding a node that has both hot and warm data roles and add a replica. What i am saying is that you should not as this node will likely be overloaded. As replica shards do the same work as primary shards and you can not control where the primaries live sich a node would likely have tbe same load as the other two nodes combined.

6

Hello christian.

Thank-you for your prompt response. Maybe i didn’t explain well. We have multiple DCs without the cross-cluster replication license. Each Elastic stack has 3 Elasticsearch nodes (data hot, data warm and master)

I need to replicate the logs of all the DCs in two Elasticsearch node that are in different DCs. These nodes will dedicate only to logs storage.

Best regards.

7

Any nodes added will form part of a stretched cluster if you deploy across two DCs, so my comments are still valid. If you need to replicate data across the environments I would recommend doing so periodically using snapshot and restore or simply write to both clusters in parallel.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.