Elasticsearch Cluster, how to create 2 data nodes single node to primary shards and the other to replica?

Elastic Stack Elasticsearch
1

Elasticsearch Cluster, what is the best configuration to create 2 data nodes single node to primary shards and the other to replica??

2

Elasticsearch will try to balance the shards, irrespective of if they are primary or replica.

3

@warkolm you mean that 2 nodes have the same configuration as data-node and elasticsearch balance replica by default

4

It balances all shards, irrespective of if they are primary or replica.

5

@warkolm OK but I need to create replica indecies in another data-node how I configure it?

Can you help me?

6

The only way to do that is to manually reassign/reroute shards and manually manage allocation.

We don't recommend doing this.

7

@warkolm OK but in case single data node is down what is the best solution to get data contained into this indecies?

8

Can you rephrase that please? I don't understand what you mean.

9

@warkolm sorry I mean that if data node is down what about data saved into indecies of this node

10

If you have a single node cluster, then you have no redundancy.

If you have 2 nodes, then your primary and replica shards will be assigned across both nodes in the cluster. So if one node goes down (doesn't matter which one), then the other node still has a complete copy of the data and you don't lose anything.

However we recommend a minimum of 3 nodes in a cluster, see here

11

@warkolm cluster with 3 master node and 2 data nodes contains all the same indexes and all of nodes master and data nodes included into the same AWS VPC
Is this the best cluster design??

12

The best depends on how much data you have, what sort of use case, etc etc.

I would probably start with 3 nodes, that are both master eligible and data nodes.

13

@warkolm OK I understand this
But if three nodes work as master and data, I think the three nodes should have highly specs ex. 32G RAM isn't it?

14

How much data is it?
What sort of data?
What sort of queries?

15

@warkolm At least 5G logs by day and queries aggregations to get data analytics
We have more indexes at least 50 index and retention is annually

16

Then you might need those at some point, but not immediately.

17

@warkolm you means we need to start with low specs and scale later due to data?

18

Yes.

19

@warkolm Very very thanks I will start to build our cluster and if I have any questions I will mention you back :pray: sorry for headache my buddy :wink::heart:

20

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right

1 Like