Setting up cluster with client, master & data node

[subhraakanil (subhranil chattopadhyay)]

Hi Team,

We are making POC on elasticsearch so that we can use it in future.Though, I am newbie in elasticsearch.

We have three CentOs boxes and our requirement is to set up a cluster containing client, master & data node.

1. We have planned to run 2 instances of elasticsearch in the first server on different port i.e. [9200,9201] and made them as client node.

   node.master: false
node.data: false
node.ingest: false

2. In the second server, we are going to run 4 instances of elasticsearch on different port i.e.[9200,9201,9202,9203] and made them behave as master node.

   node.master: true
node.data: false
node.ingest: false

3. In the third server, we are going to run 4 instances of elasticsearch on different port i.e.[9200,9201,9202,9203] and made them behave as data node.

node.master: false
node.data: false
node.ingest: true

This approach i.e. running multiple instances in the same server is just for POC, if we get success then we will make elasticsearch instance and server in [1:1] .

We have configured each elasticsearch.yml in the below mentioned way:

We have kept the cluster.name property same in all the 10 instances inside elasticsearch.yaml.

We have kept network.host for all the client nodes same, as they are running inside same server. We have done the same thing for master & data node.

For all the 10 instance running, i.e. client**(2)** , master**(4)** & data**(4)**, we have mentioned all the 10 ip's along with port number for discovery.zen.ping.unicast.hostsattribute inside all the elasticsearch.yml instance.

discovery.zen.ping.unicast.hosts: 
["<client_node_ip>:9200","<client_node_ip>:9201","<master_node_ip>:9200","<master_node_ip>:9201","<master_node_ip>:9202","<master_node_ip>:9203","<data_node_ip>:9200","<data_node_ip>:9201","<data_node_ip>:9202","<data_node_ip>:9203"]

Last day, I have tried the same, where I have run single elastic instance in each server, considering once master and two data and successfully form the cluster and validated with :

curl -XGET 'http://<server_ip>:9200/_cluster/state?pretty'

It is showing all the three nodes.

But, today after doing the above mentioned modification i.e. (2 elasticsearch client instance in first server, 4 elasticsearch master instance in second server, 4 elasticsearch data instance in third sever), I am not been able to form the cluster.

After pinging, each instance individually , I am getting response:

curl -XGET 'http://<server_ip>:9200'

But, they are not inside cluster.

While applying the below command from any of the master node, it is just showing that particular node:
curl -XGET 'http://<master_server_ip>:9200/_cluster/state?pretty'

While applying the below command from any of the client node, it is just giving the following error:
curl -XGET 'http://<client_server_ip>:9200/_cluster/state?pretty'

{
_ "error" : {_
_ "root_cause" : [_
_ {_
_ "type" : "master_not_discovered_exception",_
_ "reason" : null_
_ }_
_ ],_
_ "type" : "master_not_discovered_exception",_
_ "reason" : null_
_ },_
_ "status" : 503_
}

[subhraakanil (subhranil chattopadhyay)]

We are facing the issue as we have mentioned http host.port in place of transport.profiles.default.port as part of the value of the attribute named as discovery.zen.ping.unicast.hosts inside elasticsearch.yml file.
After changing it accordingly, my issue got resolved.And, we have successfully build the cluster.
After, running the below command from client node, we have got to see all 10 nodes that has formed the cluster.

curl -XGET 'http://<client_server_ip>:9200/_cluster/state?pretty

[Christian_Dahlqvist]

If the servers are of the same specification, I would recommend setting up one of each node (master, data and coordinating node) type per server as that is a more realistic deployment. You ideally want to have 3 dedicated master nodes and make sure they are distributed across different hosts in order too provide high availability. The same thing goes for data nodes.

[subhraakanil (subhranil chattopadhyay)]

Thank you very much for your advice @Christian_Dahlqvist
What I have understood from your advice, that it is better to run single elastic instance in each server irrespective of the node type to ensure high availability.e.g. if I want three master nodes, then we have to configure three servers to run individual elasticsearch instance , where each instance will be considered as master node(i.e. I have to configure elasticsearch.yml accordingly).Same is applicable for data nodes as well.

Kindly correct me sir, If I have not understood your point.

Is client node & coordinating node are the same thing?

[Christian_Dahlqvist]

Having a separate host for each node is ideal. For smaller clusters having 3 nodes with the default configuration is however often fine. Just because you can have dedicated node types does not necessarily mean that you should.

If you however still want to experiment with dedicated node types, you could deploy 3 nodes (one dedicated master, one dedicated data and one coordinating node) on each host.

[subhraakanil (subhranil chattopadhyay)]

Thanks @Christian_Dahlqvist . It has been acknowledged.