Node not joining Elasticsearch cluster

Elastic Stack Elasticsearch
1

I have made an Elasticsearch cluster of 3 nodes but one of the node is not joining the cluster.

Below are my elasticsearch.yml file for all 3 nodes.

NODE1
cluster.name: ES-Cluster
node.name: node-1
node.master: true
node.data: false
path.data: /solr-index/Elasticsearch/Elasticdata/elasticsearch
path.logs: /solr-index/Elasticsearch/elasticsearch
network.host: ip_node_1
http.port: 9200
transport.port: 9300
discovery.zen.ping.unicast.hosts: ["ip_node_1", "ip_node_2", "ip_node_3"]
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
discovery.zen.minimum_master_nodes: 2

NODE2
cluster.name: ES-Cluster
node.name: node-2
node.master: true
node.data: true
path.data: /solr-index/Elasticsearch/Elasticdata/elasticsearch
path.logs: /solr-index/Elasticsearch/elasticsearch
network.host: ip_node_2
http.port: 9200
transport.port: 9300
discovery.zen.ping.unicast.hosts: ["ip_node_1", "ip_node_2", "ip_node_3"]
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
discovery.zen.minimum_master_nodes: 2

NODE3
cluster.name: ES-Cluster
node.name: node-3
path.data: /solr-index/Elasticsearch/Elasticdata/elasticsearch
path.logs: /solr-index/Elasticsearch/elasticsearch
network.host: ip_node_3
http.port: 9200
transport.port: 9300
discovery.zen.ping.unicast.hosts: ["ip_node_1", "ip_node_2", "ip_node_3"]
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
discovery.zen.minimum_master_nodes: 2

All three nodes are in green status. PFB the output of curl -XGET 'http://private_ip:9200/_cluster/health?pretty' for all three node

NODE1
{
"cluster_name" : "ES-Cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 2,
"active_shards" : 2,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

NODE2
{
"cluster_name" : "ES-Cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 12,
"active_shards" : 19,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

NODE3
{
"cluster_name" : "ES-Cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 12,
"active_shards" : 19,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

Please help in resolving this issue as I am not able to proceed further because my cluster of 3 node is not getting formed. Also please let me know if any other input is required to help me out on this issue.

Thanks

2

Which version of Elasticsearch are you using? Given that node 1 reports a green state it would seem it does not use the config file you provided as minimum_master_nodes should make it red.

3

I am using ES version 6.8.15 on all the 3 nodes

4

How was Elasticsearch installed on these nodes? It does look like node 1 does not use the config file you posted as this should have resulted in a red state on that node. Can you verify that the node is usíng the expected file?

You can also run the cat nodes api on the nodes and check that the roles are as expected.

5

followed below script to install elasticsearch

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

sudo apt-get update

sudo apt-get install elasticsearch

sudo nano /etc/elasticsearch/elasticsearch.yml

sudo nano /etc/elasticsearch/jvm.options

sudo systemctl start elasticsearch

sudo systemctl enable elasticsearch

sudo systemctl status elasticsearch

now node1 is in green state but not joining the cluster. how to verify if the node is using the expected file ?

6

What is the output of the cat nodes API?

7

GET /_cat/nodes

showing node-1 and node-3

All nodes are in green status but node-2 is not joining the cluster. I am stuck with this issue and not able to proceed further.

I have made all 3 nodes as master and data node i.e i have put node.master: true and node.data: true for all three nodes

8

Please show the full output from all nodes for the cat nodes API.

9

NODE1

ip_node_1 50 86 0 0.14 0.07 0.05 mdi * node-1
ip_node_3 48 93 1 0.02 0.11 0.14 mdi - node-3

NODE2

ip_node_2 38 98 0 0.05 0.08 0.08 mdi * node-2

NODE3

ip_node_1 50 86 0 0.11 0.06 0.05 mdi * node-1
ip_node_3 49 96 10 0.26 0.16 0.16 mdi - node-3

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.