Elasticsearch configuration

Elastic Stack Elasticsearch
21

Sorry due to limitation of number of characters I shared all the nodes logs separately

22

This one seems to be resolve now.

curl -XGET 'orw-elkm1-vm:9200/_cat/nodes?v'
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.102.86.119            1          60   1    0.30    0.23     0.14 m         -      orw-elkm2-vm
10.102.85.117            1          59   1    0.42    0.31     0.15 m         *      orw-elkm1-vm
10.102.85.245           26           9   1    0.30    0.21     0.11 di        -      orw-elkd1-vm
23

I stopped the service on all nodes and wiped the lib directory.

rm -rf /var/lib/Elasticsearch/*

Thanks you all for all your support.

Thanks
Niraj

24

This helped because this was your issue:

Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: join validation on cluster state with a different cluster uuid Bp1EZ69WSFq8IR4DFe8nUw than local cluster uuid WwcRVF99Qmu4J2azujyM9g, rejecting

It seems that your nodes have joined a previous cluster before for some reason, deleting the data directory of the nodes, as you did, would wipe out this information and allow them to join a new cluster.

25

Just want to confirm 2 things.

1- I have two master nodes so in case 1 goes down the cluster would be up and running , right ?

2- I have 3 data nodes so if 1 goes down we can survive without any data lose , correct ?

26

Correct, make sure you have primary indices with at least 1 replica.

27

Is there any specific way to achieve this ?

28

No, you need 3 master nodes for high availability, with just 2 master nodes if one of them goes down, your entire cluster will not work.

Read this part of the documentation about cluster resilience and also this one about resilience in small cluster.

WIth 2 nodes you do not have resilience to failure, you will need 3 master nodes or at least 1 tie-break node as explained in the documentation.

Also, you have 2 master nodes and just 1 data node, you cannot lose any master and you can also not lose the data node, you can also not use replicas as you only have 1 data node.

The best way in this case would be to have all your 3 nodes working as both master anda data nodes.

1 Like
29

Please don't. 7.0 is very well past EOL, 7.15 is latest and you should be using that.

This doesn't make a lot of sense. Just set network.host and all of the rest will be inherited from that.

30

Thank you all for your support. Cluster is up now.

curl -X GET "rw-elkd1-vm:9200/_cluster/health?pretty"

{
  "cluster_name" : "elk-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 15,
  "active_shards" : 30,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
31

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.