3 node ES cluster on single vm

syedsfayaz · May 8, 2019, 11:00pm

HI

I want to setup a 3 node ES cluster on one VM. I tried to install and configure but seeing issues.

I installed ES to two different directories and configured it . But I am seeing error. My elasticsearch.yml file . Can you please help.

Node 1:

cluster.name: elasticsearch
node.name: node-1
path.data: ["/home/ags/Elastic/data","/data/elk/Elastic-7/data"]
path.logs: /data/elk/Elastic-7/logs
path.repo: ["/home/ags/Elastic/backup"]
network.host: 10.11.2.233
http.port: 9200
cluster.initial_master_nodes: ["master.node.com"]

Node2:

cluster.name: elasticsearch
node.name: node-2
path.data: ["/home/ags/Elastic/data1"]
path.logs: /data/elk/Elastic71/logs
path.repo: ["/home/ags/Elastic/backup"]
network.host: 10.11.2.233

node.master: false
node.data: true
node.ingest: false
cluster.remote.connect: false

http.port: 9201
cluster.initial_master_nodes: ["master.node.com"]

I am seeing logs in second node.

[2019-05-08T15:28:31,271][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node-2] master not discovered yet: have discovered []; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, [::1]:9300, [::1]:9301, [::1]:9302, [::1]:9303, [::1]:9304] from hosts providers and [{node-2}{i_WUVwRBTB-XrtwyKSTtBQ}{sFq4o45ORBu20F5Y_dKZqQ}{10.44.10.140}{10.44.10.140:9301}{ml.machine_memory=7674531840, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

dadoonet · May 9, 2019, 2:24am

I believe you need to configure what is described in this page: https://www.elastic.co/guide/en/elasticsearch/reference/current/discovery-settings.html

syedsfayaz · May 9, 2019, 7:28am

I tried the same configuration mentioned in the link. But I am seeing this errors in logs.

dadoonet · May 9, 2019, 7:51am

Which errors now? And could you share the full log please? Properly formatted please with </> icon.

syedsfayaz · May 9, 2019, 8:03am

I am unable to add logs here. Can I email you the logs.

dadoonet · May 9, 2019, 8:34am

You can add your logs on gist.github.com and share the link here.

syedsfayaz · May 9, 2019, 8:38am

gist.github.com

https://gist.github.com/syedsfayaz/4c53a63e7127e3506b1064b58bb4cfad

gistfile1.txt

[2019-05-09T00:23:46,261][INFO ][o.e.e.NodeEnvironment    ] [hostname.changed.com] using [2] data paths, mounts [[/home (/dev/mapper/vg_os-lv_home), /data (/dev/mapper/vg_os-lv_data)]], net usable_space [26.3gb], net total_space [51.1gb], types [ext4]
[2019-05-09T00:23:46,264][INFO ][o.e.e.NodeEnvironment    ] [hostname.changed.com] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-09T00:23:46,486][INFO ][o.e.n.Node               ] [hostname.changed.com] node name [hostname.changed.com], node ID [jG4MSlobSo2gNfA8Qdlzfw]
[2019-05-09T00:23:46,486][INFO ][o.e.n.Node               ] [hostname.changed.com] version[7.0.0], pid[2498], build[default/tar/b7e28a7/2019-04-05T22:55:32.697037Z], OS[Linux/3.10.0-693.21.1.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2019-05-09T00:23:46,487][INFO ][o.e.n.Node               ] [hostname.changed.com] JVM home [/data/arcgis/jdk1.8.0_121/jre]
[2019-05-09T00:23:46,487][INFO ][o.e.n.Node               ] [hostname.changed.com] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-4177066114497249996, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Dio.netty.allocator.type=pooled, -Des.path.home=/data/elk/Elastic-7, -Des.path.conf=/data/elk/Elastic-7/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2019-05-09T00:23:48,726][INFO ][o.e.p.PluginsService     ] [hostname.changed.com] loaded module [aggs-matrix-stats]
[2019-05-09T00:23:48,727][INFO ][o.e.p.PluginsService     ] [hostname.changed.com] loaded module [analysis-common]
[2019-05-09T00:23:48,727][INFO ][o.e.p.PluginsService     ] [hostname.changed.com] loaded module [ingest-common]
[2019-05-09T00:23:48,727][INFO ][o.e.p.PluginsService     ] [hostname.changed.com] loaded module [ingest-geoip]

This file has been truncated. show original

dadoonet · May 9, 2019, 9:26am

What are the logs of the other server?

Could you share also the final complete configuration of both nodes?

syedsfayaz · May 9, 2019, 11:28pm

Hi Dadoonet, I got it working. I added the wrong port in the elasticsearch.yml so it was not able to discover the master node.

Kaloyan_Dimov · June 4, 2019, 7:39pm

Would you mind showing how you got the port right ? I am having the exact same error and I am unable to make node-2 discover the master.

system · July 2, 2019, 7:39pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.