Hello,
I have 4 node cluster and after enabling xpack TLS/SSL, I am having issue in starting the elasticsearch service on 2 nodes. Getting below error message on both the nodes.
[2020-08-22T14:21:40,269][INFO ][o.e.n.Node ] [ANDDEVELASTIC01] stopping ...
[2020-08-22T14:21:40,799][INFO ][o.e.x.w.WatcherService ] [ANDDEVELASTIC01] stopping watch service, reason [shutdown initiated]
[2020-08-22T14:21:40,800][INFO ][o.e.x.w.WatcherLifeCycleService] [ANDDEVELASTIC01] watcher has stopped and shutdown
[2020-08-22T14:21:41,856][INFO ][o.e.c.c.Coordinator ] [ANDDEVELASTIC01] master node [{ANDDEVELASTIC03}{OSC_toI8QMqKplc5AgFXcg}{V6-2VoQcTGSIdTj21Wy-Tg}{ANDDEVELAST
IC03}{165.113.xx.128:9300}{ilmr}{ml.machine_memory=16637517824, ml.max_open_jobs=20, xpack.installed=true, transform.node=false}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [ANDDEVELASTIC03][165.113.21.128:9300][disconnected] disconnected
[2020-08-22T14:21:42,632][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [ANDDEVELASTIC01] [controller/90261] [Main.cc@150] Ml controller exiting
[2020-08-22T14:21:42,922][INFO ][o.e.x.m.p.NativeController] [ANDDEVELASTIC01] Native controller process has stopped - no new native processes can be started
[2020-08-22T14:21:53,001][WARN ][o.e.c.c.ClusterFormationFailureHelper] [ANDDEVELASTIC01] master not discovered or elected yet, an election requires at least 2 nodes wi
th ids from [sbqpNN1YRBCzT6hPBynm-w, OSC_toI8QMqKplc5AgFXcg, pXyVJpBBRu-5xHeCd47I0Q], have discovered [{ANDDEVELASTIC01}{sbqpNN1YRBCzT6hPBynm-w}{PKiao3NvRe2NSu1QN-cabw}
{anddevelastic01}{165.113.21.133:9300}{dilmrt}{ml.machine_memory=16637517824, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is not a quorum; di
scovery will continue using [165.113.xx.127:9300, 165.113.xx.128:9300] from hosts providers and [{ANDDEVELASTIC01}{pXyVJpBBRu-5xHeCd47I0Q}{zPEUi5DsRSS5Wg8R1GC6yA}{ANDDE
VELASTIC02}{165.113.xx.127:9300}{lmr}{ml.machine_memory=16637501440, ml.max_open_jobs=20, xpack.installed=true, transform.node=false}, {ANDDEVELASTIC03}{OSC_toI8QMqKplc
5AgFXcg}{V6-2VoQcTGSIdTj21Wy-Tg}{ANDDEVELASTIC03}{165.113.xx.128:9300}{ilmr}{ml.machine_memory=16637517824, ml.max_open_jobs=20, xpack.installed=true, transform.node=fa
lse}, {ANDDEVELASTIC01}{sbqpNN1YRBCzT6hPBynm-w}{PKiao3NvRe2NSu1QN-cabw}{anddevelastic01}{165.113.21.133:9300}{dilmrt}{ml.machine_memory=16637517824, xpack.installed=tru
e, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 3, last-accepted version 1028 in term 3
[2020-08-22T14:22:03,078][WARN ][o.e.c.c.ClusterFormationFailureHelper] [ANDDEVELASTIC01] master not discovered or elected yet, an election requires at least 2 nodes wi
th ids from [sbqpNN1YRBCzT6hPBynm-w, OSC_toI8QMqKplc5AgFXcg, pXyVJpBBRu-5xHeCd47I0Q], have discovered [{ANDDEVELASTIC01}{sbqpNN1YRBCzT6hPBynm-w}{PKiao3NvRe2NSu1QN-cabw}
{anddevelastic01}{165.113.21.133:9300}{dilmrt}{ml.machine_memory=16637517824, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is not a quorum; di
scovery will continue using [165.113.xx.127:9300, 165.113.xx.128:9300] from hosts providers and [{ANDDEVELASTIC01}{pXyVJpBBRu-5xHeCd47I0Q}{zPEUi5DsRSS5Wg8R1GC6yA}{ANDDE
VELASTIC02}{165.113.xx.127:9300}{lmr}{ml.machine_memory=16637501440, ml.max_open_jobs=20, xpack.installed=true, transform.node=false}, {ANDDEVELASTIC03}{OSC_toI8QMqKplc
5AgFXcg}{V6-2VoQcTGSIdTj21Wy-Tg}{ANDDEVELASTIC03}{165.113.xx.128:9300}{ilmr}{ml.machine_memory=16637517824, ml.max_open_jobs=20, xpack.installed=true, transform.node=fa
lse}, {ANDDEVELASTIC01}{sbqpNN1YRBCzT6hPBynm-w}{PKiao3NvRe2NSu1QN-cabw}{anddevelastic01}{165.113.xx.133:9300}{dilmrt}{ml.machine_memory=16637517824, xpack.installed=tru
e, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 3, last-accepted version 1028 in term 3
[2020-08-22T14:22:06,805][INFO ][o.e.n.Node ] [ANDDEVELASTIC01] stopped
[2020-08-22T14:22:06,805][INFO ][o.e.n.Node ] [ANDDEVELASTIC01] closing ...
[2020-08-22T14:22:07,538][INFO ][o.e.n.Node ] [ANDDEVELASTIC01] closed
Configuration file:
#
cluster.name: elasticstack-dev
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ANDDEVELASTIC01
#
# Add custom attributes to the node:
node.master: true
node.data: true
node.ingest: true
#
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
bootstrap.memory_lock: true
network.host: 165.113.XX.XX
http.port: 9200
network.publish_host: anddevelastic01
discovery.seed_hosts: ["165.113.XX.XX", "165.113.XX.XX", "165.113.XX.XX"]
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: certs/ANDDEVELASTIC01.key
xpack.security.http.ssl.certificate: certs/ANDDEVELASTIC01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.key: certs/ANDDEVELASTIC01.key
xpack.security.transport.ssl.certificate: certs/ANDDEVELASTIC01.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca.crt