Cannot connect to the Elasticsearch cluster

Elastic Stack Elasticsearch
1

Hi Everyone, I new with Kibana and Elasticsearch, I have a trouble with Kibana, when the users open Kibana show the error "Cannot connect to the Elasticsearch cluster", I search the error in the log of elasticsearch and them show this

[2022-09-16T12:58:00,749][INFO ][o.e.c.c.JoinHelper       ] [elk] failed to join {elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20} with JoinRequest{sourceNode={elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20}, optionalJoin=Optional[Join{term=82, lastAcceptedTerm=81, lastAcceptedVersion=101472, sourceNode={elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20}, targetNode={elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20}}]}
org.elasticsearch.transport.RemoteTransportException: [elk][192.168.108.40:9300][internal:cluster/coordination/join]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: incoming term 82 does not match current term 83
	at org.elasticsearch.cluster.coordination.CoordinationState.handleJoin(CoordinationState.java:223) ~[elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoin(Coordinator.java:979) ~[elasticsearch-7.4.1.jar:7.4.1]
	at java.util.Optional.ifPresent(Optional.java:176) ~[?:?]
	at org.elasticsearch.cluster.coordination.Coordinator.processJoinRequest(Coordinator.java:524) ~[elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.cluster.coordination.Coordinator.lambda$handleJoinRequest$7(Coordinator.java:490) ~[elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:367) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:354) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.cluster.coordination.Coordinator.handleJoinRequest(Coordinator.java:477) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.cluster.coordination.JoinHelper.lambda$new$0(JoinHelper.java:125) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:257) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.lambda$messageReceived$0(SecurityServerTransportInterceptor.java:306) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeSystemUser(AuthorizationService.java:376) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:184) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$1(ServerTransportFilter.java:130) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:246) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$6(AuthenticationService.java:306) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:317) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:244) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:196) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:139) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:121) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:313) [x-pack-security-7.4.1.jar:7.4.1]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:752) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.4.1.jar:7.4.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:830) [?:?]
[2022-09-16T12:58:00,827][INFO ][o.e.c.s.MasterService    ] [elk] elected-as-master ([1] nodes joined)[{elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 83, version: 101473, reason: master node changed {previous [], current [{elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20}]}
[2022-09-16T12:58:01,431][INFO ][o.e.n.Node               ] [elk] stopping ...
[2022-09-16T12:58:01,446][INFO ][o.e.x.w.WatcherService   ] [elk] stopping watch service, reason [shutdown initiated]
[2022-09-16T12:58:01,447][INFO ][o.e.x.w.WatcherLifeCycleService] [elk] watcher has stopped and shutdown
[2022-09-16T12:58:01,662][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [elk] [controller/1841] [Main.cc@150] Ml controller exiting
[2022-09-16T12:58:01,664][INFO ][o.e.x.m.p.NativeController] [elk] Native controller process has stopped - no new native processes can be started
[2022-09-16T12:58:11,709][WARN ][o.e.c.s.MasterService    ] [elk] took [11s] and then failed to publish updated cluster state (version: 101473, uuid: lHu-0t8_Q4ad8WPSd6mFhg) for [elected-as-master ([1] nodes joined)[{elk}{8sNECOpnQt2dTlAHaES51g}{KF-QmwDhSvGA0IHQs1XTGg}{192.168.108.40}{192.168.108.40:9300}{dilm}{ml.machine_memory=10465124352, xpack.installed=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_]]:
cluster uuid: HTYDxSuVTqmohvdjEIWiVA [committed: true]
version: 101473
state uuid: lHu-0t8_Q4ad8WPSd6mFhg
from_diff: false
meta data version: 98954
   coordination_metadata:
      term: 83
      last_committed_config: VotingConfiguration{8sNECOpnQt2dTlAHaES51g}
      last_accepted_config: VotingConfiguration{8sNECOpnQt2dTlAHaES51g}
      voting tombstones: []

Thanks

2

This message tells us that at some point in the past these nodes belonged to different clusters, but now you are trying to join them together ?? If you're just starting out and haven't put any data into these nodes yet then the best thing to do is to shut down the node , delete its whole data directory, and then restart it.?
Let us know if it solves the issue.

Also copying @jbudz to see if he has any additional input in this matter.

Thanks
Rashmi

3

Can you share your elasticsearch.yml?

This says something asked the node to stop. Did you stop the service?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.