Intermittent No node available exceptions

Elastic Stack Elasticsearch
1

We have a 7 node cluster with 6 of them running ES version 5.2.1. One of the nodes is on 5.6.6. We observe that this node leaves the cluster intermittently.
I have grabbed the log when the node disconnects from the cluster
As per this link

this might come from different versions of ES and different version of java as well. But since that's an issue for older version of ES I am trying to reconfirm.
Unfortunately java versions on the nodes are different as well

On 5.2.1 node
$ java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

On 5.6.6 node
$java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

Here is the log when the node disconnects from the cluster
https://pastebin.com/5wCEaQXs

I am just trying to confirm if the different ES versions is the issue here or it could be something else.

2

Apart from during rolling upgrades, you should never run clusters containing mixed versions. Shards that get placed on the node with the never version can not be read by the other nodes as the Lucene version differs. This may over time cause indices to go yellow as primaries on the newer node can not be replicates to any of the other nodes in case of failure.

I am not sure this necessarily causes the intermittent disconnect, but would recommend ensuring all nodes are running the same version and see if this has any impact.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.