Apache-tomcat - 7.0.65
Elasticsearch - 1.7.3 - no plugins installed
Java - 1.7.0_80
Java client connection properties given
elasticsearch.address=127.0.0.1
elasticsearch.port=9300
elasticsearch.writerThread=10
elasticsearch.readerThread=10
elasticsearch.clusterName=mystores
Elasticsearch is not clustered, zen multicast disabled, Nodename is hostname(taken from environment variable).
Issue : Elasticsearch connection gets lost and tomcat starts throwing the following error continuously
2016-02-02 18:59:52,279 [pool-1-thread-5] INFO com.clogic.eventstore.es.ESEventDataStore - disconnected client instance closed
2016-02-02 18:59:52,289 [pool-1-thread-5] INFO org.elasticsearch.plugins - [Doctor Strange] loaded [], sites []
2016-02-02 18:59:52,309 [pool-1-thread-5] INFO com.clogic.eventstore.es.ESEventDataStore - new client created
2016-02-02 18:59:52,309 [pool-1-thread-5] INFO com.clogic.eventstore.es.ESWriter - exception e org.elasticsearch.client.transport.NoNo
deAvailableException: None of the configured nodes are available: []
2016-02-02 18:59:52,310 [pool-1-thread-10] INFO com.clogic.eventstore.es.ESEventDataStore - no connected ES client. creating a new one
now
2016-02-02 18:59:52,315 [pool-1-thread-10] INFO com.clogic.eventstore.es.ESEventDataStore - disconnected client instance closed
2016-02-02 18:59:52,324 [pool-1-thread-10] INFO org.elasticsearch.plugins - [Hermes] loaded [], sites []
2016-02-02 18:59:52,344 [pool-1-thread-10] INFO com.clogic.eventstore.es.ESEventDataStore - new client created
2016-02-02 18:59:52,344 [pool-1-thread-10] INFO com.clogic.eventstore.es.ESWriter - exception e org.elasticsearch.client.transport.NoN
odeAvailableException: None of the configured nodes are available: []
2016-02-02 18:59:52,344 [pool-1-thread-7] INFO com.clogic.eventstore.es.ESWriter - exception e org.elasticsearch.client.transport.NoNo
deAvailableException: None of the configured nodes are available: []
One point noted here is, it seems the node name to which the Java client tries to connect changes for every attempt.
The connection issue does not start after a specific time,it gets lost randomly after tomcat is started and client tries to reconnect. This is also sometimes successful, but after some time the above error is continuously thrown until tomcat is restarted.
Also, even at startup the node name shown is random, but it works as shown below
2016-02-02 19:05:30,528 [localhost-startStop-1] INFO com.clogic.eventstore.es.ESEventDataStore - no connected ES client. creating a ne
w one now
2016-02-02 19:05:30,669 [localhost-startStop-1] INFO org.elasticsearch.plugins - [John Doe] loaded [], sites []
2016-02-02 19:05:31,666 [localhost-startStop-1] INFO com.clogic.eventstore.es.ESEventDataStore - new client created
Any input is appreciated..