Hi,
I'm running the following version of elasticsearch on my ubuntu server.
curl localhost:9200
{
"status" : 200,
"name" : "Desmond Pitt",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "1.4.5",
"lucene_version" : "4.10.4",
....
}
}
But after a while (sometimes months sometimes days) I get a connection refused on elasticsearch.
When i just restart the service, it suddenly works again.
But since this runs in production I don't want the elasticsearch service to suddenly break.
It runs on localhost (port 9200), on the same server where the rest of the code resides.
Running curl localhost:9200 also returns a connection refused.
When I look into the logs I see these errors, ES stopped working on ~10:50 when i Restarted at 11:09. The [WARN ][cluster.service ] [George Washington Bridge] failed to reconnect to node
seems to constantly appear but has no effect.
[2016-11-15 10:44:48,789][WARN ][cluster.service ] [George Washington Bridge] failed to reconnect to node [George Washington Bridge][y0XjMTweT-CTRyGtLFObhA][bitnami-nginxstack-50be][inet[/100.114.244.37:9300]]
org.elasticsearch.transport.ConnectTransportException: [George Washington Bridge][inet[/100.114.244.37:9300]] connect_timeout[30s]
at
...
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
... 3 more
[2016-11-15 11:09:10,252][INFO ][node ] [Desmond Pitt] version[1.4.5], pid[33442], build[2aaf797/2015-04-27T08:06:06Z]
[2016-11-15 11:09:10,260][INFO ][node ] [Desmond Pitt] initializing ...
[2016-11-15 11:09:10,303][INFO ][plugins ] [Desmond Pitt] loaded [], sites []
[2016-11-15 11:09:18,785][INFO ][node ] [Desmond Pitt] initialized
[2016-11-15 11:09:18,786][INFO ][node ] [Desmond Pitt] starting ...
[2016-11-15 11:09:19,241][INFO ][transport ] [Desmond Pitt] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/100.114.244.37:9300]}
[2016-11-15 11:09:19,338][INFO ][discovery ] [Desmond Pitt] elasticsearch/2-uc6WDiRwSTwsSXf8jg6A
[2016-11-15 11:09:23,185][INFO ][cluster.service ] [Desmond Pitt] new_master [Desmond Pitt][2-uc6WDiRwSTwsSXf8jg6A][bitnami-nginxstack-50be][inet[/100.114.244.37:9300]], reason: zen-disco-join (elected_as_master)
[2016-11-15 11:09:23,461][INFO ][http ] [Desmond Pitt] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/100.114.244.37:9200]}
[2016-11-15 11:09:23,462][INFO ][node ] [Desmond Pitt] started
[2016-11-15 11:09:25,327][INFO ][gateway ] [Desmond Pitt] recovered [6] indices into cluster_state
[2016-11-15 11:10:19,419][WARN ][cluster.service ] [Desmond Pitt] failed to reconnect to node [Desmond Pitt][2-uc6WDiRwSTwsSXf8jg6A][bitnami-nginxstack-50be][inet[/100.114.244.37:9300]]
org.elasticsearch.transport.ConnectTransportException: [Desmond Pitt][inet[/100.114.244.37:9300]] connect_timeout[30s]
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:807)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:741)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:714)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:150)
at org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:539)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: /100.114.244.37:9300
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
...
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
... 3 more