Hi:
I had this issue , a cluster node shutdown alone without any interaction of
people, is that possible ??
Server load ok.
Server disk ok.
Server are in amazon.
master:Comet Man
cat elasticsearch.yml|grep -v "#" |sed '/^$/d'
script.disable_dynamic: true
Secondary no master: Robin
[root@yyyy logs]# cat ../config/elasticsearch.yml |grep -v "#"
node.name: Robin
network.publish_host: yyyy.yyyyy.com
discovery.zen.ping_timeout: 60s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["xxxx.xxxxxl.com"]
node.master: false
node.data: true
Server master:Comet Man
INFO | jvm 1 | 2014/02/19 02:09:41 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/26 12:37:28 | <-- Wrapper Stopped ( i do not
do stop manually, no log of other people logged at this time)
STATUS | wrapper | 2014/05/26 12:55:04 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/26 12:55:04 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/26 12:55:04 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/26 12:55:04 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/26 12:55:04 |
STATUS | wrapper | 2014/05/26 12:55:04 | Launching a JVM...
INFO | jvm 1 | 2014/05/26 12:55:05 | WrapperManager: Initializing...
[2014-05-26 08:30:05,803][INFO ][cluster.service ] [Comet Man]
removed
{[Robin][hpP_TldFTtOvBcNccY5-rw][inet[/10.248.13.222:9300]]{master=false},},
reason:
zen-disco-node_left([Robin][hpP_TldFTtOvBcNccY5-rw][inet[/10.248.13.222:9300]]{master=false})
[2014-05-26 08:33:18,985][INFO ][cluster.service ] [Comet Man]
added
{[Robin][r8a4bx1AT3WzmoOYRo0dIA][inet[/10.248.13.222:9300]]{master=false},},
reason: zen-disco-receive(join from
node[[Robin][r8a4bx1AT3WzmoOYRo0dIA][inet[/xxxx:xxx]]{master=false}])
[2014-05-26 12:37:24,280][INFO ][action.admin.cluster.node.shutdown]
[Comet Man] [cluster_shutdown]: requested, shutting down in [1s]
[2014-05-26 12:37:25,311][INFO ][action.admin.cluster.node.shutdown] [Comet
Man] [cluster_shutdown]: done shutting down all nodes except master,
proceeding to master
[2014-05-26 12:37:25,312][INFO ][action.admin.cluster.node.shutdown] [Comet
Man] shutting down in [200ms]
[2014-05-26 12:37:25,514][INFO ][action.admin.cluster.node.shutdown] [Comet
Man] initiating requested shutdown (using service)
[2014-05-26 12:37:27,406][INFO ][node ] [Comet Man]
{0.20.3}[15380]: stopping ...
[2014-05-26 12:37:27,805][INFO ][node ] [Comet Man]
{0.20.3}[15380]: stopped
[2014-05-26 12:37:27,806][INFO ][node ] [Comet Man]
{0.20.3}[15380]: closing ...
[2014-05-26 12:37:27,889][INFO ][node ] [Comet Man]
{0.20.3}[15380]: closed
[2014-05-26 12:55:06,811][INFO ][node ] [Tinkerer]
{0.20.3}[4342]: initializing ... ( i did the start manually)
[2014-05-26 12:55:06,821][INFO ][plugins ] [Tinkerer]
loaded [], sites []
[2014-05-26 12:55:09,847][INFO ][node ] [Tinkerer]
{0.20.3}[4342]: initialized
Secondary Robin:
I do not know why secondary server was rebooted many time in the past by
self.
STATUS | wrapper | 2014/05/22 08:32:11 | Launching a JVM...
INFO | jvm 1 | 2014/05/22 08:32:12 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/23 08:30:04 | TERM trapped. Shutting down.
STATUS | wrapper | 2014/05/23 08:30:08 | <-- Wrapper Stopped
STATUS | wrapper | 2014/05/23 08:32:11 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/23 08:32:11 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/23 08:32:11 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/23 08:32:11 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/23 08:32:11 |
STATUS | wrapper | 2014/05/23 08:32:11 | Launching a JVM...
INFO | jvm 1 | 2014/05/23 08:32:12 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/24 08:30:04 | TERM trapped. Shutting down.
STATUS | wrapper | 2014/05/24 08:30:07 | <-- Wrapper Stopped
STATUS | wrapper | 2014/05/24 08:32:11 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/24 08:32:11 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/24 08:32:11 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/24 08:32:11 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/24 08:32:11 |
STATUS | wrapper | 2014/05/24 08:32:11 | Launching a JVM...
INFO | jvm 1 | 2014/05/24 08:32:13 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/25 08:30:03 | TERM trapped. Shutting down.
STATUS | wrapper | 2014/05/25 08:30:06 | <-- Wrapper Stopped
STATUS | wrapper | 2014/05/25 08:32:09 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/25 08:32:09 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/25 08:32:09 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/25 08:32:09 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/25 08:32:09 |
STATUS | wrapper | 2014/05/25 08:32:09 | Launching a JVM...
INFO | jvm 1 | 2014/05/25 08:32:10 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/26 08:30:04 | TERM trapped. Shutting down.
STATUS | wrapper | 2014/05/26 08:30:06 | <-- Wrapper Stopped
STATUS | wrapper | 2014/05/26 08:32:09 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/26 08:32:09 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/26 08:32:09 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/26 08:32:09 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/26 08:32:09 |
STATUS | wrapper | 2014/05/26 08:32:09 | Launching a JVM...
INFO | jvm 1 | 2014/05/26 08:32:10 | WrapperManager: Initializing...
STATUS | wrapper | 2014/05/26 12:37:29 | <-- Wrapper Stopped
STATUS | wrapper | 2014/05/26 12:55:57 | --> Wrapper Started as Daemon
STATUS | wrapper | 2014/05/26 12:55:57 | Java Service Wrapper Community
Edition 64-bit 3.5.14
STATUS | wrapper | 2014/05/26 12:55:57 | Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2014/05/26 12:55:57 |
http://wrapper.tanukisoftware.com
STATUS | wrapper | 2014/05/26 12:55:57 |
STATUS | wrapper | 2014/05/26 12:55:57 | Launching a JVM...
INFO | jvm 1 | 2014/05/26 12:55:58 | WrapperManager: Initializing...
[2014-05-26 12:37:25,299][INFO ][action.admin.cluster.node.shutdown]
[Robin] shutting down in [200ms]
[2014-05-26 12:37:25,509][INFO ][action.admin.cluster.node.shutdown]
[Robin] initiating requested shutdown (using service) ( why service ¨)
[2014-05-26 12:37:27,294][INFO ][node ] [Robin]
{0.20.3}[15194]: stopping ...
[2014-05-26 12:37:27,902][TRACE][discovery.zen.fd ] [Robin]
[master] [[Comet Man][nsM2LW5rQCOCAUCLgNHjTw][inet[/xxx:xxx]]] transport
disconnected (with verified connect)
[2014-05-26 12:37:27,903][DEBUG][discovery.zen.fd ] [Robin]
[master] stopping fault detection against master [[Comet
Man][nsM2LW5rQCOCAUCLgNHjTw][inet[/10.248.44.76:9300]]], reason [master
failure, transport disconnected (with verified connect)]
[2014-05-26 12:37:27,973][INFO ][discovery.zen ] [Robin]
master_left [[Comet Man][nsM2LW5rQCOCAUCLgNHjTw][inet[/xxx.xx:xx]]], reason
[transport disconnected (with verified connect)]
[2014-05-26 12:37:27,985][WARN ][discovery.zen ] [Robin]
master_left and no other node elected to become master, current nodes:
{[Robin][r8a4bx1AT3WzmoOYRo0dIA][inet[app-es2.55social.com/10.248.13.222:9300]]{master=false},}
[2014-05-26 12:37:27,986][INFO ][cluster.service ] [Robin] removed
{[Comet Man][nsM2LW5rQCOCAUCLgNHjTw][inet[/xxxx:xxx]],}, reason:
zen-disco-master_failed ([Comet
Man][nsM2LW5rQCOCAUCLgNHjTw][inet[/10.248.44.76:9300]])
[2014-05-26 12:37:27,990][TRACE][discovery.zen.ping.unicast] [Robin] [2]
connecting (light) to [#zen_unicast_1#][inet[xxx.xxxx/xxx:xxx]]
[2014-05-26 12:37:28,023][TRACE][discovery.zen.ping.unicast] [Robin] [2]
failed to connect to [#zen_unicast_1#][inet[xxxx.xxx/xxx:xxx]]
org.elasticsearch.transport.ConnectTransportException:
[][inet[app-es1.55social.com/xxx:xxx]] connect_timeout[30s]
at
org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:638)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:600)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:569)
at
org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:131)
at
org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:273)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:148)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:104)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
... 3 more
[2014-05-26 12:37:28,720][INFO ][node ] [Robin]
{0.20.3}[15194]: stopped
[2014-05-26 12:37:28,721][INFO ][node ] [Robin]
{0.20.3}[15194]: closing ...
[2014-05-26 12:37:28,806][INFO ][node ] [Robin]
{0.20.3}[15194]: closed
is this behavior possible ?? do i need to make another configuration ?
the cluster is up around and year without any error.. version is 0.20.x
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f8370df9-4aea-482c-b290-6c09fe2eba3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.