Time limit for ES node shutdown


#1

Working with ES 5.4
How can I know what is the maximum limit for ES master node to shutdown?
I have a 3 nodes cluster, 1 elected as master.
When I shut it down, a new master is elected.
In the meantime, I get
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/2/no master];
How can I control and estimate the maximum time until I won't get this ClusterBlockException?
My configuration is:


discovery.zen.commit_timeout: 2s
discovery.zen.publish_timeout: 2s
discovery.zen.fd.ping_timeout: 1s
transport.tcp.connect_timeout: 1s

I have a case where it took 5 seconds to find elect a master and I have a timeout of 3 in my code.


[2017-12-25T11:22:20,340][INFO ][o.e.c.s.ClusterService   ] [node-1] added {{node-2}{-oIenO1hQuyzGhl2p1Im8w}{0Pik8qyjQ5OLhXWpVnJWEQ}{172.16.65.117}{172.16.65.117:9300},}, reason: zen-disco-receive(from master [master {node-3}{P_mE77gsRUOAWADrVdwM-Q}{Ol3tznBkRR2Vi58kkz1KFw}{172.16.67.71}{172.16.67.71:9300} committed version [13]])
[2017-12-25T11:25:19,691][INFO ][o.e.d.z.ZenDiscovery     ] [node-1] master_left [{node-3}{P_mE77gsRUOAWADrVdwM-Q}{Ol3tznBkRR2Vi58kkz1KFw}{172.16.67.71}{172.16.67.71:9300}], reason [shut_down]
[2017-12-25T11:25:19,691][WARN ][o.e.d.z.ZenDiscovery     ] [node-1] master left (reason = shut_down), current nodes: nodes: 
   {node-1}{x1zCNZe1QYelGTGzveOjKQ}{uk1g0qIiTZ-RxXN8StPb6w}{172.16.65.114}{172.16.65.114:9300}, local
   {node-2}{-oIenO1hQuyzGhl2p1Im8w}{0Pik8qyjQ5OLhXWpVnJWEQ}{172.16.65.117}{172.16.65.117:9300}
   {node-3}{P_mE77gsRUOAWADrVdwM-Q}{Ol3tznBkRR2Vi58kkz1KFw}{172.16.67.71}{172.16.67.71:9300}, master

[2017-12-25T11:25:20,580][WARN ][o.e.c.NodeConnectionsService] [node-1] failed to connect to node {node-3}{P_mE77gsRUOAWADrVdwM-Q}{Ol3tznBkRR2Vi58kkz1KFw}{172.16.67.71}{172.16.67.71:9300} (tried [1] times)

[2017-12-25T11:25:25,717][INFO ][o.e.c.s.ClusterService   ] [node-1] detected_master {node-2}{-oIenO1hQuyzGhl2p1Im8w}{0Pik8qyjQ5OLhXWpVnJWEQ}{172.16.65.117}{172.16.65.117:9300}, reason: zen-disco-receive(from master [master {node-2}{-oIenO1hQuyzGhl2p1Im8w}{0Pik8qyjQ5OLhXWpVnJWEQ}{172.16.65.117}{172.16.65.117:9300} committed version [16]])
[2017-12-25T11:25:25,744][INFO ][o.e.c.s.ClusterService   ] [node-1] removed {{node-3}{P_mE77gsRUOAWADrVdwM-Q}{Ol3tznBkRR2Vi58kkz1KFw}{172.16.67.71}{172.16.67.71:9300},}, reason: zen-disco-receive(from master [master {node-2}{-oIenO1hQuyzGhl2p1Im8w}{0Pik8qyjQ5OLhXWpVnJWEQ}{172.16.65.117}{172.16.65.117:9300} committed version [17]])

(andy_zhou) #2

update the time 60s+
it networking error. or the machine is high CPU using or network is high..


#3

I cant wait a minute for a fail over...
How do I set the limit that I want to a fail over?


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.