Clean shutdown of an Elasticsearch node


(Dan Everton) #1

I've read the documentation for the node shutdown API
(http://www.elasticsearch.org/guide/reference/api/admin-cluster-nodes-shutdown.html)
and it's not clear to me what happens to inflight requests when a shutdown
is requested? Ideally the node would reject new requests (or direct them to
the other cluster members), allow existing requests to finish, and then
shutdown.

There's a delay parameter there which might help, but it still doesn't say
if new requests are blocked.

Cheers,
Dan


(Shay Banon) #2

The shutdown will not wait for current requests to finish. If they are
coming from other nodes in the cluster, then it will properly fail over to
other nodes, if its from a "client" directly, then a socket closed will
probably be thrown.

On Fri, Jun 8, 2012 at 7:07 AM, Dan Everton dan@iocaine.org wrote:

I've read the documentation for the node shutdown API (
http://www.elasticsearch.org/guide/reference/api/admin-cluster-nodes-shutdown.html)
and it's not clear to me what happens to inflight requests when a shutdown
is requested? Ideally the node would reject new requests (or direct them to
the other cluster members), allow existing requests to finish, and then
shutdown.

There's a delay parameter there which might help, but it still doesn't say
if new requests are blocked.

Cheers,
Dan


(Dan Everton) #3

On Monday, June 11, 2012 7:48:21 AM UTC+10, kimchy wrote:

The shutdown will not wait for current requests to finish. If they are
coming from other nodes in the cluster, then it will properly fail over to
other nodes, if its from a "client" directly, then a socket closed will
probably be thrown.

Hrm, we're using the HTTP interface to talk to the Elasticsearch cluster
and clients do get 'socket closed'. We could use the Node Client but we're
rather stick to HTTP. I assume the Transport Client suffers the same
problem as using HTTP or does it have built-in failover?

Support for clean shutdown would be really handy. Is it something that you
have on the road map?


(Shay Banon) #4

Both the node client and transport client have automatic failover, why not
build it into the HTTP client you have, should be simple to do (And you
need to handle it anyhow).

On Mon, Jun 11, 2012 at 11:55 PM, Dan Everton dan@iocaine.org wrote:

On Monday, June 11, 2012 7:48:21 AM UTC+10, kimchy wrote:

The shutdown will not wait for current requests to finish. If they are
coming from other nodes in the cluster, then it will properly fail over to
other nodes, if its from a "client" directly, then a socket closed will
probably be thrown.

Hrm, we're using the HTTP interface to talk to the Elasticsearch cluster
and clients do get 'socket closed'. We could use the Node Client but we're
rather stick to HTTP. I assume the Transport Client suffers the same
problem as using HTTP or does it have built-in failover?

Support for clean shutdown would be really handy. Is it something that you
have on the road map?


(Dan Everton) #5

On Thursday, June 14, 2012 4:31:05 AM UTC+10, kimchy wrote:

Both the node client and transport client have automatic failover, why not
build it into the HTTP client you have, should be simple to do (And you
need to handle it anyhow).

We have the Elasticsearch cluster behind a load balancer for the HTTP
interface. I guess we could change the load balancer settings to do the
fail over and retry. Thought it's starting to look like we'd be better off
just using the Transport Client :slight_smile: We went with HTTP because it was fast
enough and we wanted to be consistent with our choice of protocols for
remote calls, i.e. HTTP and JSON all the time.

What's the best practice for deploying Elasticsearch but using the HTTP
interface? Or is everyone using the Node/Transport Client?


(system) #6