Stop routing search when the node is GC


(Thomas Decaux) #1

We run a 10 data nodes cluster, 64Gb memory, 30 Gb heap, 8 cores, with elasticsearch 2.4. At least 500 searchs/seconds.

Every 3 days, we see errors because the search threads pool is full, after some investigations, the problem appears when the node is doing a GC, at 70% of heap size => 10%. It took some seconds but this is enough to cause trouble.

Is it possible to tell to the client node to not sending shard search to a node when the node is doing a GC ? (or when the heap is at X % so the node is going to GC)


(Mark Walkom) #2

No it's not, a client has no idea of that. Even if it knew a node's heap use, it doesn't know if it's in GC or will go into GC for the query it sends.


(Thomas Decaux) #3

So the balance is totally random?

I am thinking to a new component such as "routing strategy" , so we could implement different balance strategies.


(Mark Walkom) #4

What balance do you mean?


(Christian Dahlqvist) #5

I wonder if adaptive replica selection would help here? It would however require that you upgrade...

As far as I know there is no way to control this in earlier versions.


(Thomas Decaux) #6

what I called "balance" is how the client node performs the shard search (if I understood how search is working ^^ APP --> CLIENT(index) --> DATA(shards))


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.