Threadpool "caller" reject policy

Elastic Stack Elasticsearch
1

Right now my ES search thread pool config is:

"transient" : {
"threadpool.search.queue_size" : "200",
"threadpool.search.reject_policy" : "abort",
"threadpool.search.size" : "12"
}

The problem is that during peak load most of the search requests are
getting rejected. So I was thinking if I should switch to "caller" reject
policy. But I have some questions for "caller" policy:

  • Which thread would be the "caller" who would execute the searches when
    the queue is full. Is it the Netty connector?
  • If its the Netty connector what would happen to other clients who are
    making requests when the Netty "caller" thread is executing the search? In
    my local tests they are not failing/rejected. Where would such requests get
    queued (client/network)?
  • Would this guarantee every search request is eventually served (not
    rejected)?
  • What is the downside to using "caller", can this lead to network
    congestion or have any other issue with the ES node/cluster in case the
    there are too many concurrent searches for prolonged duration?

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5a8de617-7bb6-4d77-86b4-8744d4a7ce6d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Can someone help me out with this, please?

On Thursday, 6 February 2014 20:58:25 UTC+5:30, Vinod wrote:

Right now my ES search thread pool config is:

"transient" : {
"threadpool.search.queue_size" : "200",
"threadpool.search.reject_policy" : "abort",
"threadpool.search.size" : "12"
}

The problem is that during peak load most of the search requests are
getting rejected. So I was thinking if I should switch to "caller" reject
policy. But I have some questions for "caller" policy:

  • Which thread would be the "caller" who would execute the searches when
    the queue is full. Is it the Netty connector?
  • If its the Netty connector what would happen to other clients who are
    making requests when the Netty "caller" thread is executing the search? In
    my local tests they are not failing/rejected. Where would such requests get
    queued (client/network)?
  • Would this guarantee every search request is eventually served (not
    rejected)?
  • What is the downside to using "caller", can this lead to network
    congestion or have any other issue with the ES node/cluster in case the
    there are too many concurrent searches for prolonged duration?

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/94f65ed7-7c91-4768-ae70-7b9d118778a3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hey,

you can use the nodes statistics to find out which thread pool contains all
those rejected tasks, before trying to tune. If it is the search thread
pool, you can try to increase the size or the queue, alternatively add
another node to scale out your search load or try to improve your queries.

Wondering where you found the reject_policy setting (I think it has been
removed with 0.90.0).
Still before tuning, find out which threadpool has capacity issues.

The default setting of elasticsearch is to drop single requests in order to
retain performance for the remaining ones and keep the system stable
instead of overwhelming it with more and more requests.

Hope, that makes sense...

--Alex

On Mon, Feb 10, 2014 at 10:26 PM, Vinod rookie.vinod@gmail.com wrote:

Can someone help me out with this, please?

On Thursday, 6 February 2014 20:58:25 UTC+5:30, Vinod wrote:

Right now my ES search thread pool config is:

"transient" : {
"threadpool.search.queue_size" : "200",
"threadpool.search.reject_policy" : "abort",
"threadpool.search.size" : "12"
}

The problem is that during peak load most of the search requests are
getting rejected. So I was thinking if I should switch to "caller" reject
policy. But I have some questions for "caller" policy:

  • Which thread would be the "caller" who would execute the searches when
    the queue is full. Is it the Netty connector?
  • If its the Netty connector what would happen to other clients who are
    making requests when the Netty "caller" thread is executing the search? In
    my local tests they are not failing/rejected. Where would such requests get
    queued (client/network)?
  • Would this guarantee every search request is eventually served (not
    rejected)?
  • What is the downside to using "caller", can this lead to network
    congestion or have any other issue with the ES node/cluster in case the
    there are too many concurrent searches for prolonged duration?

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/94f65ed7-7c91-4768-ae70-7b9d118778a3%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-h%3D5XEb7_h8MFaHdfrcs%2BP9fT-ggLWLkBu9HyURGqNNw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.