Hello,
The search threadpool size (that is, how many requests can be actually
worked on at once) defaults to 3 times the number of processors. This might
be reduced in future, though, see:
The queue size (how many requests ES can accept before starting to reject
them) defaults to 1000.
From what I understand, this is per thread, so the answer to your question
depends on how many processors you have and how many shards get hit by each
search. For example, if a search runs on 3 indices, each with 2 shards
(number of replicas is irrelevant, because the search will only hit one
complete set of data) you'll get 6 requests in the threadpool per search.
If you have two servers with 8 cores each, you 832=48 threads available.
So the cluster can work on 8 requests at once. On top of that it can still
queue round-down(2 nodes * 1000 queue size/6 requests per search)=333
searches until it starts rejecting them.
Regarding your scaling question, I can't give you a direct answer,
unfortunately, because it depends on a whole lot of variables, mainly how
your data, queries and hardware look like and what can be changed. The fact
that your threadpool queue got full is just a symptom, it's not clear to me
what happens in there. I usually see this when there are lots of indices
and/or those indices have lots of shards. So a single request takes a lot
of requests in the threadpool, filling it up, even if the ES cluster can
keep up with the load. If that's your case increase the threadpool queue
size and make sure you don't have too many shards per index.
If your cluster can't keep up with the load (a monitoring tool like SPM
http://sematext.com/spm/ should show you that), then the first step is to
see where is the bottleneck. Again, monitoring can give some insight: are
queries too expensive, can they be optimized? do you have too many cache
evictions? is the heap size too large or too small? is memory, I/O or CPU
the bottleneck? Things like that. It could also be that you need
more/different hardware.
Finally, you can make scaling ES someone else's problem by using a hosted
service like Logsene http://sematext.com/logsene/index.html. Especially
if you're using ES for log- or metric-like data, you'll get lots of
features out of the box, and we expose most of the ES API to plug in your
custom stuff.
Best regards,
Radu
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Thu, Jan 8, 2015 at 1:32 PM, vipins sumitharjunan@gmail.com wrote:
What is the maximum limit on the concurrent search requests with default
Elastic search server settings.I am able to perform only 5 parallel search requests in my application with
default settings.how can we improve the scalability of ES server search requests apart from
increasing number of node,shards and queue size in search thread pool.thanks.
--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/concurrent-search-request-to-elasticsearch-tp4068702.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1420716748150-4068702.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHXA0_2ZoQ6bewJ_aSxFMxAO0_5Mdsu%3D9WA4m_be7AfSb%3D2%2BTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.