Search performance - Scaling options Horizontally vs Vertically

I have observed that elastic search defaults the search thread pool to 3 X
#of CPUs and even if you increase this to a fix # it does not really help
as the threads start sharing the CPU cycles.

Does this mean that to get same performance results for more concurrent searches
I either have to scale vertically by adding more CPU cores or horizontally
by increasing the nodes and replica's to eliviate the search times by shard
to replica shards?

Here is my situation:
I have a 6 node ES cluster with 6 shards storing a total of 60 million
documents (around 2kb each) and index size of around 32 GB. Each node is a
VM with 4 vCPUs and 8 GB allocated to ES cluster. When I have 15 concurrent users
the response is around 270 ms and I see that all 12 threads in the search
pool are busy. If I increase the number of concurrent users for the search
the response time keeps going higher and higher and there is more requests
pending in the search queue. I even increased the thread pool configuration
but it really did not help.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93823925-fcfb-4b6d-b4b6-42332242924c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Can anyone answer this?

ES does scale better horizontally.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 24 September 2014 08:24, pavan.bkv pavan.bkv@gmail.com wrote:

Can anyone answer this?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Search-performance-Scaling-options-Horizontally-vs-Vertically-tp4063612p4063922.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1411511076978-4063922.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZUfM50CMk2XBiGpWLRY9bBH7xoN16nrQiH8gC5TbQHtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Mark. Appreciate the response.

Is there a template which certifies the TPS and # of users in relation to performance and scaling that I could refer to. May be some bench marks or common practices? I understand that it varies by index types and nature of searches. But wanted to see if there are general guidelines for scaling and certification on TPS/number of concurrent users.

Warm Regards,
Pavan

Scaling horizontally (after you have tuned the existing nodes correctly) is
one of the design principles of ElasticSearch, yes.

You can actually increase the search throughput, even if you do not want to
increase disk requirements or increase the number of replicas, by adding
new ElasticSearch nodes or processes (maybe even on the same node?) with
"node.data = false", so they only serve as search nodes?

If that is not what you want, either asking someone to tune your current
nodes, or adding new nodes is the way forward

Anne

On Wed, Sep 24, 2014 at 12:24 AM, pavan.bkv pavan.bkv@gmail.com wrote:

Can anyone answer this?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Search-performance-Scaling-options-Horizontally-vs-Vertically-tp4063612p4063922.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1411511076978-4063922.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
Anne Veling
BeyondTrees.com
+31 6 50 969 170
@anneveling

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAwWRiJj4JUmYUXLpYz3afyqaY%3Dy0BYpvqr8cnenN6RxbfbjoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

There are a few blog posts, but nothing consolidated.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 25 September 2014 07:58, pavan.bkv pavan.bkv@gmail.com wrote:

Thanks Mark. Appreciate the response.

Is there a template which certifies the TPS and # of users in relation to
performance and scaling that I could refer to. May be some bench marks or
common practices? I understand that it varies by index types and nature of
searches. But wanted to see if there are general guidelines for scaling and
certification on TPS/number of concurrent users.

Warm Regards,
Pavan

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Search-performance-Scaling-options-Horizontally-vs-Vertically-tp4063612p4063985.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1411595912790-4063985.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZYpuy6DNcvtPd7aSe%3D29QGRx1vo_281CSUYH0CG%2BXrzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.