How to improve facet search performance?

Elastic Stack Elasticsearch
1

We are doing performance testing, to identify the right settings for facet
search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1 shard
on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests (eg:
with our SPM http://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed imdhmd@gmail.com wrote:

We are doing performance testing, to identify the right settings for facet
search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Looks like SPM registration is having a downtime. I'm unable to
register: https://apps.sematext.com/users-web/register.do

On Thursday, June 20, 2013 2:22:45 PM UTC+5:30, Radu Gheorghe wrote:

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests (eg:
with our SPMhttp://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed <imd...@gmail.com<javascript:>

wrote:

We are doing performance testing, to identify the right settings for
facet search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi,

It works for me now, can you try again?

On Fri, Jun 21, 2013 at 9:40 AM, Imdad Ahmed imdhmd@gmail.com wrote:

Looks like SPM registration is having a downtime. I'm unable to register:
https://apps.sematext.com/users-web/register.do

On Thursday, June 20, 2013 2:22:45 PM UTC+5:30, Radu Gheorghe wrote:

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests
(eg: with our SPMhttp://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed imd...@gmail.com wrote:

We are doing performance testing, to identify the right settings for
facet search.
The following are the data points. Can you help us identify settings
that can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=**true
-Des.config=/usr/local/etc/**elasticsearch/elasticsearch.**yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+**UseCMSInitiatingOccupancyOnly
-XX:+**HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings
may be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.