How to improve facet search performance?

imdhmd · June 20, 2013, 5:49am

We are doing performance testing, to identify the right settings for facet
search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1 shard
on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

radu_gheorghe · June 20, 2013, 8:52am

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests (eg:
with our SPM http://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed imdhmd@gmail.com wrote:

We are doing performance testing, to identify the right settings for facet
search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

imdhmd · June 21, 2013, 6:40am

Looks like SPM registration is having a downtime. I'm unable to
register: https://apps.sematext.com/users-web/register.do

On Thursday, June 20, 2013 2:22:45 PM UTC+5:30, Radu Gheorghe wrote:

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests (eg:
with our SPMhttp://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed <imd...@gmail.com<javascript:>

wrote:

We are doing performance testing, to identify the right settings for
facet search.
The following are the data points. Can you help us identify settings that
can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=true
-Des.config=/usr/local/etc/elasticsearch/elasticsearch.yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings may
be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

radu_gheorghe · June 21, 2013, 6:44am

Hi,

It works for me now, can you try again?

On Fri, Jun 21, 2013 at 9:40 AM, Imdad Ahmed imdhmd@gmail.com wrote:

Looks like SPM registration is having a downtime. I'm unable to register:
https://apps.sematext.com/users-web/register.do

On Thursday, June 20, 2013 2:22:45 PM UTC+5:30, Radu Gheorghe wrote:

Hello,

What's the bottleneck? CPU, IO, ...?

It's a good idea to monitor your cluster while you do your load tests
(eg: with our SPMhttp://sematext.com/spm/elasticsearch-performance-monitoring/),
so you know what's limiting your performance.

On Thu, Jun 20, 2013 at 8:49 AM, Imdad Ahmed imd...@gmail.com wrote:

We are doing performance testing, to identify the right settings for
facet search.
The following are the data points. Can you help us identify settings
that can improve the search response time?

Context:

Query:
{
"facets": {
"tag": {
"terms": {
"field": "category"
}
}
},
"fields": [
"title"
]
}

So this is basically a facet match all query, chosen to do load testing.
Note that the field "category" is keyword analyzed

Data:
Data setup was done using wikipedia river. Index size is 170GB

Setup:
3 nodes, each with 2 cores and 8GB RAM, 1 Zookeeper node, 3 shards (1
shard on each node), no replica

Some existing cluster level settings:
threadpool for search : fixed, 4 threads, 10000 queue size
mlockall : true

JAVA_OPTS from elasticsearch-env.sh
ES_JAVA_OPTS="
-server
-Djava.net.preferIPv4Stack=**true
-Des.config=/usr/local/etc/**elasticsearch/elasticsearch.**yml
-Xms2299m
-Xmx2299m
-Xss256k
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+**UseCMSInitiatingOccupancyOnly
-XX:+**HeapDumpOnOutOfMemoryError"

Current performance:
Load is: 45 concurrent requests over 2 minutes duration
Mean response time is: 23.2 seconds

Can you kindly suggest what other what other elasticsearch settings
may be tweaked to improve facet search performance.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
How to improve performance of facet queries? Elasticsearch	7	1283	July 6, 2017
Elasticsearch slow query response time using terms facet Elasticsearch	3	544	July 6, 2017
Elasticsearch slow query response time using terms facet Elasticsearch	1	424	July 6, 2017
Facets performance issue Elasticsearch	4	334	July 6, 2017
Slow Search Help Elasticsearch	3	848	July 6, 2017

How to improve facet search performance?

Context:

Context:

Context:

Context:

Related topics