Performance getting even worse after optimization

Elastic Stack Elasticsearch
1

Hi,

I just did an _optimize operation on a cluster (10 data nodes, roughly
350,000,000 docs in total). This a cluster only has one index.

However, the performance gets even worse: the response time doubled or even
tripled.

Any hint on this?

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

How big is the index, how many shards and replicas?
What ES version? What java version?

On 8 January 2015 at 20:40, Xiaoting Ye xye@spokeo.com wrote:

Hi,

I just did an _optimize operation on a cluster (10 data nodes, roughly
350,000,000 docs in total). This a cluster only has one index.

However, the performance gets even worse: the response time doubled or
even tripled.

Any hint on this?

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B_qPdBA0S8JeXmhM2e013-YxQb5roZAJvEh-r1rxfQQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

The index is of 149G, 19 shards with 1 replica.

The es version is 1.4.1, and the java version is 1.7.0_71.

I have specific routing strategy and the query used in testing only goes to
one shard:
heap.percent ram.percent load
51 58 0.33

(when it is under continues query, just one query at a time)

This specific shard has 22502484 docs, 10GB in size.

Thanks!

On Thu, Jan 8, 2015 at 2:10 AM, Mark Walkom markwalkom@gmail.com wrote:

How big is the index, how many shards and replicas?
What ES version? What java version?

On 8 January 2015 at 20:40, Xiaoting Ye xye@spokeo.com wrote:

Hi,

I just did an _optimize operation on a cluster (10 data nodes, roughly
350,000,000 docs in total). This a cluster only has one index.

However, the performance gets even worse: the response time doubled or
even tripled.

Any hint on this?

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/W49B4d9MWNk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B_qPdBA0S8JeXmhM2e013-YxQb5roZAJvEh-r1rxfQQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B_qPdBA0S8JeXmhM2e013-YxQb5roZAJvEh-r1rxfQQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGVN9k7xZTBpchkOSqp%3DyLxqSKozo1WhVUveq%2BRaLDHmd81kpA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi Xiaoting Ye,

Do you find a solution for your problem?

Best,

On Thursday, January 8, 2015 at 9:39:16 PM UTC+3:30, Xiaoting Ye wrote:

The index is of 149G, 19 shards with 1 replica.

The es version is 1.4.1, and the java version is 1.7.0_71.

I have specific routing strategy and the query used in testing only goes
to one shard:
heap.percent ram.percent load
51 58 0.33

(when it is under continues query, just one query at a time)

This specific shard has 22502484 docs, 10GB in size.

Thanks!

On Thu, Jan 8, 2015 at 2:10 AM, Mark Walkom <markw...@gmail.com
<javascript:>> wrote:

How big is the index, how many shards and replicas?
What ES version? What java version?

On 8 January 2015 at 20:40, Xiaoting Ye <x...@spokeo.com <javascript:>>
wrote:

Hi,

I just did an _optimize operation on a cluster (10 data nodes, roughly
350,000,000 docs in total). This a cluster only has one index.

However, the performance gets even worse: the response time doubled or
even tripled.

Any hint on this?

Thanks!

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d9f9ba25-4a7f-4fba-978c-8368d74bc349%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/W49B4d9MWNk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B_qPdBA0S8JeXmhM2e013-YxQb5roZAJvEh-r1rxfQQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B_qPdBA0S8JeXmhM2e013-YxQb5roZAJvEh-r1rxfQQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/542cdeb0-2f76-4995-9343-cd64ac62b1a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.