Searching from the big index - Java heap space exception

Elastic Stack Elasticsearch
1

Hi,

I have a cluster of 3 machines - each 8 CPU and 30Gb ram.

Currently it holds the events stream from my logstash. Each day I have an
index of 40Gb.

But, currently, if I try to search that index, then one node fails with the
message:

Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:

java.lang.OutOfMemoryError: Java heap space

In my config I have the next:

indices.fielddata.cache.size: "10Gb"
indices.cache.filter.size: "10Gb"

Also I've executed the next query:

curl -XPUT http://host.com:9200/_cluster/settings -d '
{
"persistent" : {
"indices.fielddata.breaker.limit" : 40%
}
}'

However, I still met the heap error and the node stops to work.

Could someone explain, how should I organize my cluster, so it's able to
perform such queries.

Regards,

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0cdcc31e-ad64-4287-b505-055021e731a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

How much data do you have in your cluster?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 8 August 2014 05:43, Pavel P pavel@kredito.de wrote:

Hi,

I have a cluster of 3 machines - each 8 CPU and 30Gb ram.

Currently it holds the events stream from my logstash. Each day I have an
index of 40Gb.

But, currently, if I try to search that index, then one node fails with
the message:

Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:

java.lang.OutOfMemoryError: Java heap space

In my config I have the next:

indices.fielddata.cache.size: "10Gb"
indices.cache.filter.size: "10Gb"

Also I've executed the next query:

curl -XPUT http://host.com:9200/_cluster/settings -d '
{
"persistent" : {
"indices.fielddata.breaker.limit" : 40%
}
}'

However, I still met the heap error and the node stops to work.

Could someone explain, how should I organize my cluster, so it's able to
perform such queries.

Regards,

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0cdcc31e-ad64-4287-b505-055021e731a7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/0cdcc31e-ad64-4287-b505-055021e731a7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624b2mF03CUuLYyeRBeinMgo%2BAnT0f6%2B9Xxpcx%3D%2BpdHtYgw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Mark,

It seems that message was queued and appeared after I created the same one:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/elasticsearch/jbHg99gqRf0/ObA9PSNwZ28J

Could you go there, please?

We have 290Gb in the cluster currently.

On Fri, Aug 8, 2014 at 11:13 AM, Mark Walkom markw@campaignmonitor.com
wrote:

How much data do you have in your cluster?

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 8 August 2014 05:43, Pavel P pavel@kredito.de wrote:

Hi,

I have a cluster of 3 machines - each 8 CPU and 30Gb ram.

Currently it holds the events stream from my logstash. Each day I have an
index of 40Gb.

But, currently, if I try to search that index, then one node fails with
the message:

Caused by: org.elasticsearch.common.util.concurrent.ExecutionError:

java.lang.OutOfMemoryError: Java heap space

In my config I have the next:

indices.fielddata.cache.size: "10Gb"
indices.cache.filter.size: "10Gb"

Also I've executed the next query:

curl -XPUT http://host.com:9200/_cluster/settings -d '
{
"persistent" : {
"indices.fielddata.breaker.limit" : 40%
}
}'

However, I still met the heap error and the node stops to work.

Could someone explain, how should I organize my cluster, so it's able to
perform such queries.

Regards,

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0cdcc31e-ad64-4287-b505-055021e731a7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/0cdcc31e-ad64-4287-b505-055021e731a7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/zoHHPOpyzfU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEM624b2mF03CUuLYyeRBeinMgo%2BAnT0f6%2B9Xxpcx%3D%2BpdHtYgw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEM624b2mF03CUuLYyeRBeinMgo%2BAnT0f6%2B9Xxpcx%3D%2BpdHtYgw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Pavel Polyakov

Software Engineer - PHP team

E-mail: pavel@kredito.de
Skype: pavel.polyakov.x1

https://www.facebook.com/kreditech
Kreditech Holding SSL GmbH
Am Sandtorkai 50, 20457 Hamburg, Germany
Office phone: +49 (0)40 - 605905-60
Authorized representatives: Sebastian Diemer, Alexander Graubner-Müller
Company registration: Hamburg HRB122027

www.kreditech.com
Kreditech https://www.facebook.com/kreditech

https://www.facebook.com/kreditech

This e-mail contains confidential and/or legally protected information. If
you are not the intended recipient or if you have received this e-mail by
error please notify the sender immediately and destroy this e-mail. Any
unauthorized review, copying, disclosure or distribution of the material in
this e-mail is strictly forbidden. The contents of this e-mail is legally
binding only if it is confirmed by letter or fax. The sending of e-mails to
us does not have any period-protecting effect. Thank you for your
cooperation.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAFVUaqNjRobYANzBm8n5ZYqbAF-GxFhbayQyWbiOyz2gQcKJ-g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.