JVm goes out of memory when I am using facets

Elastic Stack Elasticsearch
1

I am using ES version 0.19.7 . It was running fine but now jvm goes out of
memory when I use facets.

2

Hello!

You probably don't have enough memory to handle your faceting
requests. If you have enough memory, you can try increasing it, if you
don't and you can sacrifice a bit of performance you can try changing
the field data cache type and see if it works for you. You can find
more about ElasticSearch cache here:
http://www.elasticsearch.org/guide/reference/index-modules/cache.html
and a post explaining a bit

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

I am using ES version 0.19.7 . It was running fine but now jvm goes out of memory when I use facets.

3

Yes. It could happen.

Do you need a more precise answer?
Please provide more details about your settings (memory you give to the Jvm), number of docs, number of different terms for the field you want to facet.

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek jajoria.abhishek@gmail.com a écrit :

I am using ES version 0.19.7 . It was running fine but now jvm goes out of memory when I use facets.

4

I have an index of size 4 gb with 5000 documents and 0 replica .My min heap
size is 256m and max heap size of 1g reserved for ES. I want to calculate
term facet for all the terms in the docs .Is there any way to do it without
going out of memory .

On Sun, Jul 29, 2012 at 3:12 PM, David Pilato david@pilato.fr wrote:

Yes. It could happen.

Do you need a more precise answer?
Please provide more details about your settings (memory you give to the
Jvm), number of docs, number of different terms for the field you want to
facet.

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek jajoria.abhishek@gmail.com a
écrit :

I am using ES version 0.19.7 . It was running fine but now jvm goes out
of memory when I use facets.

5

Hello!

Try increasing you maximum heap size. ElasticSearch throws OOM because it doesn't have enough memory to calculate facets.

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

I have an index of size 4 gb with 5000 documents and 0 replica .My min heap size is 256m and max heap size of 1g reserved for ES. I want to calculate term facet for all the terms in the docs .Is there any way to do it without going out of memory .

On Sun, Jul 29, 2012 at 3:12 PM, David Pilato <david@pilato.fr> wrote:

Yes. It could happen.

Do you need a more precise answer?

Please provide more details about your settings (memory you give to the Jvm), number of docs, number of different terms for the field you want to facet.

David :wink:

Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek <jajoria.abhishek@gmail.com> a écrit :

> I am using ES version 0.19.7 . It was running fine but now jvm goes out of memory when I use facets.

6

Yes. And, you have 4 gb with only 5000 documents. That means that one document have many many many fields or values.

Asking for facets on such huge fields requires lot of memory.

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] De la part de Rafal Kuc
Envoyé : dimanche 29 juillet 2012 20:24
À : elasticsearch@googlegroups.com
Objet : Re: JVm goes out of memory when I am using facets

Hello!

Try increasing you maximum heap size. Elasticsearch throws OOM because it doesn't have enough memory to calculate facets.

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ http://sematext.com/ :: Solr - Lucene - Nutch - Elasticsearch

I have an index of size 4 gb with 5000 documents and 0 replica .My min heap size is 256m and max heap size of 1g reserved for ES. I want to calculate term facet for all the terms in the docs .Is there any way to do it without going out of memory .

On Sun, Jul 29, 2012 at 3:12 PM, David Pilato < mailto:david@pilato.fr david@pilato.fr> wrote:
Yes. It could happen.

Do you need a more precise answer?
Please provide more details about your settings (memory you give to the Jvm), number of docs, number of different terms for the field you want to facet.

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek < mailto:jajoria.abhishek@gmail.com jajoria.abhishek@gmail.com> a écrit :

I am using ES version 0.19.7 . It was running fine but now jvm goes out of memory when I use facets.

7

David.How much would be enough or any other way to do it .

On Sun, Jul 29, 2012 at 11:57 PM, David Pilato david@pilato.fr wrote:

Yes. And, you have 4 gb with only 5000 documents. That means that one
document have many many many fields or values.****

Asking for facets on such huge fields requires lot of memory.****

De : elasticsearch@googlegroups.com [mailto:
elasticsearch@googlegroups.com] De la part de Rafal Kuc
Envoyé : dimanche 29 juillet 2012 20:24
À : elasticsearch@googlegroups.com
Objet : Re: JVm goes out of memory when I am using facets****

Hello!

Try increasing you maximum heap size. Elasticsearch throws OOM because it
doesn't have enough memory to calculate facets.

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch****

I have an index of size 4 gb with 5000 documents and 0 replica .My min
heap size is 256m and max heap size of 1g reserved for ES. I want to
calculate term facet for all the terms in the docs .Is there any way to do
it without going out of memory .

On Sun, Jul 29, 2012 at 3:12 PM, David Pilato david@pilato.fr wrote:
Yes. It could happen.

Do you need a more precise answer?
Please provide more details about your settings (memory you give to the
Jvm), number of docs, number of different terms for the field you want to
facet.

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek jajoria.abhishek@gmail.com a
écrit :

I am using ES version 0.19.7 . It was running fine but now jvm goes out
of memory when I use facets.****

8

Hard to say without any idea of your documents and of your use case.
I follow some rules:

  • I don't facet on fields having to much different values (let say 100 000)
  • I filter facet on a smaller period (using range filter)
  • I increase RAM

Try with at least 2Gb...

HTH
David

--

Le 30 juil. 2012 à 07:17, jajoria abhishek jajoria.abhishek@gmail.com a écrit :

David.How much would be enough or any other way to do it .

On Sun, Jul 29, 2012 at 11:57 PM, David Pilato david@pilato.fr wrote:
Yes. And, you have 4 gb with only 5000 documents. That means that one document have many many many fields or values.

Asking for facets on such huge fields requires lot of memory.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] De la part de Rafal Kuc
Envoyé : dimanche 29 juillet 2012 20:24
À : elasticsearch@googlegroups.com
Objet : Re: JVm goes out of memory when I am using facets

Hello!

Try increasing you maximum heap size. Elasticsearch throws OOM because it doesn't have enough memory to calculate facets.

--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Elasticsearch

I have an index of size 4 gb with 5000 documents and 0 replica .My min heap size is 256m and max heap size of 1g reserved for ES. I want to calculate term facet for all the terms in the docs .Is there any way to do it without going out of memory .

On Sun, Jul 29, 2012 at 3:12 PM, David Pilato david@pilato.fr wrote:
Yes. It could happen.

Do you need a more precise answer?
Please provide more details about your settings (memory you give to the Jvm), number of docs, number of different terms for the field you want to facet.

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 29 juil. 2012 à 11:11, jajoria abhishek jajoria.abhishek@gmail.com a écrit :

I am using ES version 0.19.7 . It was running fine but now jvm goes out of memory when I use facets.