Out of space Exception Java Heap


(Kartavya) #1

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal


(David Pilato) #2

Same answer as for your last post.
What is your config in term of memory, shards, nodes ?

How "big" are your documents (number of fields, nested documents, ???)

Le 11 octobre 2011 à 08:17, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal
--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Kartavya) #3

Thanks David for your reply

I haven't define any configuration for memory, shards, nodes explicitly. I
am using default configuration.
My document size is around 20 fields and no nested document.

Index created as follows

curl -XPUT 'localhost:9200/_river/pa_index2/_meta' -d '{
"type": "couchdb",
"couchdb" : {
"host" : "localhost",
"port": 5984,
"db" : "p_tweet_db"
},
"index" : {
"index" : "pa_index2",
"type" : "pa_index_feed2",
"bulk_size" : "100",
"bulk_timeout" : "10ms"}
}'

Regards,
Pulkit Agrawal

On Tue, Oct 11, 2011 at 2:49 PM, david@pilato.fr david@pilato.fr wrote:

**

Same answer as for your last post.

What is your config in term of memory, shards, nodes ?

How "big" are your documents (number of fields, nested documents, ???)

Le 11 octobre 2011 à 08:17, Pulkit Agrawal pulkitdotcom@gmail.com a
écrit :

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(David Pilato) #4

So you can't ask ES to make miracles :wink:

Define your memory settings to provide at least 4gb for your node and see if you have better results.

If you are developping your software based on ES, it's not perhaps a good idea to work with 3 million docs.
If you are testing your production platform, add more nodes and memory.

HTH
David :wink:

Le 11 oct. 2011 à 11:34, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :

Thanks David for your reply

I haven't define any configuration for memory, shards, nodes explicitly. I am using default configuration.
My document size is around 20 fields and no nested document.

Index created as follows

curl -XPUT 'localhost:9200/_river/pa_index2/_meta' -d '{
"type": "couchdb",
"couchdb" : {
"host" : "localhost",
"port": 5984,
"db" : "p_tweet_db"
},
"index" : {
"index" : "pa_index2",
"type" : "pa_index_feed2",
"bulk_size" : "100",
"bulk_timeout" : "10ms"}
}'

Regards,
Pulkit Agrawal

On Tue, Oct 11, 2011 at 2:49 PM, david@pilato.fr david@pilato.fr wrote:
Same answer as for your last post.
What is your config in term of memory, shards, nodes ?

How "big" are your documents (number of fields, nested documents, ???)

Le 11 octobre 2011 à 08:17, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Kartavya) #5

I am running like

elasticsearch -f -Xms5g -Xmx5g

But still having the same problem.

Regards,
Pulkit Agrawal

On Tue, Oct 11, 2011 at 3:26 PM, David Pilato david@pilato.fr wrote:

So you can't ask ES to make miracles :wink:

Define your memory settings to provide at least 4gb for your node and see
if you have better results.

If you are developping your software based on ES, it's not perhaps a good
idea to work with 3 million docs.
If you are testing your production platform, add more nodes and memory.

HTH
David :wink:

Le 11 oct. 2011 à 11:34, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :

Thanks David for your reply

I haven't define any configuration for memory, shards, nodes explicitly. I
am using default configuration.
My document size is around 20 fields and no nested document.

Index created as follows

curl -XPUT 'localhost:9200/_river/pa_index2/_meta' -d '{
"type": "couchdb",
"couchdb" : {
"host" : "localhost",
"port": 5984,
"db" : "p_tweet_db"
},
"index" : {
"index" : "pa_index2",
"type" : "pa_index_feed2",
"bulk_size" : "100",
"bulk_timeout" : "10ms"}
}'

Regards,
Pulkit Agrawal

On Tue, Oct 11, 2011 at 2:49 PM, david@pilato.frdavid@pilato.fr <david@pilato.fr
david@pilato.fr> wrote:

**

Same answer as for your last post.

What is your config in term of memory, shards, nodes ?

How "big" are your documents (number of fields, nested documents, ???)

Le 11 octobre 2011 à 08:17, Pulkit Agrawal < pulkitdotcom@gmail.com
pulkitdotcom@gmail.com> a écrit :

Hi

My index is around 6GB in size when I run few queries. I am getting Java

heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal

--
David Pilato
http://dev.david.pilato.fr/http://dev.david.pilato.fr/
Twitter : @dadoonet


(David Pilato) #6

That’s strange. Do you use the latest ES version ?

I suggest to use Lukas’s Bigdesk [1] to get some information while you
perform facetted searches.

David.

[1] https://github.com/lukas-vlcek/bigdesk

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Pulkit Agrawal
Envoyé : mardi 11 octobre 2011 12:03
À : elasticsearch@googlegroups.com
Objet : Re: Out of space Exception Java Heap

I am running like

elasticsearch -f -Xms5g -Xmx5g

But still having the same problem.

Regards,

Pulkit Agrawal

On Tue, Oct 11, 2011 at 3:26 PM, David Pilato david@pilato.fr wrote:

So you can't ask ES to make miracles :wink:

Define your memory settings to provide at least 4gb for your node and see if
you have better results.

If you are developping your software based on ES, it's not perhaps a good
idea to work with 3 million docs.

If you are testing your production platform, add more nodes and memory.

HTH
David :wink:

Le 11 oct. 2011 à 11:34, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :

Thanks David for your reply

I haven't define any configuration for memory, shards, nodes explicitly. I
am using default configuration.

My document size is around 20 fields and no nested document.

Index created as follows

curl -XPUT 'localhost:9200/_river/pa_index2/_meta' -d '{

"type": "couchdb",

"couchdb" : {

"host" : "localhost",

"port": 5984,

"db" : "p_tweet_db"

},

"index" : {

"index" : "pa_index2",

"type" : "pa_index_feed2",

"bulk_size" : "100",

"bulk_timeout" : "10ms"}

}'

Regards,

Pulkit Agrawal

On Tue, Oct 11, 2011 at 2:49 PM, david@pilato.fr david@pilato.fr wrote:

Same answer as for your last post.

What is your config in term of memory, shards, nodes ?

How "big" are your documents (number of fields, nested documents, ???)

Le 11 octobre 2011 à 08:17, Pulkit Agrawal pulkitdotcom@gmail.com a écrit
:

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Kartavya) #7

I am using ES 0.17.7

Ok, I will check BigDesk and update here.

Thanks

Regards,
Pulkit Agrawal

On Wed, Oct 12, 2011 at 12:14 AM, David Pilato david@pilato.fr wrote:

That’s strange. Do you use the latest ES version ?****

I suggest to use Lukas’s Bigdesk [1] to get some information while you
perform facetted searches.****


David.****


[1] https://github.com/lukas-vlcek/bigdesk****



De : elasticsearch@googlegroups.com [mailto:
elasticsearch@googlegroups.com] De la part de Pulkit Agrawal
Envoyé : mardi 11 octobre 2011 12:03
À : elasticsearch@googlegroups.com
Objet : Re: Out of space Exception Java Heap****


I am running like ****


elasticsearch -f -Xms5g -Xmx5g****


But still having the same problem.****


Regards,****

Pulkit Agrawal****

On Tue, Oct 11, 2011 at 3:26 PM, David Pilato david@pilato.fr wrote:****

So you can't ask ES to make miracles ;-)****


Define your memory settings to provide at least 4gb for your node and see
if you have better results.****


If you are developping your software based on ES, it's not perhaps a good
idea to work with 3 million docs.****

If you are testing your production platform, add more nodes and memory.***
*

HTH
David ;-)****

Le 11 oct. 2011 à 11:34, Pulkit Agrawal pulkitdotcom@gmail.com a écrit :


Thanks David for your reply****


I haven't define any configuration for memory, shards, nodes explicitly. I
am using default configuration.****

My document size is around 20 fields and no nested document.****


Index created as follows****


curl -XPUT 'localhost:9200/_river/pa_index2/_meta' -d '{****

"type": "couchdb",****

"couchdb" : {****

"host" : "localhost",****

"port": 5984,****

"db" : "p_tweet_db"****

},****

"index" : {****

"index" : "pa_index2",****

"type" : "pa_index_feed2",****

"bulk_size" : "100",****

"bulk_timeout" : "10ms"}****

}'****


Regards,****

Pulkit Agrawal****


On Tue, Oct 11, 2011 at 2:49 PM, david@pilato.fr david@pilato.fr wrote:*


Same answer as for your last post. ****

What is your config in term of memory, shards, nodes ? ****


How "big" are your documents (number of fields, nested documents, ???) **
**



Le 11 octobre 2011 à 08:17, Pulkit Agrawal pulkitdotcom@gmail.com a
écrit : ****

Hi

My index is around 6GB in size when I run few queries. I am getting Java
heap out of space error.
I know that somewhere I have made some mistake.

Please help to find me the error.

Thanks.

Regards,
Pulkit Agrawal ****

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet ****




(system) #8