Running out of memory indexing on linux


(Vince Wadhwani) #1

I recently switched from a mac to using debian linux. Same data set,
indexing method, and elastic version. On OS X I can index my records (about
1 million) but on linux I can see memory usage increasing over time until I
finally run out and the process is killed. I've got 8GB of RAM and 16GB of
Swap but I still can't get through the records I could with half that RAM
on OSX.

Is there a config option somewhere I can toggle? (I've already tried
enabling bootstrap.mlockall: true) Or is there something else going on?

-Vince

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e0268477-5cc2-49a9-ad95-f90960884578%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alberto Paro-2) #2

Are you using the SUN jvm 1.7 or above? I had similar issues with standard Linux openJDk.

Hi,
Alberto Paro

Inviato da iPad

Il giorno 15/mag/2014, alle ore 17:28, Vince Wadhwani vince71@gmail.com ha scritto:

I recently switched from a mac to using debian linux. Same data set, indexing method, and elastic version. On OS X I can index my records (about 1 million) but on linux I can see memory usage increasing over time until I finally run out and the process is killed. I've got 8GB of RAM and 16GB of Swap but I still can't get through the records I could with half that RAM on OSX.

Is there a config option somewhere I can toggle? (I've already tried enabling bootstrap.mlockall: true) Or is there something else going on?

-Vince

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e0268477-5cc2-49a9-ad95-f90960884578%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/AA2390DC-7AD0-4D50-8B3F-73000459F999%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Vince Wadhwani) #3

On Friday, May 16, 2014 3:43:22 AM UTC-4, Alberto Paro wrote:

Are you using the SUN jvm 1.7 or above? I had similar issues with standard
Linux openJDk

Yes, I am. I've got 1.7.0_55 via Debian Sid. How did you solve your issue?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1d768b6b-b333-4d3a-bb3e-591536f91dc7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4