Hi, I'm evaluating using ElasticSearch and just trying to get it up
and running on my laptop (MacBook Pro running Snow Leopard) but keep
running into java.lang.OutOfMemoryError errors.
I don't have much free RAM on this machine, what's the best way to
configure ES to run in a low memory environment without dying?
I've set ES_MAX_MEM as 512m, and set the threadpool to "scaling" with
the following settings:
How do you index the data? If you push perform indexing in an async manner,
you should throttle it on the client side. In upcoming 0.10 I have improved
the blocking thread pool which is the one that you would want to use in this
case, but you might run into timeouts in this case.
-shay.banon
On Tue, Aug 24, 2010 at 7:04 PM, Richard Livsey livsey@gmail.com wrote:
Hi, I'm evaluating using Elasticsearch and just trying to get it up
and running on my laptop (MacBook Pro running Snow Leopard) but keep
running into java.lang.OutOfMemoryError errors.
I don't have much free RAM on this machine, what's the best way to
configure ES to run in a low memory environment without dying?
I've set ES_MAX_MEM as 512m, and set the threadpool to "scaling" with
the following settings:
At the moment it's just looping through and indexing via the REST API,
which isn't async as it waits for timeouts (as far as I can tell).
But makes sense that it might be overloading it, so I'll add some
throttling and see how that goes.
How do you index the data? If you push perform indexing in an async manner,
you should throttle it on the client side. In upcoming 0.10 I have improved
the blocking thread pool which is the one that you would want to use in this
case, but you might run into timeouts in this case.
-shay.banon
On Tue, Aug 24, 2010 at 7:04 PM, Richard Livsey livsey@gmail.com wrote:
Hi, I'm evaluating using Elasticsearch and just trying to get it up
and running on my laptop (MacBook Pro running Snow Leopard) but keep
running into java.lang.OutOfMemoryError errors.
I don't have much free RAM on this machine, what's the best way to
configure ES to run in a low memory environment without dying?
I've set ES_MAX_MEM as 512m, and set the threadpool to "scaling" with
the following settings:
You are looping with a single thread and waiting for each index request? It
should not require much memory in this case... . What do you use to index
the data (I have a test with a single thread that indexed 1M docs with 128mb
allocated and it works like a charm).
-shay.banon
On Tue, Aug 24, 2010 at 8:21 PM, Richard Livsey richard@livsey.org wrote:
At the moment it's just looping through and indexing via the REST API,
which isn't async as it waits for timeouts (as far as I can tell).
But makes sense that it might be overloading it, so I'll add some
throttling and see how that goes.
How do you index the data? If you push perform indexing in an async
manner,
you should throttle it on the client side. In upcoming 0.10 I have
improved
the blocking thread pool which is the one that you would want to use in
this
case, but you might run into timeouts in this case.
-shay.banon
On Tue, Aug 24, 2010 at 7:04 PM, Richard Livsey livsey@gmail.com
wrote:
Hi, I'm evaluating using Elasticsearch and just trying to get it up
and running on my laptop (MacBook Pro running Snow Leopard) but keep
running into java.lang.OutOfMemoryError errors.
I don't have much free RAM on this machine, what's the best way to
configure ES to run in a low memory environment without dying?
I've set ES_MAX_MEM as 512m, and set the threadpool to "scaling" with
the following settings:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.