ES_DIRECT_SIZE will limit the amount of direct memory, which is the one
that's allocated outside heap. So if you set ES_HEAP_SIZE to 3g and
ES_DIRECT_SIZE to 2g, for example, you can expect ES to use up to 5GB.
Also, if you're worried about running out of memory, the best way to
confirm/throw away your worries is by running a performance test: throw
more requests at a test ES than you normally do in production and see what
happens. And I wouldn't trust on top for giving accurate memory usage. I
would trust "out of memory" exceptions in ES log, or slow indexing/search
On Sun, May 5, 2013 at 5:50 PM, sub firstname.lastname@example.org wrote:
You can limit the size of direct memory ES uses by setting
ES_DIRECT_SIZE from elasticsearch.in.sh (/etc/default/elasticsearch if
you installed from the DEB package, /etc/sysconfig/elasticsearh if you
installed from the RPM package).
I've set ES_DIRECT_SIZE = Heap Size = 3072M
Are you using in-memory indices? If yes, then it's normal to see an
increased amount of direct memory being used as they grow.
I haven't configured index storage in particular so I should be using the
default 'filesystem' storage.
If not, I'm not sure what other stuff ES uses direct memory for. Maybe
someone else can shed some light. I would assume it's for buffers, if you
use the default NIO FS as a storehttp://www.elasticsearch.org/guide/reference/index-modules/store/for your indices on disk. You can use mmapfs instead, by setting
"index.store.type" to "mmapfs" - which is recommended for 64-bit machines
anyway - in your configuration or in the index settings.
Set "index.store.type: mmapfs".
ES still continues to use up more memory. A new batch of test import is in
progress and 'top' reports
4.2G usage when the ES_HEAP_SIZE was set only to 3072M. Bigdesk again
reports Committed= 2.0G and
'Used' varies from 300M to 2.4G which is very well within the
ES_HEAP_SIZE. No errors in logs also.
I shall open a new thread in this regard.
http://sematext.com/ -- ElasticSearch -- Solr -- Lucene
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
For more options, visit https://groups.google.com/groups/opt_out.