Hey guys,
I have a few questions:
- I was wondering if anyone knows how to limit how much memory
elasticsearch uses while indexing.
During our loading jobs, typically we only see 60-70% heap usage before the
garbage collector
kicks in, but recently we saw some large traffic while loading and the heap
went to 100%.
After that everything became unresponsive each time the garbage collector
would run. As an example
it spent a full minute running to clear only 46.5mb. Eventually we had to
restart ES.
We use mostly default settings for ES. We only changed the cluster and
node names, and
changed the index and search threadpools to be of type cached. Is the
unlimited threadpool
for indexing to blame, or are we missing a setting somewhere? I should
mention we are currently
using v1.0.0 ES.
-
Also, at one point we had a cluster failure (still investigating the
cause), and after
rebalancing the speed was very slow, often resulting in timeouts. When the
cluster returned,
the speed went back to normal. Is this just an inevitable consequence of
losing a cluster,
or should speeds be comparable to before the failure after rebalance? -
My lead also wanted me to ask about RAM allocation. We read that you
should not allocate
more than 50% of the available RAM to ES because Lucene needs the rest for
filesystem caching.
Will this show up as used RAM in top? He said he didn't see the rest of
the RAM being used.
Thanks in advance guys. We are new to ElasticSearch, and we thought we
would ask some advice
before crashing the site again.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8bcf59bd-c3c1-4eeb-ae33-53d06c33f685%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.