# Tune memory usage? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 2, 2014, 2:04pm UTC](https://discuss.elastic.co/t/tune-memory-usage/20051 "2014-10-02T14:04:00Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Mathias\_Adler](https://avatars.discourse-cdn.com/v4/letter/m/f05b48/32.png) [@Mathias\_Adler](https://discuss.elastic.co/u/Mathias_Adler)\ **Post date:** [October 2, 2014, 2:04pm UTC](https://discuss.elastic.co/t/tune-memory-usage/20051/1 "2014-10-02T14:04:00Z")
Hi All, I'm quite new to ES and using it as part of the ELK stack. I'm running a two node cluster where each node got 24core and 64GB RAM (32 allocated to Java), We have an index rate @ ~3000/s and total document count per 24h @ 200miljon (and storing 10days). When mem usage gets close to 80% I start having search problems in Kibana and get all kinds of exception and all gets better when mem usage gets lower again. So, one way is of course to scale out, but my question is, what mem tuning can be done and will it make a big difference? 1, Drop unused data fields in logstash already, will that make any difference? 2, Reduce index.refresh\_interval, will that reduce mem usage? 3, Set replicas to zero and get all primary shards spread over both nodes, will that impact mem usage? What else can be done to lower mem usage? Anyone out there having the same type of load or higher (document count and index rate), how does your set up look like? Br Mathias -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 2, 2014, 9:51pm UTC](https://discuss.elastic.co/t/tune-memory-usage/20051/2 "2014-10-02T21:51:45Z")
You should drop your heap to 31GB, over that you don't compress java pointers and GC will suffer. 1. That will help as it reduces document size. 2. Definitely, if you don't need near-realtime access then open it up, we run at 60sec but could probably go to 2 or more minutes. 3. This could be risky, doubly so given you are only running two nodes. To elaborate on point 3, and on general note, you should really run 3 or more nodes incrementing up to odd numbers (3,5,7,9....). The reason for this is that is that it helps to prevent split brain across your cluster. You should use disable bloom filtering on your indexes as it will give you a bit of a boost, Elasticsearch curator can handle that for you. However you are probably reaching the limits of your cluster, 2 billion docs is a fair bit of data. What versions of ES and java are you on? What amount of data (GB) is it? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 3 October 2014 00:04, Mathias Adler [mathias.adler@rebtel.com](mailto:mathias.adler@rebtel.com) wrote: > Hi All, > I'm quite new to ES and using it as part of the ELK stack. I'm running a > two node cluster where each node got 24core and 64GB RAM (32 allocated to > Java), We have an index rate @ ~3000/s and total document count per 24h @ > 200miljon (and storing 10days). > When mem usage gets close to 80% I start having search problems in Kibana > and get all kinds of exception and all gets better when mem usage gets > lower again. > So, one way is of course to scale out, but my question is, what mem tuning > can be done and will it make a big difference? > 1, Drop unused data fields in logstash already, will that make any > difference? > 2, Reduce index.refresh\_interval, will that reduce mem usage? > 3, Set replicas to zero and get all primary shards spread over both nodes, > will that impact mem usage? > What else can be done to lower mem usage? > > Anyone out there having the same type of load or higher (document count > and index rate), how does your set up look like? > > Br > Mathias > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624bs6A2z7pxvxeN7YYxkp%2BDph851nLKejjG6WC9jEasWcw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624bs6A2z7pxvxeN7YYxkp%2BDph851nLKejjG6WC9jEasWcw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mathias\_Adler](https://avatars.discourse-cdn.com/v4/letter/m/f05b48/32.png) [@Mathias\_Adler](https://discuss.elastic.co/u/Mathias_Adler)\ **Post date:** [October 3, 2014, 9:04pm UTC](https://discuss.elastic.co/t/tune-memory-usage/20051/3 "2014-10-03T21:04:29Z")
Thanks for your reply, see comment in line... Den torsdagen den 2:e oktober 2014 kl. 23:52:13 UTC+2 skrev Mark Walkom: > You should drop your heap to 31GB, over that you don't compress java > pointers and GC will suffer. _Ok, I thought the limit was 32? I'll drop to 31 then._ > 1. That will help as it reduces document size. > > \*I'll investigate what fields can be removed and then drop them. \* > 1. Definitely, if you don't need near-realtime access then open it up, > we run at 60sec but could probably go to 2 or more minutes. > > _Will reduce to 10s to see if it makes a difference_. > 1. This could be risky, doubly so given you are only running two nodes. > > To elaborate on point 3, and on general note, you should really run 3 or > more nodes incrementing up to odd numbers (3,5,7,9....). The reason for > this is that is that it helps to prevent split brain across your cluster. _I actually have three nodes for reason stated above but only two data nodes, third node is slower HW._ You should use disable bloom filtering on your indexes as it will give you > a bit of a boost, Elasticsearch curator can handle that for you. _Interesting, haven't seen this option before, thanks!_ > However you are probably reaching the limits of your cluster, 2 billion > docs is a fair bit of data. What versions of ES and java are you on? What > amount of data (GB) is it? _I'm running ES 1.3.2 and Java 1.7.0\_65 and collect ~70GB per 24h._ _I'll test the suggestions above but will add two more nodes anyway as we have more logs to through in._ _Thanks for your input, I'll post the result when done!_ Br Mathias Adler > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 3 October 2014 00:04, Mathias Adler \<[mathia...@rebtel.com](mailto:mathia...@rebtel.com) \ > > > wrote: > > > Hi All, > > I'm quite new to ES and using it as part of the ELK stack. I'm running a > > two node cluster where each node got 24core and 64GB RAM (32 allocated to > > Java), We have an index rate @ ~3000/s and total document count per 24h @ > > 200miljon (and storing 10days). > > When mem usage gets close to 80% I start having search problems in Kibana > > and get all kinds of exception and all gets better when mem usage gets > > lower again. > > So, one way is of course to scale out, but my question is, what mem > > tuning can be done and will it make a big difference? > > 1, Drop unused data fields in logstash already, will that make any > > difference? > > 2, Reduce index.refresh\_interval, will that reduce mem usage? > > 3, Set replicas to zero and get all primary shards spread over both > > nodes, will that impact mem usage? > > What else can be done to lower mem usage? > > > > Anyone out there having the same type of load or higher (document count > > and index rate), how does your set up look like? > > > > Br > > Mathias > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/2ba446f8-6017-4863-b462-a075b60780f2%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b68e68d5-93cc-4350-8f17-de137010fb4e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b68e68d5-93cc-4350-8f17-de137010fb4e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:58am UTC](https://discuss.elastic.co/t/tune-memory-usage/20051/4 "2017-07-06T00:58:20Z")