# Memory management for elasticsearch

**URL:** <https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702>\
**Category:** Elasticsearch\
**Created:** [August 7, 2021, 10:44am UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702 "2021-08-07T10:44:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![prat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prat/32/79978_2.png) [@prat](https://discuss.elastic.co/u/prat)\
**Post date:** [August 7, 2021, 10:44am UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/1 "2021-08-07T10:44:59Z")

</div>

Hi Team,

I am trying to calculate good balance of total memory in `three` node es cluster.

If I have three node e.s cluster each with `32G` memory, `8` vcpu. Which combination would be more suitable for balancing memory between all the components? I know there will be no fixed answers but just trying to get as accurate as I can.

different elasticsearch components will be used are `beats` (filebeat, metricbeat,heartbeat), `logstash`, `elasticsearch`, `kibana`.

most use case for this cluster will be, application logs getting indexed and running query on them like fetch average response time for 7 days,30 days, how many are different status codes for last 24 hrs, 7 days etc through curl calls, so aggregation will be used and other use case is monitoring, seeing logs through kibana but no ML jobs or dashboard creation etc..

After going through below official docs, its recommended to set heap size as below,

`logstash` -

> **[JVM settings | Logstash Reference \[7.14\] | Elastic](https://www.elastic.co/guide/en/logstash/current/jvm-settings.html#heap-size)**

The recommended heap size for typical ingestion scenarios should be no less than `4GB` and no more than `8GB`.

`elasticsearch` -

> **[Advanced configuration | Elasticsearch Guide \[7.14\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#set-jvm-heap-size)**

Set `Xms` and `Xmx` to no more than `50%` of your total memory. Elasticsearch requires memory for purposes other than the JVM heap

`Kibana` -

I have't found default or recommended memory for kibana but in our test cluster of single node of `8G` memory it is taking `1.4G` as total (`256 MB/1.4 GB`)

`beats` -

not found what is the default or recommended memory for beats but they will also consume more or less.

What should the ideal combination from below?

1. `32G` = 16G for OS + 16G for Elasticsearch heap.   
for logstash 4G from 16G of OS, say three beats will consume 4G, kibana 2G   
this leaves OS with 6G and if any new component has to be install in future like say APM or any other OS related then they all will have only 6G with OS.

Above is, per official recommendation for all components. (i.e 50% for OS and 50% for es)

1. `32G` = 8G for elasticsearch heap. (25% for elasticsearch)   
4G for logstash + beats 4G + kibana 2G   
this leaves 14G for OS and for any future component.

If we install `elasticsearch` on all three nodes, and `logstash` and `kibana` on two nodes then the remaining node will have less memory consumption but question still remains for the first two nodes that will have all these components.

I am missing to cover something that can change this memory combination ?

Any suggestion by changing in above combination or any new combination is appreciated.

Thanks,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 8, 2021, 11:11pm UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/2 "2021-08-08T23:11:22Z")

</div>

It depends.

Beat shouldn't need GB, Kibana and Logstash a few, and Elasticsearch can be up to 50% of available. I would just start with 8GB for Elasticsearch and then let the rest use what it needs.

---

<div class="post-metadata">

**Author:** ![prat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prat/32/79978_2.png) [@prat](https://discuss.elastic.co/u/prat)\
**Post date:** [August 9, 2021, 3:29pm UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/3 "2021-08-09T15:29:42Z")

</div>

@warkolm, Thanks for reply. Don't you think as per your reply on 50% for elasticsearch, it should be 16G instead of 8G from start?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 9, 2021, 10:10pm UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/4 "2021-08-09T22:10:01Z")

</div>

Given you're running multiple other processes on the same host, and unless you are immediately looking at large volumes of ingestion, no.

---

<div class="post-metadata">

**Author:** ![prat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prat/32/79978_2.png) [@prat](https://discuss.elastic.co/u/prat)\
**Post date:** [August 10, 2021, 6:32am UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/5 "2021-08-10T06:32:50Z")

</div>

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 7, 2021, 6:33am UTC](https://discuss.elastic.co/t/memory-management-for-elasticsearch/280702/6 "2021-09-07T06:33:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
