# Memory usage for ELK-style systems

**URL:** https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570
**Category:** Elasticsearch
**Created:** [June 29, 2015, 8:13pm UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570 "2015-06-29T20:13:57Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![rrva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rrva/32/43888_2.png) [@rrva](https://discuss.elastic.co/u/rrva)
#### Post date: [June 29, 2015, 8:13pm UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570/1 "2015-06-29T20:13:57Z")

</div>

Hi,

I'm using Elasticsearch with Kibana as a searchable/aggregatable event log, so a kind of ELK installation although no logstash. Recently we made the jump to Elasticsearch 0.20 all the way up to 1.5 and added Kibana. Things have been good. The memory profile less so for smaller machines. These machines also share memory with a cassandra installation, and both are memory-hungry.

Questions:

1. Newer Elasticsearch uses mmapfs for some parts. How does this affect memory usage? Are there alternatives to this setting which use less memory if we can sacrifice search performance?
2. How does index.number\_of\_replicas affect memory usage?

Changes we are evaluating, please comment on these:

a. ES\_HEAP\_SIZE tuning. It's at 25% of physical RAM right now to accomodate the neighbouring cassandra process which also is set to have a max heap of 25%. I think that is reasonable but we have to measure more.  
b. Modifying term\_index settings to: index.term\_index\_interval: 256 and index.term\_index\_divisor: 5  
c. We have set indices.fielddata.cache.size to 40%, considering to lower it to 20%

Thanks!

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 29, 2015, 9:53pm UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570/2 "2015-06-29T21:53:40Z")

</div>

The more replicas you have the more memory you will need.

However given you are running two memory hungry apps on the same server, you are playing with fire and we don't recommend doing this, for obvious reasons.

---

<div class="post-metadata">

### Author: ![rrva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rrva/32/43888_2.png) [@rrva](https://discuss.elastic.co/u/rrva)
#### Post date: [June 30, 2015, 1:46pm UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570/3 "2015-06-30T13:46:58Z")

</div>

> [@warkolm](#):
>
> The more replicas you have the more memory you will need.
> 
> However given you are running two memory hungry apps on the same server, you are playing with fire and we don't recommend doing this, for obvious reasons.

Thanks. We are aware of the issues with not having dedicated elasticsearch machines and agree.

The main point was that lowering replica count should lower memory requirements.

Regarding mmapfs, my understanding is that mmaped files does just change VIRT memory reported by 'top' which is anyhow tricky to interpret for a java process.

Lastly, indicies.fielddata.cache.size and index.termin\_index\_interval we will have to benchmark using our data and usage. Is there any HTTP API to get memory statistics from elasticsearch or do you only use JVM tools for that?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 30, 2015, 9:11pm UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570/4 "2015-06-30T21:11:54Z")

</div>

There's heaps of monitoring APIs.

I'd start with [https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:04am UTC](https://discuss.elastic.co/t/memory-usage-for-elk-style-systems/24570/5 "2017-07-06T00:04:27Z")

</div>


