Understanding HEAP usage

Elastic Stack Elasticsearch
1

Hi,

We are using Elasticsearch for one of our applications. As a part of which
we indexed about 3M documents and have built two indices around them. We
have used a cluster of 2 Nodes each with 7.5 GB RAM and have dedicated 4 GM
to the ES.

What we are seeing is that on one of the nodes, the amount of HEAP using by
ES is more that 60% allocated even though the most obvious ones like
filter-cache, field-data cache etc are pretty low to almost zero. So I am
trying to understand who else could be consuming the memory from ES. Any
pointers on what else should I be looking at.

Here is the snapshot of the same from elasticHQ:

Cache ActivityField Size:0.00.0Field Evictions:00Filter Cache Size:24.0B
24.0BFilter Evictions:0 per query0 per queryID Cache Size:% ID Cache:0%0%
MemoryTotal Memory:7 gb7 gbHeap Size:4 gb4 gbHeap % of RAM:54.5%54.5%%
Heap Used:66.3%26%GC MarkSweep Frequency:0 s0 sGC MarkSweep Duration:0ms0msGC
ParNew Frequency:0 s0 sGC ParNew Duration:0ms0msG1 GC Young Generation Freq:0
s0 sG1 GC Young Generation Duration:0ms0msG1 GC Old Generation Freq:0 s0

Thanks,
Karthik

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAD7PaHwAGgVj%3DL4hyJCJh--Z6pj5P7%3DVpCp-Mbf9MGyuvzOC-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

2

Measuring heap usage in Java applications is very different than measuring
memory usage for other stuff.

  1. Usually java allocates all the heap its going to need up front at
    startup. At least, we do that in server applications.
  2. Java's garbage collection is very lazy so heap usage will go up slowly
    with time. If you zoom out it'll look like a saw tooth.

So its perfectly normal for on server to be using more heap than another
because it is at a different place in the saw tooth. Its interesting to
compare the depth of the valleys in the saw tooth and the time between
peaks. There are other interesting things you can look at to but one
snapshot of % heap used isn't one of them.

Nik

On Fri, Oct 17, 2014 at 6:27 AM, karthik jayanthi <
karthikjayanthi.iitm@gmail.com> wrote:

Hi,

We are using Elasticsearch for one of our applications. As a part of which
we indexed about 3M documents and have built two indices around them. We
have used a cluster of 2 Nodes each with 7.5 GB RAM and have dedicated 4 GM
to the ES.

What we are seeing is that on one of the nodes, the amount of HEAP using
by ES is more that 60% allocated even though the most obvious ones like
filter-cache, field-data cache etc are pretty low to almost zero. So I am
trying to understand who else could be consuming the memory from ES. Any
pointers on what else should I be looking at.

Here is the snapshot of the same from elasticHQ:

Cache ActivityField Size:0.00.0Field Evictions:00Filter Cache Size:24.0B
24.0BFilter Evictions:0 per query0 per queryID Cache Size:% ID Cache:0%0%
MemoryTotal Memory:7 gb7 gbHeap Size:4 gb4 gbHeap % of RAM:54.5%54.5%%
Heap Used:66.3%26%GC MarkSweep Frequency:0 s0 sGC MarkSweep Duration:0ms
0msGC ParNew Frequency:0 s0 sGC ParNew Duration:0ms0msG1 GC Young
Generation Freq:0 s0 sG1 GC Young Generation Duration:0ms0msG1 GC Old
Generation Freq:0 s0

Thanks,
Karthik

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAD7PaHwAGgVj%3DL4hyJCJh--Z6pj5P7%3DVpCp-Mbf9MGyuvzOC-w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAD7PaHwAGgVj%3DL4hyJCJh--Z6pj5P7%3DVpCp-Mbf9MGyuvzOC-w%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0XKZfgcWkkHO5KsKrHVDz6j84CJovFkiJxOvin3Y8ZGg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi,

Any pointers on what could be happening in here ?

Thanks,
Karthik

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAD7PaHzzNFcxji3xM4iE9tosV3hmYbEkJFkG251%3DpUibwMtVqg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.