We are using Elasticsearch for one of our applications. As a part of which
we indexed about 3M documents and have built two indices around them. We
have used a cluster of 2 Nodes each with 7.5 GB RAM and have dedicated 4 GM
to the ES.
What we are seeing is that on one of the nodes, the amount of HEAP using by
ES is more that 60% allocated even though the most obvious ones like
filter-cache, field-data cache etc are pretty low to almost zero. So I am
trying to understand who else could be consuming the memory from ES. Any
pointers on what else should I be looking at.
Here is the snapshot of the same from elasticHQ:
Cache ActivityField Size:0.00.0Field Evictions:00Filter Cache Size:24.0B
24.0BFilter Evictions:0 per query0 per queryID Cache Size:% ID Cache:0%0%
MemoryTotal Memory:7 gb7 gbHeap Size:4 gb4 gbHeap % of RAM:54.5%54.5%%
Heap Used:66.3%26%GC MarkSweep Frequency:0 s0 sGC MarkSweep Duration:0ms0msGC
ParNew Frequency:0 s0 sGC ParNew Duration:0ms0msG1 GC Young Generation Freq:0
s0 sG1 GC Young Generation Duration:0ms0msG1 GC Old Generation Freq:0 s0
Measuring heap usage in Java applications is very different than measuring
memory usage for other stuff.
Usually java allocates all the heap its going to need up front at
startup. At least, we do that in server applications.
Java's garbage collection is very lazy so heap usage will go up slowly
with time. If you zoom out it'll look like a saw tooth.
So its perfectly normal for on server to be using more heap than another
because it is at a different place in the saw tooth. Its interesting to
compare the depth of the valleys in the saw tooth and the time between
peaks. There are other interesting things you can look at to but one
snapshot of % heap used isn't one of them.
We are using Elasticsearch for one of our applications. As a part of which
we indexed about 3M documents and have built two indices around them. We
have used a cluster of 2 Nodes each with 7.5 GB RAM and have dedicated 4 GM
to the ES.
What we are seeing is that on one of the nodes, the amount of HEAP using
by ES is more that 60% allocated even though the most obvious ones like
filter-cache, field-data cache etc are pretty low to almost zero. So I am
trying to understand who else could be consuming the memory from ES. Any
pointers on what else should I be looking at.
Here is the snapshot of the same from elasticHQ:
Cache ActivityField Size:0.00.0Field Evictions:00Filter Cache Size:24.0B
24.0BFilter Evictions:0 per query0 per queryID Cache Size:% ID Cache:0%0%
MemoryTotal Memory:7 gb7 gbHeap Size:4 gb4 gbHeap % of RAM:54.5%54.5%%
Heap Used:66.3%26%GC MarkSweep Frequency:0 s0 sGC MarkSweep Duration:0ms
0msGC ParNew Frequency:0 s0 sGC ParNew Duration:0ms0msG1 GC Young
Generation Freq:0 s0 sG1 GC Young Generation Duration:0ms0msG1 GC Old
Generation Freq:0 s0
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.