Understanding actual JVM usage for ES process

Elastic Stack Elasticsearch
1

Hi all,

We were testing out elastic search and the appropriate heap space to provide it with. The box we are running it on currently is quite small. There are a couple of questions I had about haw the JVM memory usage is reported by top on Linux

We started off by providing ES 384M of heap space (both Xmx and Xms values). Upon inserting a few indices and data into the cluster, we noticed that it is using a lot more space than what we have provided to the heap. We understand that there is stack, metaspace etc. that gets counted here but the actual usage seems excessive.

What we want to know is, does the Lucene file system cache also get reported as part of this number? If not does anybody know of a way to analyze what the exact memory layout of the JVM is (not jhat/jmap which allow us to analyze the heap only) or a way to figure out the Lucene cache size.

Here is the output of the top command for reference:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
27653 abcuser   20   0 2459m 675m  23m S 36.6 67.8   8:58.32 java    <----   ES process
 4959 abcuser   30  10 2236m  88m 4044 S  0.0  8.9   1:58.32 java
 4832 defuser   30  10 1975m  49m 2980 S  0.3  5.0   5:54.86 java
 3968 abcuser   20   0  113m  12m 2364 S  0.0  1.3   0:09.27 abc-linux
 3969 abcuser   20   0  124m  10m 2368 S  0.0  1.0   0:09.92 abc-linux
32097 root      20   0  115m 6740 5672 S  0.0  0.7   0:00.00 sshd
2

Lucene relies on the disk cache provided by the OS to be fast. There are other caches in Lucene and ES but I believe they are on heap. Essentially Lucene mmaps the files and accesses them like any other process would an mmap-ed file, relying on the OS to page in the portions that are needed when they are needed.

Beyond that I don't have much to say other than that 384MB is fairly low. Our integration testing is done on 1GB and 512MB heaps. Certain things, mostly aggregations at this point, need a fairly large amount of heap. We are always looking to push things out of the heap and on to unmanaged memory when we can. the top output you linked looks like I'd expect it to look but I don't have the tools to dig more. jstat will get you some numbers but I haven't ever tried digging around in the metaspace.

Ooh! Another thing! I edited your post to replace ` with ``` to make the code block more readable.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.