Elasticsearch is taking more memory than allocated heap when using bootstrap.mlockall: true

Elastic Stack Elasticsearch
1

Hi,

I’ve allocated max and min heap same with ES_HEAP_SIZE=1200m in
elasticsearch and using *bootstrap.mlockall: true *as suggested by
elasticsearch so that process memory won’t get swapped.

But when elasticsearch I start its taking more memory than max heap
mentioned; like 1.4g and holding the same memory. The usage is not getting
fluctuating now, which is a good thing.

Can you please explain why its taking more memory for JVM than allocated.

--Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/98cfaf5a-4559-4b7e-80ac-816626690e49%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

A Java JVM process takes more memory than just the heap. There are shared
libraries / classes, stacks, buffers, etc. that add to the heap memory.

If you want to analyze the process memory, you can examine the process map
on Linux with pmap -x

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGsmFzwn%2BCmHY_eqhUKy%3DpEP9gv8No1%3DCdUje%2BZGAYi%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi,

Actually if I fix the ES_MAX_MEM with some value elasticsearch is starting
with max of that memory usage only. But when used *bootstrap.mlockall: true
*and start elasticsearch it's taking more memory than ES_MAX_MEM.

What I'm trying to understand if how much memory elasticsearch will occupy
for process memory pages so that I can plan for the RAM usage.

-Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d3f91133-8867-4d1f-a552-dfad0ab6c642%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

There is no connection between ES_MAX_MEM and bootstrap.mlockall. mlockall
can not make the JVM taking more memory, the memory consumption is the
same. Maybe you set mmapfs and you have seen the difference between virtual
memory and resident memory.

The recommended setting is to reserve 50% of RAM for the heap of the JVM
ES. The other 50% are for the Java process, buffers, and other OS
processes. This is a rule of thumb, it depends on how you want to use the
heap of course.

JVM memory consumption can change a lot over the lifetime of a process. You
should set ES_MIN_MEM to ES_MAX_MEM in every case. This prevents the JVM
from allocating and releasing large chunks of memory every once a while,
together with high GC activity.

The best you can do to get precise numbers is to start your ES cluster for
testing, put it under heavy load according to your workload pattern, and
measure the system metrics. Trust only the numbers you see on your system,
nothing else.

Jörg

On Tue, Mar 11, 2014 at 10:06 AM, Subhadip Bagui i.bagui@gmail.com wrote:

Hi,

Actually if I fix the ES_MAX_MEM with some value elasticsearch is starting
with max of that memory usage only. But when used *bootstrap.mlockall:
true *and start elasticsearch it's taking more memory than ES_MAX_MEM.

What I'm trying to understand if how much memory elasticsearch will occupy
for process memory pages so that I can plan for the RAM usage.

-Subhadip

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d3f91133-8867-4d1f-a552-dfad0ab6c642%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/d3f91133-8867-4d1f-a552-dfad0ab6c642%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEo7%2B-9S3de5CT%2Bx2UM_i7QGf9ZetB508VrMfO%2Bx01TNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.