There are two mechanisms, mlockall and mmap, so I'd like to explain what
they do.
First, mlockall, when called via JNA with the MCL_CURRENT flag, enforces
the OS to keep the JVM process pages in RAM. mlockall is invoked at a very
early stage during Elasticsearch bootstrap [0] (therefore
bootstrap.mlockall). With MCL_CURRENT, the invocation does affect only one
OS call. It means, no further malloc()ed memory segments of the JVM process
will be locked. MCL_FUTURE would force each future malloc call of the
process (and forked processes) to get included into the locking. This is
out of control of the JVM, and is prone to segfaults (core dumps). With
having allocated the heap size with -Xms and -Xmx like in ES_HEAP_SIZE, the
JVM will not perform malloc() again for the heap. The result is that
heap-based operations of Elasticsearch are executed in RAM for the lifetime
of the process. mlockall is only available on POSIX OSes.
Elasticsearch mmapfs (memory mapped store implementation) is based on
Lucene's MMAPDirectory. The memory mapping of Lucene in MMAPDirectory [1]
uses ByteBuffer allocations, executed by malloc() calls, which are then
changed with the mmap() call to inform the OS that file reads on Lucene
index files should be treated directly in the virtual memory address space.
So, this mechanism relates to the input/output subsystem of the OS, it maps
the device I/O buffer directly into the process address space. Such mapped
files prevent double copying of I/O buffered data while reading. When
Lucene files are written, the ordinary OS file caching is used. mmap is
available on every modern 64-bit OS.
Additionally, Lucene's MMAPDirectory can unmap the ByteBuffers by a hack
because Java does not support this by default [2]. ByteBuffers for large
files use a lot of virtual address space, so it is recommend only on 64-bit
platforms. But there is a problem when Lucene wants to close files and
switch over to new files. ByteBuffers are maintained by a weak reference,
they are only disposed by the garbage collector when no thread is using the
buffer anymore. The risk of getting OOMs will get higher the more index
files Lucene reads from. To tackle this, Lucene enforces the disposition of
the virtual memory address space by an unmap() hack using not well
documented code.
In summary, the MCL_CURRENT flag is useful so ByteBuffers don't get locked
but mapped for I/O properly. Still both mlockall/mmap share the same
target: making optimal use of RAM for Elasticsearch.
Jörg
[0]
https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/bootstrap/Bootstrap.java
[1]
http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_4_4_0/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java
[2] http://bugs.sun.com/view_bug.do?bug_id=4724038
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.