How much virtual memory does a 30.5Gb heap(256 Gb memory in total) for Elasticsearch support?

Elastic Stack Elasticsearch
1

Hello,

Assume I have a machine with 256gb memory and 12TB SSD. Indexed document size is 100TB. I assign 30.5 GB to Elasticsearch heap. The remaining is for Lucene and OS.

My question is, how much virtual memory does Elasticsearch support? To put it in another way, how many indexed documents can I put into the virtual memory for each machine?

Thanks,

Yifei

2

It doesn't support any, it only uses the heap. The OS handles the rest.

4

Thanks very much for your reply!

Then what about Lucene? I read that on a 64-bit system, it is advised to use mmapfs. And I read that Lucene will take advantage of the Virtual memory. What is the maximum of size of virtual memory does the Lucene with 196gb physical memory support?

Thanks,

Yifei

5

Virtual memory is supported by the operating system. For example, for Red Hat Enterprise Linux, you can read about the limits here:

https://access.redhat.com/articles/rhel-limits

ES uses the Java VM, and a Java VM runs as a process on your operating system. I recommend 64 bit so you can make use of mmapfs.

RHEL7 can support 128 TB virtual address space per process on x86_64 architecture. Note, this is far more than any Java VM process can use in practice.