I have a es cluster with 70gb data, using mmapfs. There is a lot of available memory, but es cluster don't use so performance is not good
-Xms32g -Xmx32g -XX:PermSize=1000m -XX:MaxPermSize=1000m -XX:NewSize=3000m -XX:MaxNewSize=3000m -XX:MaxDirectMemorySize=2000m
Nodes: 5 Indices: 9 Shards: 80 Data: 71.56 GB CPU: 10% Memory: 7.73 GB / 158.54 GB
How can I force es to use available memory
I know, but can I force as more as possible index data to memory using mmapfs ?
I have another es cluster using mmapfs, I think it take advantage of available memory so the search time is very low.
another es cluster info:
Nodes: 40 Indices: 14 Shards: 868 Data: 265.25 GB CPU: 6300% Memory: 1.32 TB / 2.72 TB
It doesn't index to memory, that is what I mean. It might just be terminology but it's important here.
Heap, and memory, is used in different ways so this isn't something that can be easily answered.
What sort of data is it? What sort of queries? How often do they run? Are you using warmers?
It doesn't index to memory, but I think the index using mmapfs is also in the memory
The MMap FS type stores the shard index on the file system (maps to Lucene MMapDirectory) by
mapping a file into memory (mmap). Memory mapping uses up a portion of the virtual memory address space in your process equal to the size of the file being mapped. Before using this class, be sure you have allowed plenty of virtual address space.
There are many warmer but feeling useless, the search time is not low as my another es cluster
Yes, mmapfs is using virtual memory. It is not using memory. That is an important difference.
Why do you think that you can blame mmapfs?
Look at slowest part of I/O and bottlenecks first and what workload pattern you exercise on the machines, they all create different requirements.
-XX:MaxDirectMemorySize=2000m is a very tight restriction. Why do you select this setting?
Also you should not change GC settings, it will make your JVM much slower.
I don't blame mmapfs, I am sure mmapfs type is high performance because of my another es cluster. I am just curious to why es process not to take available memory. Although I set the xms jvm args
Now I solve the problem:
ulimit -m unlimited
Then everything is ok
ulimit -m unlimited
This is just a temporary solution. For permanent change, please add the following to
es soft memlock unlimited
es hard memlock unlimited
es is the assumed Elasticsearch user. Do not forget to re-login the user to enable the settings.