Elasticsearch utilize too many file system cache

Elastic Stack Elasticsearch
1

Currently I had a cluster with 2 master nodes and 3 data nodes, with each data nodes having 32GB of RAM...

At first everything went fine, but after adding many indices I found out that the RAM usage for the data nodes are already reaching 97%...

I allocate 8GB of JVM heap for data nodes and the usage only about 50%...So other usage are cache memory...

Is this normal? Coz if this goes on, the RAM usage would reach 99% and I afraid it will cause problem on my data node...

Or there are settings or steps to prevent the high cache memory usage?

2

Yes. I mean it’s somewhat good that you are using all the available resources. :slight_smile:

The other part of the memory (not the HEAP) is managed by the OS. We don’t have any control on that.

Is there anything wrong you can see?
If yes, may be you would need to add a new data node?

3

I mean if all ur physical RAM is being used, isn't it unhealthy?

Before this the data node only have 16GB of RAM, and when I running process which required lookup on ES, the completion time is very slow...

When I check the stats, I found out that the RAM usage already 99%, so I conclude maybe ES need more RAM, hence I add the RAM to 32 GB...

After the processing time went back faster, with RAM usage only 60%+...But then after I run processing again now it reach 97%...

So does memory usage affect ES performance actually?
Is it ok for ES to still run fine even though memory usage reach 99%?

4

image
image.jpg1029×576 355 KB

This is my server stats, I mean is this even healthy? or normal?

The cache memory usage is over the roof isn't it?

5

You need to distinguish between memory used for heap and memory used by the operating system cache. Memory is there to be used, and the operating system manages the cache and will evict from it to free up memory as needed, so high cache usage is generally not a problem. This cache is very important to get the best out of Elasticsearch, especially in later versions that use doc_values more extensively, so heap should be set to 50% of RAM to leave a large enough cache.

6

I already did, even fact lower by 50% by just allocating 12GB to my ES JVM Heap...

So does it mean that even though physical memory RAM usage is 99%, with buff/cache taken most of it, it won't affect ES performance, I shouldn't worry about it?

7

No, let the OS handle it :slight_smile:

That's really bad, see Important Configuration Changes | Elasticsearch: The Definitive Guide [2.x] | Elastic for more.

8

It means that reason on why my cluster performance is not due to cache problem...Maybe another one...

Thanks!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.