Elasticsearch memory is not completely utilized

Elastic Stack Elasticsearch
1

Hi Team,

I have deployed elasticsearch in k8s environment with below configuration.

Es-client:-
replicas: 1
resources:
limits:
cpu: 2
memory: "4Gi"
requests:
cpu: "500m"
memory: "2Gi"
es_java_opts: "-Xms2g -Xmx2g"

Es-Master:-
replicas: 1
resources:
limits:
cpu: 1
memory: "2Gi"
requests:
cpu: "500m"
memory: "1Gi"
es_java_opts: "-Xms1g -Xmx1g"

Es-data:-
replicas: 2
resources:
limits:
cpu: "2"
memory: "4Gi"
requests:
cpu: "500m"
memory: "2Gi"
es_java_opts: "-Xms2g -Xmx2g"

We are using multiple fluentd to push the data, Now the concern is we are not able to completely utilize elasticsearch client and data pod memory. Irrespective of configuring multiple threads to push data to elasticsearch, memory utilization of client and data pod is not increasing. Max memory utilization of elasticsearch client and data pods is up to 2.5gb but the configured memory limit is 4gb. Please refer below screenshot for reference.

image
image751×91 3.24 KB

Can you guys please help here.

Regards,
Varun S

2

Hello,

es_java_opts: "-Xms2g -Xmx2g"

Limite ressources to 2 GB

3

Elasticsearch uses both heap and off-heap storage as well as the operating system page cache for performance. This is why heap is recommended to be set to 50% of available RAM. The amount of page cache used, which will often result in how much memory the page cache uses depends on the amount of data stored in the cluster. If you have little data, the page cache may not be filled yet.

4

If I understand, Varun_S issue is that :

he fixed memory: "4Gi"
and does not understand why
"Max memory utilization of elasticsearch client and data pods is up to 2.5gb"

If I understand es_java_opts :
He fixed
es_java_opts: "-Xms2g -Xmx2g"

then Elastic stack will never go above 2.5gb

5

Elasticsearch will use 2GB for heap, but in addition to that it requires off-heap memory for some data structures as well as the page cache. It will therefore use all 4GB of RAM available once it fills up with data. Please see the documentation for further details.

6

thanks for the explaination it was not clear for me on what this limit : es_java_opts: "-Xms2g -Xmx2g" was applied on.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.