Swap usage on Elastic Search Node

Elastic Stack Elasticsearch
1

ES Version : 1.7
Java Version : 1.7.0_45-b18
Os : Linux 2.6.18-406.el5 #1 SMP Wed Jun 3 05:51:45 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

ES Cluster : 5 Nodes
300 millions documents per day
3 billion documents in cluster.

I am running this cluster from some time [few months] in production and observing swap is getting used fully on production ES nodes.
Node configuration : 16GB RAM and 8 CPUs.
Xmx and Xms set to 8GB.
vm.swappiness = 0 in /etc/sysctl.conf
Only ES process running on the node
I tried to identify what is causing swap memory to use completely. but not able to identify the root cause.
Can some one please help me to identify the root cause for why swap is getting used on ES nodes?
If you need any further information please let me know.

2

Can any one please help me to identify the root cause for the swap issue?

3

Please be patient when waiting for an answer to your questions. This is a community forum and as such it may take some time before someone replies to your question. Not everyone on the forum is an expert in every area so you may need to wait for someone who knows about the area you are asking about to come online and have the time to look into your problem.

Please see https://www.elastic.co/community/codeofconduct for more details on our code of conduct (in particular the "be patient" section).

There are no SLAs on responses to questions posted on this forum, if you require help with an SLA on responses you should look into purchasing a subscription package that includes support with an SLA such as those offered by Elastic: https://www.elastic.co/subscriptions

4

Ok, I will wait.
Thank you.

5

Does any one facing the same issue?

6

Did you try disabling swap altogether? Or setting bootstrap.mlockall?

7

I disabled the swaping at OS level [vm.swappiness = 0] and also enabled bootstrap.mockall. But still swap is getting used.

8

Does any one facing the same issue?
As such, my cluster is running properly, all response timings are within limits.
Just I want to understand, what causes to use the swap even after disabling the swapping at OS level.