Memory Utilization in ELK

[rohitarorait82 (Rohit arora)]

Hi All,

Memory utilization on my all the data_nodes is reaching around 70-80 percent. Can someone suggest, how to reduce it. As we are afraid that after adding all the instances it might reach even 100%.

Thanks for you help in advance, PFB details

IP Address Total CPU Total Memory CPU Utilization Memory utilization /ELK Total space /ELK Usage
IP1 8 62 3-6% 15-16% 1T 2.8G
IP2 8 62 10-15% 70-80% 1T 109G
IP3 8 62 8-10% 70-80% 1T 101G
IP4 8 62 5-10% 70-80% 1T 104G
IP5 8 62 5-10% 70-80% 1T 102G
IP6 8 62 5-10% 70-80% 1T 108G

[rugenl (Len Rugen)]

What statistic are you using for free memory?

Sample stats from 'atop'

MEM | tot    92.9G  | free  413.2M | cache  51.0G  | dirty 122.2M |  buff  279.1M | slab    2.7G  | slrec   2.6G | shmem  64.2M  | shrss   2.5M |  vmbal   0.0M | hptot   0.0M |  hpuse   0.0M |
SWP | tot    16.0G  | free   13.4G |               |              |               |               |              |               |              |               | vmcom  40.2G |  vmlim  62.4G |

This shows free memory of 413M on a 96G system. IMHO, Linux would rather use memory if it can find any beneficial use for memory instead of "freeing" it.

Better stat are JVM heap usage and GC frequency; or other signs of memory stress.