Memory "leak" like behaviour in ES

Elastic Stack Elasticsearch
1

Did anyone experience a "memory leak" like behaviour , even in Idle mode.
It seems that we forgot to bound something and ES is just eating all the
memory until it dies.

Here is a memory heap size screenshot from graphite :

https://lh3.googleusercontent.com/-LCrBcFvwNX0/UMSanw4r0wI/AAAAAAAALds/MhTMn55CWmo/s1600/Capture.JPG

This is an heap size image from one of the our nodes ( all the other behave
the exact same way ).

The cluster is most of the time in idle mode and does not get any
traffic. The big downfall near to 12/05 12PM is where it died , so we had
to restart it again and it started to rise contantly .

The somewhere near the end of the graph we redirected traffic to the
nodes so you see the spikes in the memory.
Some info on the data that is there.

  • ES version : 0.19.8
  • 23 M documents indexed ,
  • 25GB of index data.
  • 12 shards , 1 replica
  • Runs 4 nodes on 4 x 48GB VM's

All the nodes behave the same.

Thanks for any help

Roman

--

2

Hi,

you did not mention what the maximum size of heap is that you configured.

What do you mean with "where it died" - did the process exited? Did you
encounter OOM? What is the exact message?

What plugins/other Java codes are running on the JVM?

Can you reproduce the issue with examining jvisualvm or jstack heap dumps?

To hunt memory leaks, you need to set a small memory heap and run a
fraction of your workload while enabling GC logging, then you can better
watch what is happening.

At the moment, I can only see rising memory usage on the heap, which is
kind of expected.

Best regards,

Jörg

--