Elasticsearch crashing - out of memory


Recently, we've had a server running elasticsearch randomly crashing without errors.

Upon further investigation we noticed it was crashing due to out-of-memory, as the following was in /var/log/kern.log:

[11533] 106 11533 18907505 8142258 17202 77 0 0 java
Out of memory: Kill process 11533 (java) score 247 or sacrifice child
Killed process 11533 (java) total-vm:75630020kB, anon-rss:32450404kB, file-rss:118628kB

My server settings are:

  • The server has 128 GB ram
  • Ubuntu 14.04 LTS
  • No swap configured, memory lock enabled
  • Heap size 31g
  • ES Version: 2.4.4
  • Java(TM) SE Runtime Environment (build 1.8.0_121-b13)

The weird thing is, I've never seen the machine getting close to using all of its 128GB ram (according to top), so I don't know why the process is being killed.

Any ideas?

What's in the ES logs?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.