Concerns about ES Unable to lock JVM Memory: error=12,reason=Cannot allocate memory


(David Montgomery) #1

Hi,

What is the deal with ES and memory? I am usig using 2.1.3.

I set heap_size of half of ram

I set the below

echo 'elasticsearch soft memlock unlimited' | tee -a /etc/security/limits.conf
echo 'elasticsearch hard memlock unlimited' | tee -a /etc/security/limits.conf
swapoff -a

ES.....how does one resovle on a 4 gig ram server?

[2016-05-26 20:27:58,638][WARN ][bootstrap                ] Unable to lock JVM Memory: error=12,reason=Cannot allocate memory
[2016-05-26 20:27:58,638][WARN ][bootstrap                ] This can result in part of the JVM being swapped out.
[2016-05-26 20:27:58,638][WARN ][bootstrap                ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2016-05-26 20:27:58,638][WARN ][bootstrap                ] These can be adjusted by modifying /etc/security/limits.conf, for example:
        # allow user 'elasticsearch' mlockall
        elasticsearch soft memlock unlimited
        elasticsearch hard memlock unlimited
[2016-05-26 20:27:58,638][WARN ][bootstrap                ] If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2016-05-26 20:27:59,950][INFO ][node                     ] [James Dr. Power] version[2.3.1], pid[24379], build[bd98092/2016-04-04T12:25:05Z]
[2016-05-26 20:27:59,950][INFO ][node                     ] [James Dr. Power] initializing ...
[2016-05-26 20:28:02,408][INFO ][plugins                  ] [James Dr. Power] modules [lang-groovy, reindex, lang-expression], plugins [marvel-agent, license], sites []
[2016-05-26 20:28:02,494][INFO ][env                      ] [James Dr. Power] using [1] data paths, mounts [[/data (/dev/xvdf)]], net usable_space [0b], net total_space [9.7gb], spins?$
[2016-05-26 20:28:02,507][INFO ][env                      ] [James Dr. Power] heap size [1007.3mb], compressed ordinary object pointers [true]
[2016-05-26 20:28:02,507][WARN ][env                      ] [James Dr. Power] max file descriptors [65535] for elasticsearch process likely too low, consider increasing to at least [65$
[2016-05-26 20:28:11,555][INFO ][node                     ] [James Dr. Power] initialized
[2016-05-26 20:28:11,555][INFO ][node                     ] [James Dr. Power] starting ...
[2016-05-26 20:28:11,832][INFO ][transport                ] [James Dr. Power] publish_address {172.31.28.230:9300}, bound_addresses {[::]:9300}
[2016-05-26 20:28:11,848][INFO ][discovery                ] [James Dr. Power] elasticsearchzenawssgdevelopmentlogging/xSWXNMKCRS61xoBNZ7C3ew
[2016-05-26 20:28:14,885][INFO ][cluster.service          ] [James Dr. Power] new_master {James Dr. Power}{xSWXNMKCRS61xoBNZ7C3ew}{172.31.28.230}{172.31.28.230:9300}, reason: zen-disco$
[2016-05-26 20:28:14,942][INFO ][http                     ] [James Dr. Power] publish_address {172.31.28.230:9200}, bound_addresses {[::]:9200}
[2016-05-26 20:28:14,942][INFO ][node                     ] [James Dr. Power] started
[2016-05-26 20:28:19,060][INFO ][license.plugin.core      ] [James Dr. Power] license [06f10d9f-43a1-4587-ba5c-9fac2f344273] - expired
[2016-05-26 20:28:19,096][ERROR][license.plugin.core      ] [James Dr. Power]
#
# LICENSE EXPIRED ON [Monday, April 11, 2016]. IF YOU HAVE A NEW LICENSE, PLEASE
# UPDATE IT. OTHERWISE, PLEASE REACH OUT TO YOUR SUPPORT CONTACT.
#
# COMMERCIAL PLUGINS OPERATING WITH REDUCED FUNCTIONALITY
# - marvel
#  - The agent will stop collecting cluster and indices metrics
#  - The agent will stop to automatically clean up indices older than [marvel.history.duration]
[2016-05-26 20:28:19,995][WARN ][gateway                  ] [James Dr. Power] [_global]: failed to write global state
java.io.IOException: No space left on device
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
        at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
        at sun.nio.ch.IOUtil.write(IOUtil.java:65)
        at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
        at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)

(David Pilato) #2

I reformat a bit your post. Please use </> button when posting.

You have a more major issue than the RAM I believe: No space left on device.

About your memory settings issue, I'm not qualified to answer but I'd just like to share this link in case it helps: https://www.elastic.co/guide/en/elasticsearch/reference/2.3/setup-service.html#_linux


(ZillaG) #3

Where do I set these values described here, in the elasticsearch.yml file?


(system) #4