hi,all,
- *I have 4 es node with 32 shard 1 replica, each with 8G memory(no
mlock). my elasticsearch version is elasticsearch-0.19.7(recently upgraded
from 0.18.5).
I start each node with command:
nohup bin/elasticsearch -Xms8g -Xmx8g -Des.max-open-files=true
-Dbootstrap.mlockall=true -p es.pid
as umlit -l prints 8388608, that is 8g, the above command prints log Unknown
mlockall error 0 and as a result actually i started es with no mlock.
last night, i received bunch of alarm smses of this service and i logged in
and found that 3 of the 4 nodes are out of memory and gc logs is crazily
rolling.
and current the index size is:
size: 35.8gb (71.7gb)
docs: 16250075 (17764421)
good news different from 0.18.5, the cpu useage drops off to around 100%
not 200+% any more.
as I have no root access to the production machine, I now restart each node
with command
nohup bin/elasticsearch -Xms7g -Xmx7g -Des.max-open-files=true
-Dbootstrap.mlockall=true -p es.pid
does anyone has similar experience and know how to fix this problem
for periodical OOM hangs.
thanks ~