Hi Jörg,
Running just 1 JVM with 32GB on a 24-core 256GB machine is a waste. CPU,
I/O, memory metrics substantiate this. And off course we need to explore
multi-instance before asking mgmt for more money.
Regarding memlock: if no contiguous RAM is available I'd expect a fast
error and not a totally hanging process and call traces (mentioning a 120
second timeouts) in the dmesg. Do you think this is maybe a jvm or
elasticsearch bug? If so, i'll file it.
Regards,
Renzo
Op dinsdag 26 augustus 2014 17:10:56 UTC+2 schreef Jörg Prante:
You should run one node per host.
Two nodes add overhead and suffer from the effects you described.
For mlockall, the user needs privilege to allocate the specified locked
mem, and the OS need contiguous RAM per mlockall call. If the user's
memlock limit is exhausted, or if RAM allocation gets fragmented,
memlocking is no longer possible and fails.Jörg
On Tue, Aug 26, 2014 at 2:54 PM, R. Toma <renzo...@gmail.com <javascript:>
wrote:
Hi all,
In an attempt to squeeze more power out of our physical servers we want
to run multiple ES jvm's per server.Some specs:
- servers has 24 cores, 256GB ram
- each instance binds on different (alias) ip
- each instance has 32GB heap
- both instances run under user 'elastic'
- limits for 'elastic' user: memlock=unlimited
- es config for both instances: bootstrap.mlockall=true
The 1st instance has been running for weeks.
When starting the 2nd instance the following things happen:
- increase of overal cpu load
- lots of I/O to disks
- no logging for 2nd instance
- 2nd instance hangs
- 1st instance keeps running, but gets slowish
- cd /proc/ causes a hang of cd process (until 2nd instance is
killed)- exec 'ps axuw' causes a hang of ps process (until 2nd instance is
killed)Maybe (un)related: I have never been able to run Elasticsearch in a
virtualbox with memlock=unlimited and mlockall=true.After an hour of trial & errors I found that removing setting
'bootstrap.mlockall' (setting it to false) from 2nd instance's
configuration fixes things.I am confused, but acknowledge I do not know anything about memlocking.
Any ideas?
Regards,
Renzo--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b5e4770a-4194-48c9-aec4-4919dc53342a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b5e4770a-4194-48c9-aec4-4919dc53342a%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dbe8fb95-2054-45ac-a07a-5bf2955e8869%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.