Elasticsearch rpm creates file /usr/lib/sysctl.d/elasticsearch.conf, but doesn't apply it, so vm.max_map_count remains the default until reboot

Elastic Stack Elasticsearch
#1

Hi. We checked our elasticsearch and OS settings, and found that the system
parameter vm.max_map_count was too low. It was still the linux
default, 65530, but should be increased to 262144. This value is set in
/usr/lib/sysctl.d/elasticsearch.conf (and also in
/etc/sysconfig/elasticsearch, MAX_MAP_COUNT=262144). But we have to call
"sysctl -p /usr/lib/sysctl.d/elasticsearch.conf" for this to take effect,
or reboot the server. I don't know if anything speaks against doing stuff
like this in the rpm's install scripts. If it's OK, please add it. This was
the behaviour on RHEL6 and CentOS6 with the
rpms elasticsearch.noarch 1.2.1-1 and probably also 1.0.1.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/521c23f0-5f2c-4479-b224-34f4542f1291%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#2

This is a kernel parameter and requires a reboot.

It is bad habit to enforce kernel parameter change from an application.
This is due to the administrator.

Note that increasing vm.max_map_count is only required for applications
that use many small mmap calls (with mlockall, Elasticsearch does not). A
number which is too high will potentially increase memory consumption on
the server and reduce performance.

Jörg

On Tue, Oct 28, 2014 at 3:53 PM, Suny suny.kim@gmx.net wrote:

Hi. We checked our elasticsearch and OS settings, and found that the
system parameter vm.max_map_count was too low. It was still the linux
default, 65530, but should be increased to 262144. This value is set in
/usr/lib/sysctl.d/elasticsearch.conf (and also in
/etc/sysconfig/elasticsearch, MAX_MAP_COUNT=262144). But we have to call
"sysctl -p /usr/lib/sysctl.d/elasticsearch.conf" for this to take effect,
or reboot the server. I don't know if anything speaks against doing stuff
like this in the rpm's install scripts. If it's OK, please add it. This was
the behaviour on RHEL6 and CentOS6 with the
rpms elasticsearch.noarch 1.2.1-1 and probably also 1.0.1.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/521c23f0-5f2c-4479-b224-34f4542f1291%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/521c23f0-5f2c-4479-b224-34f4542f1291%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHmyO3QJwakrDMH950pbDevsdbGdNQ86Zzf0k6Ysa9SLQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

#3

Thanks Jörg, your answer is really helpful. I didn't know of the connection
between mlockall and max_map_count. I agree that the system parameters are
the sys ads' job. Still, some documentation on this setting from
elasticsearch.org would help, like in which cases it's important, as you
explained. BTW, i'm using your post
http://jprante.github.io/2012/11/28/Elasticsearch-Java-Virtual-Machine-settings-explained.html
a lot.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ac09e37f-d603-46ca-a499-7fa542e86b06%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#4

Unfortunately, the rpms puts its file into the wrong directory. In RHEL,
the sysctl reads /etc/sysctl.d, not /usr/lib/sysctl.d/.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8c69f6e1-2b23-474a-957a-3cc7bfbd1d55%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#5

Wait. I'd say it doesn't require a reboot in RHEL6, it's a tunable
parameter because it sits in /proc/sys/vm/max_map_count. In fact, it is
activated at service start in the init script
sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT
So I'd say things are fine once the service has started. That
/usr/lib/sysctl.d/elasticsearch.conf doesn't mean a thing on RHEL6, the
MAX_MAP_COUNT in /etc/sysconfig/elasticsearch counts. I apologize for the
confusion. It was on a customer's server, and i wasn't aware that
elasticsearch wasn't running when he queried the system parameters. For my
part, learned sth again.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ba7bece3-8b61-474a-9e42-b6d5e7e8a216%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#6