What is the best configuration to run on linux VPS server without crashing?

Elastic Stack Elasticsearch
1

Hello guys,
I am trying to run elasticsearch on a VPS server running CentOS with 4GB of
RAM. It starts successfully but with the following errors and warning
message below.

sudo service elasticsearch start

error: permission denied on key 'vm.max_map_count'
Starting elasticsearch: [ OK ]
root@vps15042 [~]# log4j:WARN No appenders could be found for logger
(common)
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
morefo.

As you can notice, there is no permission to set the vm.max_map_count
setting on this VPS server ( even with root access ). So the only way to
get rid of the message is by commenting the max_map_count setting out.

It then runs for a maximum of 4 hours and then tell me "Elastic service
dead but pid still exists". It is most probably running out of memory and
getting killed. The following is a log report as such :

Mar 18 22:37:51 ecvp156s kernel: [3712702.735583] 22234 (java) invoked
oom-killer in ub 15042 generation 0 gfp 0x200d2
Mar 18 22:37:51 ecvp156s kernel: [3712702.736531] UB-15042-Mem-Info:
Mar 18 22:37:51 ecvp156s kernel: [3712702.756570] Out of memory in UB
15042: OOM killed process 22232 (java) score 0 vm:7815680kB, rss:3786544kB,
swap:0kB
Mar 18 22:37:52 ecvp156s kernel: [3712703.677851] oom-killer in ub 15042
generation 0 ends: task died
Mar 18 22:38:21 ecvp156s kernel: [3712732.546844] 25904 (java) invoked
oom-killer in ub 15042 generation 1 gfp 0x200d2
Mar 18 22:38:21 ecvp156s kernel: [3712732.547766] UB-15042-Mem-Info:
Mar 18 22:38:21 ecvp156s kernel: [3712732.567524] Out of memory in UB
15042: OOM killed process 25901 (java) score 0 vm:7815680kB, rss:3781624kB,
swap:0kB
Mar 18 22:38:22 ecvp156s kernel: [3712733.475601] oom-killer in ub 15042
generation 1 ends: task died
Mar 19 00:24:42 ecvp156s kernel: [3719119.150426] UB-9340-Mem-Info:

My host provider is telling me to increase the amount of RAM. However I am
running ES on a windows surface pro tablet with 4GB of RAM and core i3, and
I am getting absolutely no issues with not a so powerful machine. Can I
change some configurations to make it work perfectly? I've already tried
the following and other combinations, but nothing keeps it running for long
:

/etc/security/limits.conf
elasticsearch hard memlock 300000

/etc/sysconfig/elasticsearch
ES_HEAP_SIZE 1g
MAX_LOCKED_MEMORY to 300000

/etc/elasticsearch/elasticsearch.yml
index.number_of_shards: 1
index.number_of_replicas: 0
index.term_index_interval: 256
index.term_index_divisor: 5

bootstrap.mlockall: true

Thank you for your time. I am stuck with this and I'll have to change
server if I cannot resolve it.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

By the looks of things, you should look for a new provider as they are
doing some things on the underlying hypervisor restricting ES from locking
memory access.

However your heap size is very small, how much data is in your cluster.

On 20 March 2015 at 07:13, Yashin Soraballee yashin.soraballee@gmail.com
wrote:

Hello guys,
I am trying to run elasticsearch on a VPS server running CentOS with 4GB
of RAM. It starts successfully but with the following errors and warning
message below.

sudo service elasticsearch start

error: permission denied on key 'vm.max_map_count'
Starting elasticsearch: [ OK ]
root@vps15042 [~]# log4j:WARN No appenders could be found for logger
(common)
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions for
morefo.

As you can notice, there is no permission to set the vm.max_map_count
setting on this VPS server ( even with root access ). So the only way to
get rid of the message is by commenting the max_map_count setting out.

It then runs for a maximum of 4 hours and then tell me "Elastic service
dead but pid still exists". It is most probably running out of memory and
getting killed. The following is a log report as such :

Mar 18 22:37:51 ecvp156s kernel: [3712702.735583] 22234 (java) invoked
oom-killer in ub 15042 generation 0 gfp 0x200d2
Mar 18 22:37:51 ecvp156s kernel: [3712702.736531] UB-15042-Mem-Info:
Mar 18 22:37:51 ecvp156s kernel: [3712702.756570] Out of memory in UB
15042: OOM killed process 22232 (java) score 0 vm:7815680kB, rss:3786544kB,
swap:0kB
Mar 18 22:37:52 ecvp156s kernel: [3712703.677851] oom-killer in ub 15042
generation 0 ends: task died
Mar 18 22:38:21 ecvp156s kernel: [3712732.546844] 25904 (java) invoked
oom-killer in ub 15042 generation 1 gfp 0x200d2
Mar 18 22:38:21 ecvp156s kernel: [3712732.547766] UB-15042-Mem-Info:
Mar 18 22:38:21 ecvp156s kernel: [3712732.567524] Out of memory in UB
15042: OOM killed process 25901 (java) score 0 vm:7815680kB, rss:3781624kB,
swap:0kB
Mar 18 22:38:22 ecvp156s kernel: [3712733.475601] oom-killer in ub 15042
generation 1 ends: task died
Mar 19 00:24:42 ecvp156s kernel: [3719119.150426] UB-9340-Mem-Info:

My host provider is telling me to increase the amount of RAM. However I am
running ES on a windows surface pro tablet with 4GB of RAM and core i3, and
I am getting absolutely no issues with not a so powerful machine. Can I
change some configurations to make it work perfectly? I've already tried
the following and other combinations, but nothing keeps it running for long
:

/etc/security/limits.conf
elasticsearch hard memlock 300000

/etc/sysconfig/elasticsearch
ES_HEAP_SIZE 1g
MAX_LOCKED_MEMORY to 300000

/etc/elasticsearch/elasticsearch.yml
index.number_of_shards: 1
index.number_of_replicas: 0
index.term_index_interval: 256
index.term_index_divisor: 5

bootstrap.mlockall: true

Thank you for your time. I am stuck with this and I'll have to change
server if I cannot resolve it.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_LUZ5bN9XMyMxh4KseQJ%3D%3DmYxYSb_jiPT-vyuu-2o6_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hello Mark,
Thanks for the information. There is no much data in the cluster. My
website is still on pre-production, and it crashes with the least amount of
data. I've made only about ten entries! Even if I don't use the service (
no web interaction ), it crashes itself after a while :frowning:
If you tell me to change host, then I will reclaim my money since I have a
money back guarantee I am very tired of trying to make it run since almost
three weeks with no success. Instead of trying another host provider, can I
be recommended a few which I am sure will run it perfectly? My website is a
community/social website, and not profit making, so am on a budget.

On Friday, March 20, 2015 at 7:29:36 PM UTC+4, Mark Walkom wrote:

By the looks of things, you should look for a new provider as they are
doing some things on the underlying hypervisor restricting ES from locking
memory access.

However your heap size is very small, how much data is in your cluster.

On 20 March 2015 at 07:13, Yashin Soraballee <yashin.s...@gmail.com
<javascript:>> wrote:

Hello guys,
I am trying to run elasticsearch on a VPS server running CentOS with 4GB
of RAM. It starts successfully but with the following errors and warning
message below.

sudo service elasticsearch start

error: permission denied on key 'vm.max_map_count'
Starting elasticsearch: [ OK ]
root@vps15042 [~]# log4j:WARN No appenders could be found for logger
(common)
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions for
morefo.

As you can notice, there is no permission to set the vm.max_map_count
setting on this VPS server ( even with root access ). So the only way to
get rid of the message is by commenting the max_map_count setting out.

It then runs for a maximum of 4 hours and then tell me "Elastic service
dead but pid still exists". It is most probably running out of memory and
getting killed. The following is a log report as such :

Mar 18 22:37:51 ecvp156s kernel: [3712702.735583] 22234 (java) invoked
oom-killer in ub 15042 generation 0 gfp 0x200d2
Mar 18 22:37:51 ecvp156s kernel: [3712702.736531] UB-15042-Mem-Info:
Mar 18 22:37:51 ecvp156s kernel: [3712702.756570] Out of memory in UB
15042: OOM killed process 22232 (java) score 0 vm:7815680kB, rss:3786544kB,
swap:0kB
Mar 18 22:37:52 ecvp156s kernel: [3712703.677851] oom-killer in ub 15042
generation 0 ends: task died
Mar 18 22:38:21 ecvp156s kernel: [3712732.546844] 25904 (java) invoked
oom-killer in ub 15042 generation 1 gfp 0x200d2
Mar 18 22:38:21 ecvp156s kernel: [3712732.547766] UB-15042-Mem-Info:
Mar 18 22:38:21 ecvp156s kernel: [3712732.567524] Out of memory in UB
15042: OOM killed process 25901 (java) score 0 vm:7815680kB, rss:3781624kB,
swap:0kB
Mar 18 22:38:22 ecvp156s kernel: [3712733.475601] oom-killer in ub 15042
generation 1 ends: task died
Mar 19 00:24:42 ecvp156s kernel: [3719119.150426] UB-9340-Mem-Info:

My host provider is telling me to increase the amount of RAM. However I
am running ES on a windows surface pro tablet with 4GB of RAM and core i3,
and I am getting absolutely no issues with not a so powerful machine. Can I
change some configurations to make it work perfectly? I've already tried
the following and other combinations, but nothing keeps it running for long
:

/etc/security/limits.conf
elasticsearch hard memlock 300000

/etc/sysconfig/elasticsearch
ES_HEAP_SIZE 1g
MAX_LOCKED_MEMORY to 300000

/etc/elasticsearch/elasticsearch.yml
index.number_of_shards: 1
index.number_of_replicas: 0
index.term_index_interval: 256
index.term_index_divisor: 5

bootstrap.mlockall: true

Thank you for your time. I am stuck with this and I'll have to change
server if I cannot resolve it.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b4a6ab74-1a5e-4a06-a7eb-afe113a3255f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Personally, I'd change hosting providers as a default, empty install of
Elasticsearch will not OOM like that.

You can try AWS, DigitalOcean, Linode, I don't really have any
recommendations there though.

On 20 March 2015 at 09:16, Yashin Soraballee yashin.soraballee@gmail.com
wrote:

Hello Mark,
Thanks for the information. There is no much data in the cluster. My
website is still on pre-production, and it crashes with the least amount of
data. I've made only about ten entries! Even if I don't use the service (
no web interaction ), it crashes itself after a while :frowning:
If you tell me to change host, then I will reclaim my money since I have a
money back guarantee I am very tired of trying to make it run since almost
three weeks with no success. Instead of trying another host provider, can I
be recommended a few which I am sure will run it perfectly? My website is a
community/social website, and not profit making, so am on a budget.

On Friday, March 20, 2015 at 7:29:36 PM UTC+4, Mark Walkom wrote:

By the looks of things, you should look for a new provider as they are
doing some things on the underlying hypervisor restricting ES from locking
memory access.

However your heap size is very small, how much data is in your cluster.

On 20 March 2015 at 07:13, Yashin Soraballee yashin.s...@gmail.com
wrote:

Hello guys,
I am trying to run elasticsearch on a VPS server running CentOS with 4GB
of RAM. It starts successfully but with the following errors and warning
message below.

sudo service elasticsearch start

error: permission denied on key 'vm.max_map_count'
Starting elasticsearch: [ OK ]
root@vps15042 [~]# log4j:WARN No appenders could be found for logger
(common)
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions
for morefo.

As you can notice, there is no permission to set the vm.max_map_count
setting on this VPS server ( even with root access ). So the only way to
get rid of the message is by commenting the max_map_count setting out.

It then runs for a maximum of 4 hours and then tell me "Elastic service
dead but pid still exists". It is most probably running out of memory and
getting killed. The following is a log report as such :

Mar 18 22:37:51 ecvp156s kernel: [3712702.735583] 22234 (java) invoked
oom-killer in ub 15042 generation 0 gfp 0x200d2
Mar 18 22:37:51 ecvp156s kernel: [3712702.736531] UB-15042-Mem-Info:
Mar 18 22:37:51 ecvp156s kernel: [3712702.756570] Out of memory in UB
15042: OOM killed process 22232 (java) score 0 vm:7815680kB, rss:3786544kB,
swap:0kB
Mar 18 22:37:52 ecvp156s kernel: [3712703.677851] oom-killer in ub 15042
generation 0 ends: task died
Mar 18 22:38:21 ecvp156s kernel: [3712732.546844] 25904 (java) invoked
oom-killer in ub 15042 generation 1 gfp 0x200d2
Mar 18 22:38:21 ecvp156s kernel: [3712732.547766] UB-15042-Mem-Info:
Mar 18 22:38:21 ecvp156s kernel: [3712732.567524] Out of memory in UB
15042: OOM killed process 25901 (java) score 0 vm:7815680kB, rss:3781624kB,
swap:0kB
Mar 18 22:38:22 ecvp156s kernel: [3712733.475601] oom-killer in ub 15042
generation 1 ends: task died
Mar 19 00:24:42 ecvp156s kernel: [3719119.150426] UB-9340-Mem-Info:

My host provider is telling me to increase the amount of RAM. However I
am running ES on a windows surface pro tablet with 4GB of RAM and core i3,
and I am getting absolutely no issues with not a so powerful machine. Can I
change some configurations to make it work perfectly? I've already tried
the following and other combinations, but nothing keeps it running for long
:

/etc/security/limits.conf
elasticsearch hard memlock 300000

/etc/sysconfig/elasticsearch
ES_HEAP_SIZE 1g
MAX_LOCKED_MEMORY to 300000

/etc/elasticsearch/elasticsearch.yml
index.number_of_shards: 1
index.number_of_replicas: 0
index.term_index_interval: 256
index.term_index_divisor: 5

bootstrap.mlockall: true

Thank you for your time. I am stuck with this and I'll have to change
server if I cannot resolve it.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b4a6ab74-1a5e-4a06-a7eb-afe113a3255f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b4a6ab74-1a5e-4a06-a7eb-afe113a3255f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8B-YeR%2Bd7rjV%3DEWJ-ZWe0cyOtVwaVHZ8QpJgoL5Qkfwg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Hi Mark,

Our server have 7.5GB RAM, 4vCPUs (in Amazon EC2), our data about 50
thousand records (about 50MB)

I think Elasticsearch (ES) server handle 10 connections at the same time
(per second)

I have assign 2GB RAM for HEAP_SIZE, our server will run normally for a
while about 5 minutes and after that, the ES server don't respond anymore
connection - It seems Elasticsearch Service dead.

Could you tell me the best configuration?

Vào 22:29:36 UTC+7 Thứ Sáu, ngày 20 tháng 3 năm 2015, Mark Walkom đã viết:

By the looks of things, you should look for a new provider as they are
doing some things on the underlying hypervisor restricting ES from locking
memory access.

However your heap size is very small, how much data is in your cluster.

On 20 March 2015 at 07:13, Yashin Soraballee <yashin.s...@gmail.co Mam
<javascript:>> wrote:

Hello guys,
I am trying to run elasticsearch on a VPS server running CentOS with 4GB
of RAM. It starts successfully but with the following errors and warning
message below.

sudo service elasticsearch start

error: permission denied on key 'vm.max_map_count'
Starting elasticsearch: [ OK ]
root@vps15042 [~]# log4j:WARN No appenders could be found for logger
(common)
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions for
morefo.

As you can notice, there is no permission to set the vm.max_map_count
setting on this VPS server ( even with root access ). So the only way to
get rid of the message is by commenting the max_map_count setting out.

It then runs for a maximum of 4 hours and then tell me "Elastic service
dead but pid still exists". It is most probably running out of memory and
getting killed. The following is a log report as such :

Mar 18 22:37:51 ecvp156s kernel: [3712702.735583] 22234 (java) invoked
oom-killer in ub 15042 generation 0 gfp 0x200d2
Mar 18 22:37:51 ecvp156s kernel: [3712702.736531] UB-15042-Mem-Info:
Mar 18 22:37:51 ecvp156s kernel: [3712702.756570] Out of memory in UB
15042: OOM killed process 22232 (java) score 0 vm:7815680kB, rss:3786544kB,
swap:0kB
Mar 18 22:37:52 ecvp156s kernel: [3712703.677851] oom-killer in ub 15042
generation 0 ends: task died
Mar 18 22:38:21 ecvp156s kernel: [3712732.546844] 25904 (java) invoked
oom-killer in ub 15042 generation 1 gfp 0x200d2
Mar 18 22:38:21 ecvp156s kernel: [3712732.547766] UB-15042-Mem-Info:
Mar 18 22:38:21 ecvp156s kernel: [3712732.567524] Out of memory in UB
15042: OOM killed process 25901 (java) score 0 vm:7815680kB, rss:3781624kB,
swap:0kB
Mar 18 22:38:22 ecvp156s kernel: [3712733.475601] oom-killer in ub 15042
generation 1 ends: task died
Mar 19 00:24:42 ecvp156s kernel: [3719119.150426] UB-9340-Mem-Info:

My host provider is telling me to increase the amount of RAM. However I
am running ES on a windows surface pro tablet with 4GB of RAM and core i3,
and I am getting absolutely no issues with not a so powerful machine. Can I
change some configurations to make it work perfectly? I've already tried
the following and other combinations, but nothing keeps it running for long
:

/etc/security/limits.conf
elasticsearch hard memlock 300000

/etc/sysconfig/elasticsearch
ES_HEAP_SIZE 1g
MAX_LOCKED_MEMORY to 300000

/etc/elasticsearch/elasticsearch.yml
index.number_of_shards: 1
index.number_of_replicas: 0
index.term_index_interval: 256
index.term_index_divisor: 5

bootstrap.mlockall: true

Thank you for your time. I am stuck with this and I'll have to change
server if I cannot resolve it.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/73d39af1-9254-4bce-9590-789554db90fb%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/46f892d9-1c9f-495f-87ce-c90a49b241c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.