Elasticsearch using huge amount of processes

Elastic Stack Elasticsearch
4

And how many index did you create until now? Do you have rolling indexes?

May be you should increase your values?

echo "elasticsearch soft nproc unlimited" | sudo tee -a /etc/security/limits.conf
echo "elasticsearch hard nproc unlimited" | sudo tee -a /etc/security/limits.conf

I think you will need to logout and login again.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 10 sept. 2013 à 11:48, Oliver tw1nh34d@gmail.com a écrit :

Hi David,

thank you for this fast response. At the moment we are running elasticsearch with the default settings, means "index.number_of_shards: 5". We have only one (master) node running.
We are sending the log content (catalina.out) only from one machine. So I am wondering that this setup creates so many threads, and no thread will close after some period of time. It is growing to the max ulimit setting and then crashes.

Regards
Oliver

Am Dienstag, 10. September 2013 09:49:06 UTC+2 schrieb David Pilato:
How many shards do you create per node?
May be you should create less shards or add more machines?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 10 sept. 2013 à 09:32, Oliver tw1n...@gmail.com a écrit :

Hello,

we send logfiles (catalina.out) with logstash (version 1.1.13-flatjar) using multiline{} to our elasticsearch server (version 0.90.2).

After several hours elasticsearch crashes because of not having enough memory, more precisely, the limit of 10240 open processes for the elasticsearch user has been reached. No more threads can be created.

The actual setting of ulimit is as follows:

elasticsearch soft nofile 65535
elasticsearch hard nofile 65535
elasticsearch soft nproc 10240
elasticsearch hard nproc 10240

I don't have any clue at the moment, how we can fix this issues. Maybe someone can help.

Regards
Oliver

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.