at the moment, there are these indices written in node 0
ls -la /var/lib/elasticsearch/elasticsearch/nodes/0/indices/
total 60
4 drwxr-xr-x 15 elasticsearch elasticsearch 4096 Sep 10 02:00 .
4 drwxr-xr-x 4 elasticsearch elasticsearch 4096 Sep 9 11:55 ..
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Aug 29 10:50
logstash-2013.08.29
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Aug 30 09:37
logstash-2013.08.30
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 1 11:54
logstash-2013.08.31
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 1 11:54
logstash-2013.09.01
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 2 14:03
logstash-2013.09.02
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 3 02:00
logstash-2013.09.03
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 4 14:04
logstash-2013.09.04
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 5 02:00
logstash-2013.09.05
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 6 02:00
logstash-2013.09.06
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 9 10:08
logstash-2013.09.07
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 9 10:08
logstash-2013.09.08
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 9 10:08
logstash-2013.09.09
4 drwxr-xr-x 8 elasticsearch elasticsearch 4096 Sep 10 02:00
logstash-2013.09.10
We could extend the setting to unlimited, but why will no thread be closed
after a while?
Anyway, I will try your suggestion.
Thanks and Regards
Oliver
Am Dienstag, 10. September 2013 16:13:34 UTC+2 schrieb David Pilato:
And how many index did you create until now? Do you have rolling indexes?
May be you should increase your values?
echo "elasticsearch soft nproc unlimited" | sudo tee -a /etc/security/limits.conf
echo "elasticsearch hard nproc unlimited" | sudo tee -a /etc/security/limits.conf
I think you will need to logout and login again.
--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs
Le 10 sept. 2013 à 11:48, Oliver <tw1n...@gmail.com <javascript:>> a
écrit :
Hi David,
thank you for this fast response. At the moment we are running
elasticsearch with the default settings, means "index.number_of_shards: 5".
We have only one (master) node running.
We are sending the log content (catalina.out) only from one machine. So I
am wondering that this setup creates so many threads, and no thread will
close after some period of time. It is growing to the max ulimit setting
and then crashes.
Regards
Oliver
Am Dienstag, 10. September 2013 09:49:06 UTC+2 schrieb David Pilato:
How many shards do you create per node?
May be you should create less shards or add more machines?
--
David Pilato | Technical Advocate | *Elasticsearch.comhttp://elasticsearch.com/
*
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs
Le 10 sept. 2013 à 09:32, Oliver tw1n...@gmail.com a écrit :
Hello,
we send logfiles (catalina.out) with logstash (version 1.1.13-flatjar)
using multiline{} to our elasticsearch server (version 0.90.2).
After several hours elasticsearch crashes because of not having enough
memory, more precisely, the limit of 10240 open processes for the
elasticsearch user has been reached. No more threads can be created.
The actual setting of ulimit is as follows:
elasticsearch soft nofile 65535
elasticsearch hard nofile 65535
elasticsearch soft nproc 10240
elasticsearch hard nproc 10240
I don't have any clue at the moment, how we can fix this issues. Maybe
someone can help.
Regards
Oliver
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.