Too many log files?

Elastic Stack Elasticsearch
1

Setup:

Rsyslog provides log from mutiple servers to a central server. Directory
structure is like:

/var/log/rsyslog/SERVER-1/2014-10-21/*.log

/var/log/rsyslog/SERVER-2/2014-10-21/*.log

So multiple server directories each having multiple dates within and
multiple log files.

Now, we setup ELK to read store and view these logs.

Eg: Logstash

 file{
            path => [
             "/var/log/rsyslog/**/cron.log",
             "/var/log/rsyslog/**/kernel.log",
            ]
 
     }

Issue:

Every day a new directory with current date say 2014-11-25 is created and
logstash tracks it with the glob input.

So logstash will keep on opening files as they are added and after a point
nofile limit will cross.

What is the best way to handle this?

I read online about log rotation but that I guess is when file size grows
lot, but my issue is of number of files.

Any help is appreciated.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c7030a9-2e9c-40fa-a4bb-8fa2ae38089e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.