Logstash close file handles?


We have rsyslog creating log files directories like: /var/log/rsyslog/SERVER-NAME/LOG-DATE/LOG-FILE-NAME
So multiple servers are spilling out their logs of different dates to a central location.

Now to read these logs and store them in elasticsearch for analysing I have my logstash config file something like this:

   path => /var/log/rsyslog/**/*.log


Now after some time no of log files get increased to a big number. As logtsash keeps the file handle open for every file in its purview even though its of no use after reading the file as log file for a particular date will not be updated after that date.
I have increased the file openings limit to 65K in /etc/security/limits.conf

Can we make logstash close the handle after some time ??

1 Like