Logstash close file handles?

Elastic Stack Logstash
1

BACKGROUND:

We have rsyslog creating log files directories like: /var/log/rsyslog/SERVER-NAME/LOG-DATE/LOG-FILE-NAME
So multiple servers are spilling out their logs of different dates to a central location.

Now to read these logs and store them in elasticsearch for analysing I have my logstash config file something like this:

file{
   path => /var/log/rsyslog/**/*.log
}

ISSUE :

Now after some time no of log files get increased to a big number. As logtsash keeps the file handle open for every file in its purview even though its of no use after reading the file as log file for a particular date will not be updated after that date.
I have increased the file openings limit to 65K in /etc/security/limits.conf

Can we make logstash close the handle after some time ??

1 Like