Logstash with rotating logs

Elastic Stack Logstash
1

Hi!
We use logstash 6.2.4. which sends logs to Kafka nodes. We need to to track application logs which are rotated each 24h, it is not possible to control log file names and their format is like:
mylogfile_20181007_000000.log
mylogfile_20181008_000000.log
mylogfile_20181009_000000.log

Easiest would be to issue:
input {
file {
path => "/my/application/logs/mylogfile__*.log"

But in this case it captures all the logs all the time, consuming CPU. I tried to add
ignore_older => "300"
max_open_files => "1"
Unfortunately, it still wants to capture all logs but one by one because of "max_open_files"

Issue is that I want logstash to work only with one (current day) file. Is there any possibility to enter path only to work with current day file?
E.g.
path => "/my/application/logs/mylogfile__"%{CURRENTDATE}"_"%{[0-9.]+}".log

where
CURRENTDATE is today's date in format YYYYMMDD
[0-9.]+ is some regex for any number, because instead of "000000" there could be e.g. "000001"

2

Maybe with something like that :

input {
  exec {
    command => "tail -f mylogfile_`date +%Y%m%d`_000000.log
    schedule => "0 0 * * *"
  }
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.