Dynamic date log file input to Filebeat

[Dibyakanta (Dibyakanta Sahoo)]

Hi,

I have a requirement to add log files to my filebeat server which will create an index in Elasticsearch.
The location from where Filebeat would be taking the log file contains log file of different dates. Suppose for today's date, the log file generated by Tomcat is user_log-2019.06.29 and for tomorrow a new log file with name log-2019.06.30 will be generated. So how shall I configure the path in filebeat.yml so that it will take that particular date's log file. My filebeat server will be running for the whole month. My yml file looks like below:

filebeat.prospectors:

• type: log
  paths:
  • D:/MyFolder/filebeat-6.1.0-windows-x86_64/filebeat-6.1.0-windows-x86_64/user_log-2019.06.29.log
    This is the yml file for one day's log.

I tried with

paths:
- D:/MyFolder/filebeat-6.1.0-windows-x86_64/filebeat-6.1.0-windows-x86_64/user_log-%{+YYYY.MM.dd}.log

But it didn't work. Can you please help me with this.

Thank You.

[martinr_ubi (Martin R.)]

you can’t use that in a glob pattern for the path setting. use a * as a wildcard character for the part of the filename that varies between files.

To ignore older files use the ignore_older setting as mentioned in your other post.

[Dibyakanta (Dibyakanta Sahoo)]

Thank you for your help.