Filebeat not picking up log files

Elastic Stack Beats
1

Hello Team,
I'm using Filebeat 7.10.2 to ship logs directly to Elasticsearch. But I'm facing an issue while giving input to filebeat. Filebeat only picks up one log file even when I'm specifying to read all files from input directory in filebeat.yml.
Here's my filebeat.yml config:

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /usr/local/logdirectory/pkt_20*.log

output.elasticsearch:
  hosts: "localhost:9200"
  username: admin
  password: admin

The logdirectory folder contains multiple files named as pkt_20220101.log, pkt_20220102.log and so on. Filebeat only picks up the pkt_20220110.log file and ignores the rest. Could anyone from the team help me out with this issue?

2

Hi!

paths setting is a glob based setting as mentioned at Log input | Filebeat Reference [7.16] | Elastic. Maybe the pattern you are setting is not correct. Could you check the debug logs of Filebeat for any indicators?

C.

3

Hello ChrsMark,

I've checked the debug logs of Filebeat and can see that the harvester starts only for files with pattern like pkt_2022011x.log and not the pattern that I've specified in paths. I've tried several different glob patterns but none of them seem to work.

4

Hi!

This is weird. Not sure what could be the problem here :thinking: .
Could you try with something like /usr/local/logdirectory/*.log? If so what happens?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.