Exclude_files regex

Elastic Stack Beats
1

Hello,

We have some duplicated log entries and would like to exclude a particular log file. There is an 'exclude_files:' directive in the .yml config file, but we are having difficulty making that work.

Question: Is this directive working?
To exclude a file named: badfile.log what would a proper regex be?
How long after restarting filebeat would you expect the file to NOT appear in Kibana/Elasticsearch?

We have tried (with stopping/starting filebeat between each iteration):
exclude_files:"badfile.log"
exclude_files:["badfile.log"]
exclude_files:["^badfile.log$"]
exclude_files:["^//var//log//badfile.log$"]
and a few others.

Thanks,
W

2

Hi,

exclude_files: ['badfile\.log$'] should work.

Template is exclude_files: ['regex']
$ says that the end of the file name is badfile.log
You have to escape the . with \ in a regex, otherwise, . just means "any character"

Cheers.

3

Can you share your full filebeat config?

4

Hi,

exclude_files: ["badfile\.log$"]

filebeat will not start with the above.

filebeat:
prospectors:
-
paths:
- /var/log/messages
- /var/log/syslog
- /var/log/*.log

  input_type: log

exclude_files: ["badfile\.log$"]

output:
logstash:
hosts: ["localhost:5044"]
worker: 1
index: filebeat
shipper:

logging:

files: