BUG report filebeat version : filebeat-7.10.1-2021.01.13-000001 problem : The exclude_lines option in filebeat.yml does not work.
Example item to be filtered out :
2021-01-27T02:08:31.775-0500#011INFO#011log/harvester.go:302#011Harvester started for file: /var/log/syslog
2021-01-27T02:08:22.758-0500#011INFO#011log/harvester.go:333#011File is inactive: /var/log/syslog. Closing because close_inactive of 5m0s reached.
Tested all kinds regex syntax filters none works
# as : is used in regex i use . instead
exclude_lines: ['\bharvester/.go.333|\bharvester/.go.302']
exclude_lines: ['\bgo.333','\bgo.303','^DBG']
exclude_lines: ['.*go.333.*', '.*go.302.*', '^DBG']
\b first word pattern matching
.* any characters till pattern
. any sungle character (replacing the : )
filebeat.inputs
- type: log
enabled: true
paths:
- /var/log/*.log
exlude_lines : # see all the above attempts none work
While still a bug, and please developers explain exclude_lines better
I found an unreliable? work around that works for this case
I still would prefer the other syntax using regex, but it doesnt work.
I tried with when: regexp message:
302 and 333 are line numbers in the harvester.go file from where the log messages are being generated. It seems unreliable to refer to specific line numbers in the exclude pattern. In some future release these line numbers could change.
Without using specific line numbers could you explain what types of lines you are trying to exclude? Maybe there is a better expression we can come up with.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.