Exclude_lines per file

Elastic Stack Beats
1

Hello,
I have a filebeat which has 2 files, I would like to have exclude lines different for each one, is it possible to add exclude_lines per path?
the potential config is as below i.e. for access.log I want to exclude different lines while allowing the same lines to go through from the weblogic.log
Thanks for taking the time to read this,

filebeat.inputs:
- type: log
enabled: true
paths:
  - /scratch/tmp/elk_docker/access.log
    exclude_lines:['\/health(\/[a-z]*)*\t200', '(?:.*registry)(?:.*\t200)']
 - /scratch/tmp/elk_docker/weblogic.log
   exclude_lines:['BEA-000628', 'BEA-001128', '\/authorization\/rest\/[v0-9\.]+\/auth\/query\t200']
2

No, that is not possible. This is not even valid YAML. But you can configure 2 inputs instead of 2 paths:

filebeat.inputs:
- type: log
  paths: [/scratch/tmp/elk_docker/access.log]
  exclude_lines: ['\/health(\/[a-z]*)*\t200', '(?:.*registry)(?:.*\t200)']
- type: log
  paths: [/scratch/tmp/elk_docker/weblogic.log]
  exclude_lines: ['BEA-000628', 'BEA-001128', '\/authorization\/rest\/[v0-9\.]+\/auth\/query\t200']

As you use docker, this looks like a good use-case for docker auto-discovery.

3

Thanks Steffen, yes it's an invalid yaml, was just checking if something like it was possible. I was trying to avoid different file beat processes. Seems that's the route I will have to take.

4

Why you need 2 filebeat processes? My sample configuration shows how it can be done within one filebeat process, just with 2 input configurations.

5

Thanks! I misread the previous answer. This works as expected

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.