Exclude_lines per file


#1

Hello,
I have a filebeat which has 2 files, I would like to have exclude lines different for each one, is it possible to add exclude_lines per path?
the potential config is as below i.e. for access.log I want to exclude different lines while allowing the same lines to go through from the weblogic.log
Thanks for taking the time to read this,

filebeat.inputs:
- type: log
enabled: true
paths:
  - /scratch/tmp/elk_docker/access.log
    exclude_lines:['\/health(\/[a-z]*)*\t200', '(?:.*registry)(?:.*\t200)']
 - /scratch/tmp/elk_docker/weblogic.log
   exclude_lines:['BEA-000628', 'BEA-001128', '\/authorization\/rest\/[v0-9\.]+\/auth\/query\t200']

(Steffen Siering) #2

No, that is not possible. This is not even valid YAML. But you can configure 2 inputs instead of 2 paths:

filebeat.inputs:
- type: log
  paths: [/scratch/tmp/elk_docker/access.log]
  exclude_lines: ['\/health(\/[a-z]*)*\t200', '(?:.*registry)(?:.*\t200)']
- type: log
  paths: [/scratch/tmp/elk_docker/weblogic.log]
  exclude_lines: ['BEA-000628', 'BEA-001128', '\/authorization\/rest\/[v0-9\.]+\/auth\/query\t200']

As you use docker, this looks like a good use-case for docker auto-discovery.


#3

Thanks Steffen, yes it's an invalid yaml, was just checking if something like it was possible. I was trying to avoid different file beat processes. Seems that's the route I will have to take.


(Steffen Siering) #4

Why you need 2 filebeat processes? My sample configuration shows how it can be done within one filebeat process, just with 2 input configurations.


#5

Thanks! I misread the previous answer. This works as expected


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.