IIS Filebeat module exclude_lines not working

Elastic Stack Beats
1

Hi. This is a simple install of filebeat 6.6.1 on windows.

I've enabled the IIS module and set the location so it doesn't pick up the FTP logs and have grok failures. I also need it to ignore certain lines which contain the user-agent of the load-balancer constantly checking IIS. My iis.yml file is:

 - module: iis
  # Access logs
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["C:/inetpub/logs/LogFiles/W3*/*.log"]
    exclude_lines: ['ALBX']

  # Error logs
  error:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

A line I need to ignore is:

2019-04-01 00:00:03 1.2.3.4 GET / - 80 - 1.2.3.4 ALBX/4.0 - 200 0 0 0

Is this supported?

Thanks

2

Hey @Darren_Mansell

please try to take a look at processors, specifically Drop event

you can combine it with a regex to fulfil a condition and then events will be dropped

3

@Michal_Pristas thanks this looks perfect.

Where would I put the processor for the IIS module? Under the module?

- module: iis
  # Access logs
  access:
    enabled: true
    var.paths: ["C:/inetpub/logs/LogFiles/W3*/*.log"]

  # Error logs
  error:
    enabled: true
    #var.paths:
    
  processors:
   - drop_event:
       when:
          regexp:
             iis.access.user_agent.original: "ALBX.*"
4

please refer to this section where are processors valid

also you can also check filebeat.reference.yml if in doubt.

generally what you wrote is correct except the indentation of the processor. make sure it starts at the beginning of the line without any whitespaces before it.