System module: error when defining syslog.input.processors

Elastic Stack Beats
1

Hey, I'm trying to define a processor in the system module just for syslog to drop certain spammy, non-relevant events. But when I define a drop_event I get an error about a processor having multiple actions:
#011ERROR#011fileset/factory.go:105#011Error creating input: each processor must have exactly one action, but found 2 actions (add_locale,drop_event)

According to the documentation that should be a valid location: https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html#where-valid
I also tried defining it in the modules top scope which failed with "Fileset system/input is configured but doesn't exist"

I'd rather avoid defining the processor at top scope since I have other inputs where this filter would just cause unnecessary computation.

Any help for this would be appreciated.

Config 
- module: system
  # Syslog
  syslog:
    enabled: true
    input:
      processors:
      - drop_event:
          when:
            and:
            - contains:
                message: "error on subcontainer"
            - contains:
                message: "ia_addr"

  # Authorization logs
  auth:
    enabled: true
2

Hey @TatsuKishi, welcome to discuss :slight_smile:

This error can be a side effect of having processors predefined in modules. This add_locale processor is used by default, but your config tries to override it. Could you try to include this processor in your config? It would be something like this:

- module: system
  # Syslog
  syslog:
    enabled: true
    input:
      processors:
      - add_locale: ~
      - drop_event:
          when:
            and:
            - contains:
                message: "error on subcontainer"
            - contains:
                message: "ia_addr"

  # Authorization logs
  auth:
    enabled: true

Another option you have is to define your drop_event processor in the main configuration file of filebeat, but then you have to take into account that the processor would be applied to all events generated.

3

Adding the "add_locale" to the processors does the job, thank you!

Another thing I noticed is that this error was not picked up when doing a "filebeat check config". That threw me for quite the loop since it went through our CI due to that when it really shouldn't have.

4

Yeah, probably filebeat considers this configuration correct, so config checks work, but later it cannot instantiate the pipelines because of the conflict with the predefined processors.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.