Winlogbeat multiple add_fields processors throws an error

Elastic Stack Beats
1

Some of our services write their logs to native windows Event Application logs with different SourceName(event.providers) and also to custom Event Logger, which we want add some fields to route data to appropriate indices.

For this, I've used multiple add_fields processors with appropriate conditionals to route data, but this has been throwing error and winlogbeat service stops.
When I try to start the winlogbeat service, it fails and logs the following error -

Exiting: Failed to create new event log. missing fields option in winlogbeat.event_logs.0.processors.0.add_fields

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
    processors:
      - add_fields:
         when:
          regexp:
           event.provider: ".*Job$"      
          fields:
            index: db1
            environment: stg
            providerBase: db1
            indexWildcard: "db1-*" 
          target: ''
      - add_fields:          
         when:
          regexp:
           event.provider: "^ENM.*"
          fields:
            index: enm
            environment: stg
            providerBase: enm
            indexWildcard: "enm-*"
          target: ''    
      - add_fields:          
         when:
          regexp:
           event.provider: "^FUNC.*"      
          fields:
            index: enm
            environment: stg
            providerBase: func
            indexWildcard: "enm-*"
          target: ''                        
      - drop_fields:
         when:
          regexp:
           event.provider: "^ENM.*|^FUNC.*|.*Job$"
         fields: ["winlog.event_data.param1", "event.code", "winlog.event_id",  "winlog.keywords", "winlog.task", "winlog.opcode", "event.kind"]
         ignore_missing: true

  - name: EMNCV4Logs
    ignore_older: 72h
    processors:
      - add_fields:
          target: ''
          fields:
            index: emn
            environment: stg
            providerBase: emn
            indexWildcard: "emn-*"             
      - drop_fields:
         fields: ["winlog.event_data.param1", "event.code", "winlog.event_id",  "winlog.keywords", "winlog.task", "winlog.opcode", "event.kind"]
         ignore_missing: true
  
  - name: EXPLogs
    ignore_older: 72h
    processors:
      - add_fields:
          target: ''
          fields:
            index: exp
            environment: stg
            providerBase: exp
            indexWildcard: "exp-*"             
      - drop_fields:
         fields: ["winlog.event_data.param1", "event.code", "winlog.event_id",  "winlog.keywords", "winlog.task", "winlog.opcode", "event.kind"]
         ignore_missing: true

  - name: System
    ignore_older: 72h

  - name: Security
    ignore_older: 72h
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
2

Got it working. YAML indentation was the issue for the fields under the add_fields processor, the error does give little indication, although not very clear, as it did not complain of any YAML errors when tested with yamllint and other online YAML testing tools.

Perhaps, the exception message can be improved, saves lot of troubleshooting time for new users like me :slight_smile:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.