Winlogbeat drop_fields not working

Elastic Stack Beats
1

Hi All,

I have configured Winlogbeat to drop some fields from all the events using below configuration. But it seems the "drop_fields" process is not applying to the events since I can see those fields in the Kibana. Can someone plese tell me what went wrong with the config. (Drop_event proceesses are working fine though)

- name: ForwardedEvents
 tags: [forwarded]
 processors:
  - drop_event:
      when.and:
        - equals.winlog.event_data.EnabledPrivilegeList: "-"
        - equals.winlog.event_id: 4703
  - drop_event:
      when.and:
        - equals.winlog.event_id: 4703
        - regexp.winlog.event_data.TargetUserName: ".*$"
  - drop_fields:
      fields: ["agent.id", "agent.ephemeral_id", "agent.hostname", "ecs.version", "host.id", "host.os.type", "winlog.api", "event.kind"]
      ignore_missing: true 
  - script:
      when.equals.winlog.channel: Security
      lang: javascript
      id: security
      file: ${path.home}/module/security/config/winlogbeat-security.js
  - script:
      when.equals.winlog.channel: Microsoft-Windows-Sysmon/Operational
      lang: javascript
      id: sysmon
      file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js
  - script:
      when.equals.winlog.channel: Windows PowerShell
      lang: javascript
      id: powershell
      file: ${path.home}/module/powershell/config/winlogbeat-powershell.js
  - script:
      when.equals.winlog.channel: Microsoft-Windows-PowerShell/Operational
      lang: javascript
      id: powershell
      file: ${path.home}/module/powershell/config/winlogbeat-powershell.js

BR,
Someunguy1026

2

Could you please temporarly comment out all processors from your configuration (both global and local) and run Winlogbeat in debug mode (winlogbeat -e -d "*") and share its output?

3

I think winlog.event_id is a string in the event so your data type in the condition needs to match. Add some quotes around the value like equals.winlog.event_id: "123"

In regexp.winlog.event_data.TargetUserName: ".*$", this regex matches everything. I think you want '\$$' to match things that end in $.

1 Like
4

Hi Andrewkroh.

Thanks for the advice and suggested regex worked fine.
But my main problem is with "drop_fields" process. It does not drop required fields mentioned in the process.

BR,
Someunguy1026

5

I think the issue with drop_fields is the order in which they applied. agent, ecs, and host are all added after this group of processors is run. And event.kind is added by the script processors. The winlog.api should be working since that is added as soon as the event is produced.

So I think if you move this to the global processors list then it will work (as opposed to the the event log specific processors).

1 Like
6

Hi Andrew,

Yes, moving to global processors solved the issue.

As you mentioned, some fields cannot be deleted using local processors. Once I utilized the global processors, i could drop all the fields i wanted.

Thank you.

BR,
Someunguy1026

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.