Filtering not working

Hi,

  • Running ELK 6.8.9 on premises
  • Output in Winlogbeat config file is Eslasticsearch (not Logstash)

I am trying to add some processing on the client side to filter only the Win events I need before sending the data to ELK. I am trying a drop_event filter following steps found in the following URL:

https://www.elastic.co/guide/en/beats/winlogbeat/current/configuration-winlogbeat-options.html

When applying the filter, no logs are received by ELK.

My goal is to send to ELK only the events listed in the config below.

winlogbeat.event_logs:

  • name: Application
    ignore_older: 72h
  • name: Security
    processors:
    • drop_event.when.not.or:
      • equals.winlog.event_id: 4624
      • equals.winlog.event_id: 4625
      • equals.winlog.event_id: 4634
      • equals.winlog.event_id: 4648
      • equals.winlog.event_id: 4719
      • equals.winlog.event_id: 4964
      • equals.winlog.event_id: 1102
      • equals.winlog.event_id: 4720
      • equals.winlog.event_id: 4722
      • equals.winlog.event_id: 4723
      • equals.winlog.event_id: 4725
      • equals.winlog.event_id: 4728
      • equals.winlog.event_id: 4732
      • equals.winlog.event_id: 4756
      • equals.winlog.event_id: 4738
      • equals.winlog.event_id: 4740
      • equals.winlog.event_id: 4767
      • equals.winlog.event_id: 4735
      • equals.winlog.event_id: 4737
      • equals.winlog.event_id: 4755
      • equals.winlog.event_id: 4772
      • equals.winlog.event_id: 4777
      • equals.winlog.event_id: 4782
      • equals.winlog.event_id: 4616
      • equals.winlog.event_id: 4657
      • equals.winlog.event_id: 4697
      • equals.winlog.event_id: 4698
      • equals.winlog.event_id: 4699
      • equals.winlog.event_id: 4700
      • equals.winlog.event_id: 4701
      • equals.winlog.event_id: 4702
      • equals.winlog.event_id: 4946
      • equals.winlog.event_id: 4947
      • equals.winlog.event_id: 4950
      • equals.winlog.event_id: 4954
      • equals.winlog.event_id: 5025
      • equals.winlog.event_id: 5031
      • equals.winlog.event_id: 5152
      • equals.winlog.event_id: 5153
      • equals.winlog.event_id: 5155
      • equals.winlog.event_id: 5157
      • equals.winlog.event_id: 5447
  • name: System
  • name: Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity,Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose,Microsoft-Windows-Windows Firewall With Advanced Security/Firewall,Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose,Microsoft-Windows-Windows Firewall With Advanced Security/Network Isolation Operational

What could be wrong here?

Thank you

I have been trying different combinations found around the forum. This time I reduced the amount of processors "drop_event", but still it is not filtering. Also considering event ID as string instead of integer. ELK keep receiving events not included on the list and also events were SubjetUserName or TargetUserName involves accounts ending in $ (service managed accounts).

If somebody could tell me what I am doing wrong....

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - drop_event:
      when.or:
        # This filters logons from managed service accounts.
        # The trailing dollar sign is reserved for managed service accounts.
        - regexp.winlog.event_data.TargetUserName: '.*\$'
        - regexp.winlog.event_data.SubjectUserName: '.*\$'

        # This filters logon type 0 which is used for system accounts.
        - equals.winlog.event_data.LogonType: '0'

        # This filters logon type 5 which is used for service accounts.
        - equals.winlog.event_data.LogonType: '5'

        # This filters anonymous logons which are typically benign.
        # Anonymous users have extremely limited privileges.
        - equals.winlog.event_data.TargetUserName: 'ANONYMOUS LOGON'
      
      when.not.or:
        - equals.winlog.event_id: '4624'
        - equals.winlog.event_id: '4625'
        - equals.winlog.event_id: '4634'
        - equals.winlog.event_id: '4648'
        - equals.winlog.event_id: '4719'
        - equals.winlog.event_id: '4964'
        - equals.winlog.event_id: '1102'
        - equals.winlog.event_id: '4720'
        - equals.winlog.event_id: '4722'
        - equals.winlog.event_id: '4723'
        - equals.winlog.event_id: '4725'
        - equals.winlog.event_id: '4728'
        - equals.winlog.event_id: '4732'
        - equals.winlog.event_id: '4756'
        - equals.winlog.event_id: '4738'
        - equals.winlog.event_id: '4740'
        - equals.winlog.event_id: '4767'
        - equals.winlog.event_id: '4735'
        - equals.winlog.event_id: '4737'
        - equals.winlog.event_id: '4755'
        - equals.winlog.event_id: '4772'
        - equals.winlog.event_id: '4777'
        - equals.winlog.event_id: '4782'
        - equals.winlog.event_id: '4616'
        - equals.winlog.event_id: '4657'
        - equals.winlog.event_id: '4697'
        - equals.winlog.event_id: '4698'
        - equals.winlog.event_id: '4699'
        - equals.winlog.event_id: '4700'
        - equals.winlog.event_id: '4701'
        - equals.winlog.event_id: '4702'
        - equals.winlog.event_id: '4946'
        - equals.winlog.event_id: '4947'
        - equals.winlog.event_id: '4950'
        - equals.winlog.event_id: '4954'
        - equals.winlog.event_id: '5025'
        - equals.winlog.event_id: '5031'
        - equals.winlog.event_id: '5152'
        - equals.winlog.event_id: '5153'
        - equals.winlog.event_id: '5155'
        - equals.winlog.event_id: '5157'
        - equals.winlog.event_id: '5447'

hi @ManuelF , can you try removing the winlog prefix from the field name and let us know if it worked?
ex:

- regexp.event_data.TargetUserName: '.*\$'

instead of

- regexp.winlog.event_data.TargetUserName: '.*\$'

same for the event_id , also values can be integer.

Hi @MarianaD, I tried what you said. Removed all "winlog" strings. Restarted Winlogbeat service with no errors. ELK keeps receiving many events not listed in my filter (Ex: 5158, 5156, 4800, etc...).

With the processor working as it should, those Windows events should not reach ELK. Right?

I am available and ready to try any other solution.

Thank you

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - drop_event:
      when.or:
        # This filters logons from managed service accounts.
        # The trailing dollar sign is reserved for managed service accounts.
        - regexp.event_data.TargetUserName: '.*\$'
        - regexp.event_data.SubjectUserName: '.*\$'

        # This filters logon type 0 which is used for system accounts.
        - equals.event_data.LogonType: '0'

        # This filters logon type 5 which is used for service accounts.
        - equals.event_data.LogonType: '5'

        # This filters anonymous logons which are typically benign.
        # Anonymous users have extremely limited privileges.
        - equals.event_data.TargetUserName: 'ANONYMOUS LOGON'
      
      when.not.or:
        - equals.event_id: 4624
        - equals.event_id: 4625
        - equals.event_id: 4634
        - equals.event_id: 4648
        - equals.event_id: 4719
        - equals.event_id: 4964
        - equals.event_id: 1102
        - equals.event_id: 4720
        - equals.event_id: 4722
        - equals.event_id: 4723
        - equals.event_id: 4725
        - equals.event_id: 4728
        - equals.event_id: 4732
        - equals.event_id: 4756
        - equals.event_id: 4738
        - equals.event_id: 4740
        - equals.event_id: 4767
        - equals.event_id: 4735
        - equals.event_id: 4737
        - equals.event_id: 4755
        - equals.event_id: 4772
        - equals.event_id: 4777
        - equals.event_id: 4782
        - equals.event_id: 4616
        - equals.event_id: 4657
        - equals.event_id: 4697
        - equals.event_id: 4698
        - equals.event_id: 4699
        - equals.event_id: 4700
        - equals.event_id: 4701
        - equals.event_id: 4702
        - equals.event_id: 4946
        - equals.event_id: 4947
        - equals.event_id: 4950
        - equals.event_id: 4954
        - equals.event_id: 5025
        - equals.event_id: 5031
        - equals.event_id: 5152
        - equals.event_id: 5153
        - equals.event_id: 5155
        - equals.event_id: 5157
        - equals.event_id: 5447

Also, when processors are enabled, ELK is getting many (not sure if all of them) events duplicated.

Hi,

Any other idea to resolve this. Please help.

Thank you

@MarianaD or somebody else from Elastic Team (or a regular user) that could please help me to resolve the issue I am experiencing with Winlogbeat processors?

Thank you

Hello,

I'm not sure about using two "when" in the same processor (drop_event) - I only tried with only 1 "when" per processor.

You are in 6.8, i'm assuming your beat versions is the same. As @MarianaD said, you should not use the 'winlog' subfield in your conf.

Maybe you can try this :

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - drop_event.when.or:
      # This filters logons from managed service accounts.
      # The trailing dollar sign is reserved for managed service accounts.
      - regexp.event_data.TargetUserName: '.*\$'
      - regexp.event_data.SubjectUserName: '.*\$'

      # This filters logon type 0 which is used for system accounts.
      - equals.event_data.LogonType: '0'

      # This filters logon type 5 which is used for service accounts.
      - equals.event_data.LogonType: '5'

      # This filters anonymous logons which are typically benign.
      # Anonymous users have extremely limited privileges.
      - equals.event_data.TargetUserName: 'ANONYMOUS LOGON'
      
      - not.or:
        - equals.event_id: 4624
        - equals.event_id: 4625
        - equals.event_id: 4634
        - equals.event_id: 4648
        - equals.event_id: 4719
        - equals.event_id: 4964
        - equals.event_id: 1102
        - equals.event_id: 4720
        - equals.event_id: 4722
        - equals.event_id: 4723
        - equals.event_id: 4725
        - equals.event_id: 4728
        - equals.event_id: 4732
        - equals.event_id: 4756
        - equals.event_id: 4738
        - equals.event_id: 4740
        - equals.event_id: 4767
        - equals.event_id: 4735
        - equals.event_id: 4737
        - equals.event_id: 4755
        - equals.event_id: 4772
        - equals.event_id: 4777
        - equals.event_id: 4782
        - equals.event_id: 4616
        - equals.event_id: 4657
        - equals.event_id: 4697
        - equals.event_id: 4698
        - equals.event_id: 4699
        - equals.event_id: 4700
        - equals.event_id: 4701
        - equals.event_id: 4702
        - equals.event_id: 4946
        - equals.event_id: 4947
        - equals.event_id: 4950
        - equals.event_id: 4954
        - equals.event_id: 5025
        - equals.event_id: 5031
        - equals.event_id: 5152
        - equals.event_id: 5153
        - equals.event_id: 5155
        - equals.event_id: 5157
        - equals.event_id: 5447

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.