Duplicate events user log in winlogbeat using drop event filter

Elastic Stack Beats
1

Hello all,

What I want to achieve is to remove the duplicate events.
How should I do that? I am stuck here.

Here is the scenario:
When I successfully login into my lab computer there is the event ID 4672 that duplicates the Administrator users.

image
image1532×193 23.5 KB

Here is my config.
This works well, but what I can't find out how to do, is to remove the duplicate events from event ID 4672 and 4624

proccessors:
- drop_event.when.or:
      - equals.winlog.event_id: 4624
      - equals.event_data.LogonType: '0'
      - equals.event_data.LogonType: '5'
      - equals.winlog.event_data.SubjectUserSid: 'S-1-5-18'
      - equals.winlog.event_data.SubjectUserSid: 'S-1-0-0'
      - equals.winlog.event_data.TargetUserName: 'SYSTEM'
2

I wouldn't call those duplicates events per se. They are different event record documents. Each has a unique login ID and I assume unique event record IDs.

But I think the issue with your expression is that the winlog.event_id field is a string and you are using an number. So if you quote that value then it should be able to satisfy the equals condition.

- equals.winlog.event_id: '4624'

3

Dear @andrewkroh ,

Sorry for the late response.
I have these duplicate logs that are almost the same except for that logon ID.
What I want to do is remove the duplicate and leave one

is this possible with drop_event?

these are the logs I'm talking about that are almost the same except for Logon ID:

log 1:

{
  "_index": ".ds-jdname-8.7.0-2023.06.23-000001",
  "_id": "Y_LS5YgB1EcMsXmHZkgW",
  "_version": 1,
  "_score": 0,
  "_source": {
    "agent": {
      "name": "Test-Server",
      "id": "f0a79511-0169-4d2b-9b70-fd5a7194ab07",
      "type": "winlogbeat",
      "ephemeral_id": "ba18f376-59fe-4dde-9972-d51a465ba491",
      "version": "8.7.0"
    },
    "process": {
      "name": "-",
      "pid": 0,
      "executable": "-"
    },
    "winlog": {
      "computer_name": "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph",
      "process": {
        "pid": 620,
        "thread": {
          "id": 5232
        }
      },
      "keywords": [
        "Audit Success"
      ],
      "logon": {
        "id": "0x0",
        "type": "Network"
      },
      "channel": "Security",
      "event_data": {
        "LogonGuid": "{00000000-0000-0000-0000-000000000000}",
        "TargetOutboundDomainName": "-",
        "VirtualAccount": "%%1843",
        "TransmittedServices": "-",
        "LmPackageName": "NTLM V2",
        "RestrictedAdminMode": "-",
        "ElevatedToken": "%%1842",
        "SubjectDomainName": "-",
        "LogonProcessName": "NtLmSsp ",
        "TargetDomainName": "WIN-I9PVBGAGQP2",
        "LogonType": "3",
        "SubjectLogonId": "0x0",
        "KeyLength": "128",
        "TargetOutboundUserName": "-",
        "TargetLogonId": "0x4bfe10",
        "SubjectUserName": "-",
        "TargetLinkedLogonId": "0x0",
        "ImpersonationLevel": "%%1833",
        "TargetUserName": "Administrator",
        "SubjectUserSid": "S-1-0-0",
        "TargetUserSid": "S-1-5-21-1057966994-2306264790-361777542-500",
        "AuthenticationPackageName": "NTLM"
      },
      "opcode": "Info",
      "version": 2,
      "record_id": "141295",
      "task": "Logon",
      "event_id": "4624",
      "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
      "activity_id": "{1bcf1615-a564-0000-fb16-cf1b64a5d901}",
      "api": "wineventlog",
      "provider_name": "Microsoft-Windows-Security-Auditing"
    },
    "log": {
      "level": "information"
    },
    "source": {
      "port": 0,
      "ip": "192.168.111.25",
      "domain": "JD-ESS"
    },
    "message": "An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-21-1057966994-2306264790-361777542-500\n\tAccount Name:\t\tAdministrator\n\tAccount Domain:\t\tWIN-I9PVBGAGQP2\n\tLogon ID:\t\t0x4BFE10\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tJD-ESS\n\tSource Network Address:\t192.168.111.25\n\tSource Port:\t\t0\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",
    "tags": [
      "dc"
    ],
    "@timestamp": "2023-06-23T01:15:48.666Z",
    "ecs": {
      "version": "1.12.0"
    },
    "related": {
      "ip": [
        "192.168.111.25"
      ],
      "user": [
        "Administrator"
      ]
    },
    "host": {
      "name": "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    },
    "event": {
      "ingested": "2023-06-23T01:15:51.190256606Z",
      "code": "4624",
      "provider": "Microsoft-Windows-Security-Auditing",
      "kind": "event",
      "created": "2023-06-23T01:15:50.066Z",
      "module": "security",
      "action": "logged-in",
      "category": [
        "authentication"
      ],
      "type": [
        "start"
      ],
      "outcome": "success"
    },
    "user": {
      "domain": "WIN-I9PVBGAGQP2",
      "name": "Administrator",
      "id": "S-1-5-21-1057966994-2306264790-361777542-500"
    }
  },
  "fields": {
    "winlog.event_data.AuthenticationPackageName": [
      "NTLM"
    ],
    "event.category": [
      "authentication"
    ],
    "process.name.text": [
      "-"
    ],
    "winlog.provider_name": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "winlog.provider_guid": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "process.pid": [
      0
    ],
    "winlog.computer_name": [
      "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    ],
    "winlog.process.pid": [
      620
    ],
    "winlog.event_data.KeyLength": [
      "128"
    ],
    "winlog.keywords": [
      "Audit Success"
    ],
    "winlog.record_id": [
      "141295"
    ],
    "winlog.logon.id": [
      "0x0"
    ],
    "winlog.event_data.VirtualAccount": [
      "%%1843"
    ],
    "source.ip": [
      "192.168.111.25"
    ],
    "log.level": [
      "information"
    ],
    "agent.name": [
      "Test-Server"
    ],
    "host.name": [
      "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    ],
    "event.kind": [
      "event"
    ],
    "winlog.activity_id": [
      "{1bcf1615-a564-0000-fb16-cf1b64a5d901}"
    ],
    "event.outcome": [
      "success"
    ],
    "winlog.version": [
      2
    ],
    "winlog.event_data.TargetUserName": [
      "Administrator"
    ],
    "user.id": [
      "S-1-5-21-1057966994-2306264790-361777542-500"
    ],
    "winlog.event_data.RestrictedAdminMode": [
      "-"
    ],
    "winlog.event_data.TargetUserSid": [
      "S-1-5-21-1057966994-2306264790-361777542-500"
    ],
    "agent.hostname": [
      "Test-Server"
    ],
    "tags": [
      "dc"
    ],
    "related.user": [
      "Administrator"
    ],
    "process.name": [
      "-"
    ],
    "event.provider": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "event.code": [
      "4624"
    ],
    "source.port": [
      0
    ],
    "winlog.event_data.TransmittedServices": [
      "-"
    ],
    "agent.id": [
      "f0a79511-0169-4d2b-9b70-fd5a7194ab07"
    ],
    "ecs.version": [
      "1.12.0"
    ],
    "winlog.event_data.LmPackageName": [
      "NTLM V2"
    ],
    "event.created": [
      "2023-06-23T01:15:50.066Z"
    ],
    "winlog.event_data.LogonGuid": [
      "{00000000-0000-0000-0000-000000000000}"
    ],
    "agent.version": [
      "8.7.0"
    ],
    "winlog.event_data.SubjectUserSid": [
      "S-1-0-0"
    ],
    "winlog.process.thread.id": [
      5232
    ],
    "winlog.event_data.TargetLinkedLogonId": [
      "0x0"
    ],
    "winlog.event_data.ElevatedToken": [
      "%%1842"
    ],
    "user.name": [
      "Administrator"
    ],
    "winlog.event_data.TargetOutboundUserName": [
      "-"
    ],
    "source.domain": [
      "JD-ESS"
    ],
    "agent.type": [
      "winlogbeat"
    ],
    "process.executable.text": [
      "-"
    ],
    "winlog.event_data.SubjectLogonId": [
      "0x0"
    ],
    "event.module": [
      "security"
    ],
    "winlog.event_data.TargetLogonId": [
      "0x4bfe10"
    ],
    "related.ip": [
      "192.168.111.25"
    ],
    "winlog.api": [
      "wineventlog"
    ],
    "user.domain": [
      "WIN-I9PVBGAGQP2"
    ],
    "process.executable": [
      "-"
    ],
    "winlog.event_data.ImpersonationLevel": [
      "%%1833"
    ],
    "winlog.task": [
      "Logon"
    ],
    "winlog.event_data.SubjectUserName": [
      "-"
    ],
    "winlog.logon.type": [
      "Network"
    ],
    "winlog.event_data.TargetOutboundDomainName": [
      "-"
    ],
    "winlog.event_data.LogonProcessName": [
      "NtLmSsp "
    ],
    "winlog.event_id": [
      "4624"
    ],
    "message": [
      "An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-21-1057966994-2306264790-361777542-500\n\tAccount Name:\t\tAdministrator\n\tAccount Domain:\t\tWIN-I9PVBGAGQP2\n\tLogon ID:\t\t0x4BFE10\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tJD-ESS\n\tSource Network Address:\t192.168.111.25\n\tSource Port:\t\t0\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested."
    ],
    "event.action": [
      "logged-in"
    ],
    "event.ingested": [
      "2023-06-23T01:15:51.190Z"
    ],
    "@timestamp": [
      "2023-06-23T01:15:48.666Z"
    ],
    "winlog.channel": [
      "Security"
    ],
    "winlog.event_data.LogonType": [
      "3"
    ],
    "winlog.event_data.TargetDomainName": [
      "WIN-I9PVBGAGQP2"
    ],
    "event.type": [
      "start"
    ],
    "winlog.opcode": [
      "Info"
    ],
    "agent.ephemeral_id": [
      "ba18f376-59fe-4dde-9972-d51a465ba491"
    ],
    "winlog.event_data.SubjectDomainName": [
      "-"
    ],
    "user.name.text": [
      "Administrator"
    ]
  }
}

log 2:

{
  "_index": ".ds-jdname-8.7.0-2023.06.23-000001",
  "_id": "YfLS5YgB1EcMsXmHZkgW",
  "_version": 1,
  "_score": 0,
  "_source": {
    "agent": {
      "name": "Test-Server",
      "id": "f0a79511-0169-4d2b-9b70-fd5a7194ab07",
      "ephemeral_id": "ba18f376-59fe-4dde-9972-d51a465ba491",
      "type": "winlogbeat",
      "version": "8.7.0"
    },
    "process": {
      "name": "-",
      "pid": 0,
      "executable": "-"
    },
    "winlog": {
      "computer_name": "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph",
      "process": {
        "pid": 620,
        "thread": {
          "id": 5232
        }
      },
      "keywords": [
        "Audit Success"
      ],
      "logon": {
        "id": "0x0",
        "type": "Network"
      },
      "channel": "Security",
      "event_data": {
        "LogonGuid": "{00000000-0000-0000-0000-000000000000}",
        "TargetOutboundDomainName": "-",
        "VirtualAccount": "%%1843",
        "TransmittedServices": "-",
        "LmPackageName": "NTLM V2",
        "RestrictedAdminMode": "-",
        "ElevatedToken": "%%1842",
        "SubjectDomainName": "-",
        "LogonProcessName": "NtLmSsp ",
        "TargetDomainName": "WIN-I9PVBGAGQP2",
        "LogonType": "3",
        "SubjectLogonId": "0x0",
        "KeyLength": "128",
        "TargetOutboundUserName": "-",
        "TargetLogonId": "0x4bfd38",
        "TargetLinkedLogonId": "0x0",
        "SubjectUserName": "-",
        "ImpersonationLevel": "%%1833",
        "TargetUserName": "Administrator",
        "SubjectUserSid": "S-1-0-0",
        "TargetUserSid": "S-1-5-21-1057966994-2306264790-361777542-500",
        "AuthenticationPackageName": "NTLM"
      },
      "opcode": "Info",
      "version": 2,
      "record_id": "141290",
      "task": "Logon",
      "event_id": "4624",
      "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
      "activity_id": "{1bcf1615-a564-0000-fb16-cf1b64a5d901}",
      "api": "wineventlog",
      "provider_name": "Microsoft-Windows-Security-Auditing"
    },
    "log": {
      "level": "information"
    },
    "source": {
      "port": 0,
      "ip": "192.168.111.25",
      "domain": "JD-ESS"
    },
    "message": "An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-21-1057966994-2306264790-361777542-500\n\tAccount Name:\t\tAdministrator\n\tAccount Domain:\t\tWIN-I9PVBGAGQP2\n\tLogon ID:\t\t0x4BFD38\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tJD-ESS\n\tSource Network Address:\t192.168.111.25\n\tSource Port:\t\t0\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",
    "tags": [
      "dc"
    ],
    "@timestamp": "2023-06-23T01:15:48.038Z",
    "ecs": {
      "version": "1.12.0"
    },
    "related": {
      "ip": [
        "192.168.111.25"
      ],
      "user": [
        "Administrator"
      ]
    },
    "host": {
      "name": "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    },
    "event": {
      "ingested": "2023-06-23T01:15:51.189916647Z",
      "code": "4624",
      "provider": "Microsoft-Windows-Security-Auditing",
      "kind": "event",
      "created": "2023-06-23T01:15:50.066Z",
      "module": "security",
      "action": "logged-in",
      "category": [
        "authentication"
      ],
      "type": [
        "start"
      ],
      "outcome": "success"
    },
    "user": {
      "domain": "WIN-I9PVBGAGQP2",
      "name": "Administrator",
      "id": "S-1-5-21-1057966994-2306264790-361777542-500"
    }
  },
  "fields": {
    "winlog.event_data.AuthenticationPackageName": [
      "NTLM"
    ],
    "event.category": [
      "authentication"
    ],
    "process.name.text": [
      "-"
    ],
    "winlog.provider_name": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "winlog.provider_guid": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "process.pid": [
      0
    ],
    "winlog.computer_name": [
      "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    ],
    "winlog.process.pid": [
      620
    ],
    "winlog.event_data.KeyLength": [
      "128"
    ],
    "winlog.keywords": [
      "Audit Success"
    ],
    "winlog.record_id": [
      "141290"
    ],
    "winlog.logon.id": [
      "0x0"
    ],
    "winlog.event_data.VirtualAccount": [
      "%%1843"
    ],
    "source.ip": [
      "192.168.111.25"
    ],
    "log.level": [
      "information"
    ],
    "agent.name": [
      "Test-Server"
    ],
    "host.name": [
      "WIN-I9PVBGAGQP2.acccorp.allcard.com.ph"
    ],
    "event.kind": [
      "event"
    ],
    "winlog.activity_id": [
      "{1bcf1615-a564-0000-fb16-cf1b64a5d901}"
    ],
    "event.outcome": [
      "success"
    ],
    "winlog.version": [
      2
    ],
    "winlog.event_data.TargetUserName": [
      "Administrator"
    ],
    "user.id": [
      "S-1-5-21-1057966994-2306264790-361777542-500"
    ],
    "winlog.event_data.RestrictedAdminMode": [
      "-"
    ],
    "winlog.event_data.TargetUserSid": [
      "S-1-5-21-1057966994-2306264790-361777542-500"
    ],
    "agent.hostname": [
      "Test-Server"
    ],
    "tags": [
      "dc"
    ],
    "related.user": [
      "Administrator"
    ],
    "process.name": [
      "-"
    ],
    "event.provider": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "event.code": [
      "4624"
    ],
    "source.port": [
      0
    ],
    "winlog.event_data.TransmittedServices": [
      "-"
    ],
    "agent.id": [
      "f0a79511-0169-4d2b-9b70-fd5a7194ab07"
    ],
    "ecs.version": [
      "1.12.0"
    ],
    "winlog.event_data.LmPackageName": [
      "NTLM V2"
    ],
    "event.created": [
      "2023-06-23T01:15:50.066Z"
    ],
    "winlog.event_data.LogonGuid": [
      "{00000000-0000-0000-0000-000000000000}"
    ],
    "agent.version": [
      "8.7.0"
    ],
    "winlog.event_data.SubjectUserSid": [
      "S-1-0-0"
    ],
    "winlog.process.thread.id": [
      5232
    ],
    "winlog.event_data.TargetLinkedLogonId": [
      "0x0"
    ],
    "winlog.event_data.ElevatedToken": [
      "%%1842"
    ],
    "user.name": [
      "Administrator"
    ],
    "winlog.event_data.TargetOutboundUserName": [
      "-"
    ],
    "source.domain": [
      "JD-ESS"
    ],
    "agent.type": [
      "winlogbeat"
    ],
    "process.executable.text": [
      "-"
    ],
    "winlog.event_data.SubjectLogonId": [
      "0x0"
    ],
    "event.module": [
      "security"
    ],
    "winlog.event_data.TargetLogonId": [
      "0x4bfd38"
    ],
    "related.ip": [
      "192.168.111.25"
    ],
    "winlog.api": [
      "wineventlog"
    ],
    "user.domain": [
      "WIN-I9PVBGAGQP2"
    ],
    "process.executable": [
      "-"
    ],
    "winlog.event_data.ImpersonationLevel": [
      "%%1833"
    ],
    "winlog.task": [
      "Logon"
    ],
    "winlog.event_data.SubjectUserName": [
      "-"
    ],
    "winlog.logon.type": [
      "Network"
    ],
    "winlog.event_data.TargetOutboundDomainName": [
      "-"
    ],
    "winlog.event_data.LogonProcessName": [
      "NtLmSsp "
    ],
    "winlog.event_id": [
      "4624"
    ],
    "message": [
      "An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-21-1057966994-2306264790-361777542-500\n\tAccount Name:\t\tAdministrator\n\tAccount Domain:\t\tWIN-I9PVBGAGQP2\n\tLogon ID:\t\t0x4BFD38\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tJD-ESS\n\tSource Network Address:\t192.168.111.25\n\tSource Port:\t\t0\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested."
    ],
    "event.action": [
      "logged-in"
    ],
    "event.ingested": [
      "2023-06-23T01:15:51.189Z"
    ],
    "@timestamp": [
      "2023-06-23T01:15:48.038Z"
    ],
    "winlog.channel": [
      "Security"
    ],
    "winlog.event_data.LogonType": [
      "3"
    ],
    "winlog.event_data.TargetDomainName": [
      "WIN-I9PVBGAGQP2"
    ],
    "event.type": [
      "start"
    ],
    "winlog.opcode": [
      "Info"
    ],
    "agent.ephemeral_id": [
      "ba18f376-59fe-4dde-9972-d51a465ba491"
    ],
    "winlog.event_data.SubjectDomainName": [
      "-"
    ],
    "user.name.text": [
      "Administrator"
    ]
  }
}
4

I think discarding events in a security use case could be a bad idea unless you know for sure that you don't need this data. How do you know which of the two events should be discarded? Imagine that you need to correlate events using the logon ID, but you dropped the 4624 event that contained that logon ID.

Nonetheless, one approach could be to assign the same Elasticsearch _id to the two events. Elasticsearch requires that the _id be unique so the subsequent events with the same ID are discarded.

processors:
  # This assigns logon events that were generated in the same second (not a one second window)
  # to same Elasticsearch ID which means that only the first event will be indexed. All fields
  # within winlog.* except the record_id and logon.id are used to build the ID.
  - if:
      equals.winlog.event_id: '4624'
    then:
      - fingerprint:
          method: xxhash
          ignore_missing: true
          fields:
            -  winlog.activity_id
            -  winlog.api
            -  winlog.channel
            -  winlog.computer_name
            -  winlog.event_data.AuthenticationPackageName
            -  winlog.event_data.ElevatedToken
            -  winlog.event_data.ImpersonationLevel
            -  winlog.event_data.KeyLength
            -  winlog.event_data.LmPackageName
            -  winlog.event_data.LogonGuid
            -  winlog.event_data.LogonProcessName
            -  winlog.event_data.LogonType
            -  winlog.event_data.RestrictedAdminMode
            -  winlog.event_data.SubjectDomainName
            -  winlog.event_data.SubjectLogonId
            -  winlog.event_data.SubjectUserName
            -  winlog.event_data.SubjectUserSid
            -  winlog.event_data.TargetDomainName
            -  winlog.event_data.TargetLinkedLogonId
            #-  winlog.event_data.TargetLogonId
            -  winlog.event_data.TargetOutboundDomainName
            -  winlog.event_data.TargetOutboundUserName
            -  winlog.event_data.TargetUserName
            -  winlog.event_data.TargetUserSid
            -  winlog.event_data.TransmittedServices
            -  winlog.event_data.VirtualAccount
            -  winlog.event_id
            #-  winlog.keywords
            #-  winlog.logon.id
            -  winlog.logon.type
            -  winlog.opcode
            -  winlog.process.pid
            -  winlog.process.thread.id
            -  winlog.provider_guid
            -  winlog.provider_name
            #-  winlog.record_id
            -  winlog.task
            -  winlog.version
      - script:
          lang: javascript
          source: |-
            function process(evt) {
              // Create an Elasticsearch _id that follows the recommendations from
              // https://www.elastic.co/blog/efficient-duplicate-prevention-for-event-based-data-in-elasticsearch.
              var epochSecHex = evt.Get('@timestamp').Unix().toString(16);
              var fingerprint = evt.Get('fingerprint');
              evt.Put('@metadata._id', epochSecHex + fingerprint);
              evt.Delete('fingerprint');
            }

For example, the two events given above would both get this value added by Winlogbeat.

    "@metadata": {
      "_id": "6494f244a986d02150857c53"
    },
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.