Enriching Data with Custom Index

Elastic Stack Kibana
1

I'm trying to enrich an alert generating event with a user name. The alert generating event only currently contains the SID of the account and it's randomly generated. I created a custom index to contain the SID and account name from previous events that include both. But I can't figure out how to write these elements to the custom index. Support helped with the below snippet, but "index" is apparently not a supported processor type. Any other way to do this?

##Create the ingest pipline for the alert triggering events
PUT _ingest/pipeline/lenovo_account_creation
{
  "processors": [
    {
      "script": {
        "source": """
          if (ctx?.user?.target?.name != null && ctx.user.target.name.startsWith('lenovo_tmp_')) {
            Map accountMapping = ['sid': ctx.user.target.id, 'account_name': ctx.user.target.name];
            ctx.account_mapping = accountMapping;
          }
        """
      }
    },
    {
      "index": {
        "target_index": "lenovo_account_mappings",
        "if": "ctx.account_mapping != null"
      }
    }
  ]
}
2

Hi @FlyNavy Welcome to the community

What version are you on?

There is a standard approach for enriching data perhaps take a look at this

3

Thanks. I do have the rest of the enrichment process in place by what I think is this process. I just can't get the data detected in the ingestion pipeline to write to the enriching index for the mapped pair.

PUT /_enrich/policy/lenovo_account_enrichment_policy
{
"match": {
 "indices": "lenovo_account_mappings",
 "match_field": "sid",
 "enrich_fields": ["account_name"]
}
}

POST /_enrich/policy/lenovo_account_enrichment_policy/_execute

PUT _ingest/pipeline/enrich_alert_events
{
"processors": [
 {
   "enrich": {
     "policy_name": "lenovo_account_enrichment_policy",
     "field": "user.target.id",
     "target_field": "related.user_info",
     "max_matches": "1"
   }
 },
 {
   "script": {
     "source": """
       if (ctx?.related?.user_info?.account_name != null && ctx.related.user_info.account_name.startsWith('lenovo_tmp_')) {
         ctx.event.ignore_alert = true;
       }
     """
   }
 }
]
}
4

You will need to show the source enrich data couple samples
A couple of docs you want to enrich

The pipeline (which is what you are showing)
I would probably take out the script part till you see the enrich work

You are not showing the results you can use the _simulate on the pipeline to see if it is working...

Give full samples and we can probably help (you can anonymize the data) just need to see it

5

Assuming you have created mappings for your indices?

6

I've copied the first several lines of the event data that includes both name and random SID. Not sure the rest matters.

A user account was created. Subject: Security ID: S-1-5-18 Account Name: XXXXXXXX$ Account Domain: XXXXXXXXX Logon ID: 0x3E7 New Account: Security ID: S-1-5-21-949799925-2528041633-2783471443-4440 Account Name: lenovo_tmp_sjstILTZ Account Domain: XXXXXXX Attributes: SAM Account Name: lenovo_tmp_sjstILTZ Display Name: User Principal Name: - Home Directory: Home Drive: Script Path: Profile Path: User Workstations:

And here's the alert generating event.
A member was added to a security-enabled local group. Subject: Security ID: S-1-5-18 Account Name: XXXXXXXX$ Account Domain: XXXXXXXXXX Logon ID: 0x3E7 Member: Security ID: S-1-5-21-949799925-2528041633-2783471443-4440 Account Name: - Group: Security ID: S-1-5-32-544 Group Name: Administrators Group Domain: Builtin Additional Information: Privileges: -@timestampJul 25, 2024 @ 13:23:41.976agent.ephemeral_idfd18652b-adeb-49b1-a2e2-97239f97df8aagent.hostnameSophosagent.id82295858-5992-45e2-bce1-966e78fb8d42agent.nameXXXXXXagent.typewinlogbeatagent.version8.0.0ecs.version1.12.0event.actionadded-member-to-groupevent.categoryiamevent.code4732event.created

Since the SID is random, I need to use the account name format "lenovo_tmp_********' to track and stop alert generation for these events. I would like to enrich the second event listed with the account name so I can tell the rule to ignore these events.

7

I have already created the custom index for the mapped pair values.

PUT /lenovo_account_mappings
{
"mappings": {
 "properties": {
   "sid": { "type": "keyword" },
   "account_name": { "type": "keyword" }
 }
}
}
8

Apologies I want to see what the documents look like in Elastic after you have ingested them not the raw logs ... the JSON representation of in elastic After you have ingested them.

Have you parsed the logs? What does that data look like?

If you have not parsed the logs into fields that will be your first step...

I see the mapping for the enrich data ... good...

I mean have you created a mapping for the logs, ingested and parsed them.

How are the logs getting ingested are you using filebeat, Elastic Agent?

What version are you on?

9

Here's the message after ingestion that kicks off the alert. It contains only the SID

{
  "_index": ".ds-winlogbeat-8.0.0-2024.07.30-000570",
  "_id": "TMz7BJEBTE3soZyKwcni",
  "_version": 1,
  "_score": 0,
  "_source": {
    "agent": {
      "name": "XXXXX",
      "id": "82295858-5992-45e2-bce1-966e78fb8d42",
      "type": "winlogbeat",
      "ephemeral_id": "fd18652b-adeb-49b1-a2e2-97239f97df8a",
      "version": "8.0.0"
    },
    "@timestamp": "2024-07-30T18:50:59.567Z",
    "winlog": {
      "computer_name": "XXXX.XXXXX.XXXXXX",
      "process": {
        "pid": 672,
        "thread": {
          "id": 1096
        }
      },
      "keywords": [
        "Audit Success"
      ],
      "logon": {
        "id": "0x3e7"
      },
      "channel": "Security",
      "event_data": {
        "SubjectUserName": "XXXXXXX",
        "MemberSid": "S-1-5-21-2366788312-814031040-1301217162-1086",
        "TargetSid": "S-1-5-32-544",
        "SubjectDomainName": "XXXXXXXX",
        "SubjectLogonId": "0x3e7",
        "MemberName": "-",
        "TargetUserName": "Administrators",
        "TargetDomainName": "Builtin",
        "SubjectUserSid": "S-1-5-18",
        "PrivilegeList": "-"
      },
      "opcode": "Info",
      "record_id": "529303",
      "task": "Security Group Management",
      "event_id": "4732",
      "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
      "activity_id": "{41b78ff5-d8e1-0002-f890-b741e1d8da01}",
      "api": "wineventlog",
      "provider_name": "Microsoft-Windows-Security-Auditing"
    },
    "ecs": {
      "version": "1.12.0"
    },
    "related": {
      "user": [
        "-",
        "XXXXXXXX"
      ]
    },
    "log": {
      "level": "information"
    },
    "host": {
      "name": "XXXXXX.XXXXXX.XXX"
    },
    "event": {
      "ingested": "2024-07-30T18:51:33.724955676Z",
      "code": "4732",
      "provider": "Microsoft-Windows-Security-Auditing",
      "created": "2024-07-30T18:51:33.147Z",
      "kind": "event",
      "action": "added-member-to-group",
      "category": [
        "iam"
      ],
      "type": [
        "group",
        "change"
      ],
      "outcome": "success"
    },
    "message": "A member was added to a security-enabled local group.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tXXXXXXXXX\n\tAccount Domain:\t\tXXXXXXXXXX\n\tLogon ID:\t\t0x3E7\n\nMember:\n\tSecurity ID:\t\tS-1-5-21-2366788312-814031040-1301217162-1086\n\tAccount Name:\t\t-\n\nGroup:\n\tSecurity ID:\t\tS-1-5-32-544\n\tGroup Name:\t\tAdministrators\n\tGroup Domain:\t\tBuiltin\n\nAdditional Information:\n\tPrivileges:\t\t-",
    "user": {
      "domain": "XXXXXXXXXX",
      "name": "XXXXXXXX",
      "id": "S-1-5-18",
      "target": {
        "name": "-",
        "group": {
          "domain": "Builtin",
          "name": "Administrators",
          "id": "S-1-5-32-544"
        }
      }
    },
    "tags": [
      "forwarded"
    ],
    "group": {
      "domain": "Builtin",
      "name": "Administrators",
      "id": "S-1-5-32-544"
    }
  },
  "fields": {
    "event.category": [
      "iam"
    ],
    "winlog.provider_guid": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "winlog.provider_name": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "user.target.group.domain": [
      "Builtin"
    ],
    "winlog.computer_name": [
      "XXXXXXX.XXXXXXXX.XXX"
    ],
    "winlog.process.pid": [
      672
    ],
    "winlog.keywords": [
      "Audit Success"
    ],
    "winlog.record_id": [
      "529303"
    ],
    "winlog.logon.id": [
      "0x3e7"
    ],
    "log.level": [
      "information"
    ],
    "agent.name": [
      "XXXXXXX"
    ],
    "host.name": [
      "XXXXX.XXXXXXXXXX.XXX"
    ],
    "user.target.name.text": [
      "-"
    ],
    "event.kind": [
      "event"
    ],
    "winlog.activity_id": [
      "{41b78ff5-d8e1-0002-f890-b741e1d8da01}"
    ],
    "event.outcome": [
      "success"
    ],
    "group.name": [
      "Administrators"
    ],
    "winlog.event_data.TargetUserName": [
      "Administrators"
    ],
    "user.id": [
      "S-1-5-18"
    ],
    "user.target.group.id": [
      "S-1-5-32-544"
    ],
    "agent.hostname": [
      "XXXXXXX"
    ],
    "related.user": [
      "-",
      "XXXXXXXXXX"
    ],
    "tags": [
      "forwarded"
    ],
    "user.target.name": [
      "-"
    ],
    "event.provider": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "event.code": [
      "4732"
    ],
    "agent.id": [
      "82295858-5992-45e2-bce1-966e78fb8d42"
    ],
    "ecs.version": [
      "1.12.0"
    ],
    "event.created": [
      "2024-07-30T18:51:33.147Z"
    ],
    "agent.version": [
      "8.0.0"
    ],
    "winlog.event_data.SubjectUserSid": [
      "S-1-5-18"
    ],
    "winlog.process.thread.id": [
      1096
    ],
    "user.target.group.name": [
      "Administrators"
    ],
    "winlog.event_data.PrivilegeList": [
      "-"
    ],
    "group.id": [
      "S-1-5-32-544"
    ],
    "user.name": [
      "XXXXXXXXX"
    ],
    "agent.type": [
      "winlogbeat"
    ],
    "winlog.event_data.MemberName": [
      "-"
    ],
    "winlog.event_data.SubjectLogonId": [
      "0x3e7"
    ],
    "winlog.event_data.TargetSid": [
      "S-1-5-32-544"
    ],
    "group.domain": [
      "Builtin"
    ],
    "winlog.api": [
      "wineventlog"
    ],
    "user.domain": [
      "XXXXXXXXXXXX"
    ],
    "winlog.task": [
      "Security Group Management"
    ],
    "winlog.event_data.SubjectUserName": [
      "XXXXXXXXXX"
    ],
    "message": [
      "A member was added to a security-enabled local group.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tXXXXXXXXX\n\tAccount Domain:\t\tXXXXXXXXXXXXX\n\tLogon ID:\t\t0x3E7\n\nMember:\n\tSecurity ID:\t\tS-1-5-21-2366788312-814031040-1301217162-1086\n\tAccount Name:\t\t-\n\nGroup:\n\tSecurity ID:\t\tS-1-5-32-544\n\tGroup Name:\t\tAdministrators\n\tGroup Domain:\t\tBuiltin\n\nAdditional Information:\n\tPrivileges:\t\t-"
    ],
    "winlog.event_id": [
      "4732"
    ],
    "event.action": [
      "added-member-to-group"
    ],
    "event.ingested": [
      "2024-07-30T18:51:33.724Z"
    ],
    "@timestamp": [
      "2024-07-30T18:50:59.567Z"
    ],
    "winlog.channel": [
      "Security"
    ],
    "winlog.event_data.MemberSid": [
      "S-1-5-21-2366788312-814031040-1301217162-1086"
    ],
    "event.type": [
      "group",
      "change"
    ],
    "winlog.event_data.TargetDomainName": [
      "Builtin"
    ],
    "winlog.opcode": [
      "Info"
    ],
    "agent.ephemeral_id": [
      "fd18652b-adeb-49b1-a2e2-97239f97df8a"
    ],
    "winlog.event_data.SubjectDomainName": [
      "XXXXXXXXXX"
    ],
    "user.name.text": [
      "XXXXXXXXX"
    ]
  }
}
10

And here's the account creation event

{
  "_index": ".ds-winlogbeat-8.0.0-2024.07.30-000570",
  "_id": "Fsz7BJEBTE3soZyKZrkQ",
  "_version": 1,
  "_score": 0,
  "_source": {
    "agent": {
      "name": "XXXXXXXX",
      "id": "82295858-5992-45e2-bce1-966e78fb8d42",
      "type": "winlogbeat",
      "ephemeral_id": "fd18652b-adeb-49b1-a2e2-97239f97df8a",
      "version": "8.0.0"
    },
    "@timestamp": "2024-07-30T18:50:59.547Z",
    "winlog": {
      "computer_name": "XXXXXX.XXXXXXXXXX.XXX",
      "process": {
        "pid": 672,
        "thread": {
          "id": 1096
        }
      },
      "keywords": [
        "Audit Success"
      ],
      "logon": {
        "id": "0x3e7"
      },
      "channel": "Security",
      "event_data": {
        "TargetSid": "S-1-5-21-2366788312-814031040-1301217162-1086",
        "NewUACList": [
          "LOCKOUT",
          "SCRIPT"
        ],
        "SidHistory": "-",
        "LogonHours": "%%1797",
        "ScriptPath": "%%1793",
        "DisplayName": "%%1793",
        "HomePath": "%%1793",
        "SubjectDomainName": "XXXXXXXXXX",
        "AllowedToDelegateTo": "-",
        "TargetDomainName": "XXXXXXXXX",
        "PrivilegeList": "-",
        "UserWorkstations": "%%1793",
        "SamAccountName": "lenovo_tmp_ygauCNFY",
        "UserParameters": "%%1793",
        "HomeDirectory": "%%1793",
        "OldUacValue": "0x0",
        "SubjectLogonId": "0x3e7",
        "NewUacValue": "0x15",
        "PrimaryGroupId": "513",
        "AccountExpires": "%%1794",
        "ProfilePath": "%%1793",
        "UserAccountControl": [
          "2080",
          "2082",
          "2084"
        ],
        "SubjectUserName": "XXXXXXXX",
        "PasswordLastSet": "%%1794",
        "UserPrincipalName": "-",
        "TargetUserName": "lenovo_tmp_ygauCNFY",
        "SubjectUserSid": "S-1-5-18"
      },
      "opcode": "Info",
      "record_id": "529299",
      "event_id": "4720",
      "task": "User Account Management",
      "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
      "activity_id": "{41b78ff5-d8e1-0002-f890-b741e1d8da01}",
      "api": "wineventlog",
      "provider_name": "Microsoft-Windows-Security-Auditing"
    },
    "ecs": {
      "version": "1.12.0"
    },
    "related": {
      "user": [
        "XXXXXXXXXXX",
        "lenovo_tmp_ygauCNFY"
      ]
    },
    "log": {
      "level": "information"
    },
    "host": {
      "name": "XXXXXXXX.XXXXXXXXXX.XXX"
    },
    "event": {
      "ingested": "2024-07-30T18:51:10.223631818Z",
      "code": "4720",
      "provider": "Microsoft-Windows-Security-Auditing",
      "created": "2024-07-30T18:51:10.110Z",
      "kind": "event",
      "action": "added-user-account",
      "type": [
        "user",
        "creation"
      ],
      "category": [
        "iam"
      ],
      "outcome": "success"
    },
    "message": "A user account was created.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tXXXXXXXXXXX\n\tAccount Domain:\t\tXXXXXXXXXXXX\n\tLogon ID:\t\t0x3E7\n\nNew Account:\n\tSecurity ID:\t\tS-1-5-21-2366788312-814031040-1301217162-1086\n\tAccount Name:\t\tlenovo_tmp_ygauCNFY\n\tAccount Domain:\t\tXXXXXXXXXX\n\nAttributes:\n\tSAM Account Name:\tlenovo_tmp_ygauCNFY\n\tDisplay Name:\t\t<value not set>\n\tUser Principal Name:\t-\n\tHome Directory:\t\t<value not set>\n\tHome Drive:\t\t<value not set>\n\tScript Path:\t\t<value not set>\n\tProfile Path:\t\t<value not set>\n\tUser Workstations:\t<value not set>\n\tPassword Last Set:\t<never>\n\tAccount Expires:\t\t<never>\n\tPrimary Group ID:\t513\n\tAllowed To Delegate To:\t-\n\tOld UAC Value:\t\t0x0\n\tNew UAC Value:\t\t0x15\n\tUser Account Control:\t\n\t\tAccount Disabled\n\t\t'Password Not Required' - Enabled\n\t\t'Normal Account' - Enabled\n\tUser Parameters:\t<value not set>\n\tSID History:\t\t-\n\tLogon Hours:\t\tAll\n\nAdditional Information:\n\tPrivileges\t\t-",
    "user": {
      "domain": "XXXXXXXXXXXXX",
      "name": "XXXXXXXX",
      "id": "S-1-5-18",
      "target": {
        "domain": "XXXXXXXXX",
        "name": "lenovo_tmp_ygauCNFY",
        "id": "S-1-5-21-2366788312-814031040-1301217162-1086"
      }
    },
    "tags": [
      "forwarded"
    ]
  },
  "fields": {
    "winlog.event_data.UserAccountControl": [
      "2080",
      "2082",
      "2084"
    ],
    "winlog.event_data.SidHistory": [
      "-"
    ],
    "event.category": [
      "iam"
    ],
    "winlog.event_data.NewUACList": [
      "LOCKOUT",
      "SCRIPT"
    ],
    "winlog.provider_guid": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "winlog.provider_name": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "winlog.computer_name": [
      "XXXXXXXX.XXXXXXXXXXXXX.XXX"
    ],
    "user.target.id": [
      "S-1-5-21-2366788312-814031040-1301217162-1086"
    ],
    "winlog.process.pid": [
      672
    ],
    "winlog.event_data.UserParameters": [
      "%%1793"
    ],
    "winlog.keywords": [
      "Audit Success"
    ],
    "winlog.record_id": [
      "529299"
    ],
    "winlog.logon.id": [
      "0x3e7"
    ],
    "log.level": [
      "information"
    ],
    "agent.name": [
      "XXXXXXXX"
    ],
    "winlog.event_data.AllowedToDelegateTo": [
      "-"
    ],
    "winlog.event_data.HomePath": [
      "%%1793"
    ],
    "host.name": [
      "XXXXXXXX.XXXXXXXXXXXX.XXX"
    ],
    "user.target.name.text": [
      "lenovo_tmp_ygauCNFY"
    ],
    "event.kind": [
      "event"
    ],
    "winlog.activity_id": [
      "{41b78ff5-d8e1-0002-f890-b741e1d8da01}"
    ],
    "event.outcome": [
      "success"
    ],
    "winlog.event_data.OldUacValue": [
      "0x0"
    ],
    "winlog.event_data.UserWorkstations": [
      "%%1793"
    ],
    "winlog.event_data.TargetUserName": [
      "lenovo_tmp_ygauCNFY"
    ],
    "user.id": [
      "S-1-5-18"
    ],
    "agent.hostname": [
      "XXXXXXX"
    ],
    "related.user": [
      "XXXXXXXXXXX",
      "lenovo_tmp_ygauCNFY"
    ],
    "tags": [
      "forwarded"
    ],
    "user.target.name": [
      "lenovo_tmp_ygauCNFY"
    ],
    "event.provider": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "event.code": [
      "4720"
    ],
    "agent.id": [
      "82295858-5992-45e2-bce1-966e78fb8d42"
    ],
    "ecs.version": [
      "1.12.0"
    ],
    "event.created": [
      "2024-07-30T18:51:10.110Z"
    ],
    "winlog.event_data.ScriptPath": [
      "%%1793"
    ],
    "agent.version": [
      "8.0.0"
    ],
    "winlog.event_data.SubjectUserSid": [
      "S-1-5-18"
    ],
    "winlog.process.thread.id": [
      1096
    ],
    "winlog.event_data.PrivilegeList": [
      "-"
    ],
    "user.name": [
      "XXXXXXXXX"
    ],
    "winlog.event_data.SamAccountName": [
      "lenovo_tmp_ygauCNFY"
    ],
    "winlog.event_data.PrimaryGroupId": [
      "513"
    ],
    "winlog.event_data.AccountExpires": [
      "%%1794"
    ],
    "winlog.event_data.DisplayName": [
      "%%1793"
    ],
    "agent.type": [
      "winlogbeat"
    ],
    "winlog.event_data.SubjectLogonId": [
      "0x3e7"
    ],
    "winlog.event_data.TargetSid": [
      "S-1-5-21-2366788312-814031040-1301217162-1086"
    ],
    "winlog.event_data.ProfilePath": [
      "%%1793"
    ],
    "winlog.api": [
      "wineventlog"
    ],
    "winlog.event_data.NewUacValue": [
      "0x15"
    ],
    "user.target.domain": [
      "XXXXXXXXX"
    ],
    "user.domain": [
      "XXXXXXXXXXXXXX"
    ],
    "winlog.event_data.UserPrincipalName": [
      "-"
    ],
    "winlog.task": [
      "User Account Management"
    ],
    "winlog.event_data.HomeDirectory": [
      "%%1793"
    ],
    "winlog.event_data.SubjectUserName": [
      "XXXXXXXXXXX"
    ],
    "message": [
      "A user account was created.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tXXXXXXXX\n\tAccount Domain:\t\tXXXXXXXXXXXXXX\n\tLogon ID:\t\t0x3E7\n\nNew Account:\n\tSecurity ID:\t\tS-1-5-21-2366788312-814031040-1301217162-1086\n\tAccount Name:\t\tlenovo_tmp_ygauCNFY\n\tAccount Domain:\t\tXXXXXXXXX\n\nAttributes:\n\tSAM Account Name:\tlenovo_tmp_ygauCNFY\n\tDisplay Name:\t\t<value not set>\n\tUser Principal Name:\t-\n\tHome Directory:\t\t<value not set>\n\tHome Drive:\t\t<value not set>\n\tScript Path:\t\t<value not set>\n\tProfile Path:\t\t<value not set>\n\tUser Workstations:\t<value not set>\n\tPassword Last Set:\t<never>\n\tAccount Expires:\t\t<never>\n\tPrimary Group ID:\t513\n\tAllowed To Delegate To:\t-\n\tOld UAC Value:\t\t0x0\n\tNew UAC Value:\t\t0x15\n\tUser Account Control:\t\n\t\tAccount Disabled\n\t\t'Password Not Required' - Enabled\n\t\t'Normal Account' - Enabled\n\tUser Parameters:\t<value not set>\n\tSID History:\t\t-\n\tLogon Hours:\t\tAll\n\nAdditional Information:\n\tPrivileges\t\t-"
    ],
    "winlog.event_id": [
      "4720"
    ],
    "winlog.event_data.LogonHours": [
      "%%1797"
    ],
    "event.action": [
      "added-user-account"
    ],
    "event.ingested": [
      "2024-07-30T18:51:10.223Z"
    ],
    "@timestamp": [
      "2024-07-30T18:50:59.547Z"
    ],
    "winlog.channel": [
      "Security"
    ],
    "event.type": [
      "user",
      "creation"
    ],
    "winlog.event_data.TargetDomainName": [
      "XXXXXXXXXX"
    ],
    "winlog.opcode": [
      "Info"
    ],
    "agent.ephemeral_id": [
      "fd18652b-adeb-49b1-a2e2-97239f97df8a"
    ],
    "winlog.event_data.PasswordLastSet": [
      "%%1794"
    ],
    "winlog.event_data.SubjectDomainName": [
      "XXXXXXXXXXXX"
    ],
    "user.name.text": [
      "XXXXXXXXX"
    ]
  }
}

@FlyNavy Please format your code by putting 3 backticks ``` before and after your code

11

what is a sample account mapping document?

12

It would be mapping:
winlog.event_data.TargetSid to winlog.event_data.TargetSID
And
winlog.event_data.MemberName to winlog.event_data.MemberName

The winlog.event_data.MemberName is the missing element from the alert generating event. It's "-" in the above sample.

13

Sorry not tracking...

Here is my complete code... the problem is this...

"processors": [
 {
   "enrich": {
     "policy_name": "lenovo_account_enrichment_policy",
     "field": "user.target.id", <<<<. This field does not exist in your source document... 
     "target_field": "related.user_info",
     "max_matches": "1"
   }
 },

Your Doc has either

user.id
or
user.target.group.id

but not

"field": "user.target.id", which is in your pipeline does not exist in the incoming source document so it is not matching...

    "user": {
      "domain": "XXXXXXXXXX",
      "name": "XXXXXXXX",
      "id": "S-1-5-18",  <<<<< this Field
      "target": {
        "name": "-",
        "group": {
          "domain": "Builtin",
          "name": "Administrators",
          "id": "S-1-5-32-544"  <<<<< this Field
        }
      }
    },

My Complete code...

PUT /lenovo_account_mappings
{
"mappings": {
 "properties": {
   "sid": { "type": "keyword" },
   "account_name": { "type": "keyword" }
 }
}
}


POST lenovo_account_mappings/_doc
{
  "sid": "S-1-5-32-544",
  "account_name" : "brown"
}


PUT /_enrich/policy/lenovo_account_enrichment_policy
{
  "match": {
    "indices": "lenovo_account_mappings",
    "match_field": "sid",
    "enrich_fields": ["account_name"]
  }
}

PUT /_enrich/policy/lenovo_account_enrichment_policy/_execute?wait_for_completion=false


PUT _ingest/pipeline/enrich_alert_events
{
  "processors": [
    {
      "enrich": {
        "policy_name": "lenovo_account_enrichment_policy",
        "field": "user.target.group.id", <<< NOTE 
        "target_field": "related.user_info",
        "max_matches": "1"
      }
    }
  ]
}

POST _ingest/pipeline/enrich_alert_events/_simulate
{
  "docs": [
    {
      "_source": {
        "agent": {
          "name": "XXXXX",
          "id": "82295858-5992-45e2-bce1-966e78fb8d42",
          "type": "winlogbeat",
          "ephemeral_id": "fd18652b-adeb-49b1-a2e2-97239f97df8a",
          "version": "8.0.0"
        },
        "@timestamp": "2024-07-30T18:50:59.567Z",
        "winlog": {
          "computer_name": "XXXX.XXXXX.XXXXXX",
          "process": {
            "pid": 672,
            "thread": {
              "id": 1096
            }
          },
          "keywords": [
            "Audit Success"
          ],
          "logon": {
            "id": "0x3e7"
          },
          "channel": "Security",
          "event_data": {
            "SubjectUserName": "XXXXXXX",
            "MemberSid": "S-1-5-21-2366788312-814031040-1301217162-1086",
            "TargetSid": "S-1-5-32-544",
            "SubjectDomainName": "XXXXXXXX",
            "SubjectLogonId": "0x3e7",
            "MemberName": "-",
            "TargetUserName": "Administrators",
            "TargetDomainName": "Builtin",
            "SubjectUserSid": "S-1-5-18",
            "PrivilegeList": "-"
          },
          "opcode": "Info",
          "record_id": "529303",
          "task": "Security Group Management",
          "event_id": "4732",
          "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
          "activity_id": "{41b78ff5-d8e1-0002-f890-b741e1d8da01}",
          "api": "wineventlog",
          "provider_name": "Microsoft-Windows-Security-Auditing"
        },
        "ecs": {
          "version": "1.12.0"
        },
        "related": {
          "user": [
            "-",
            "XXXXXXXX"
          ]
        },
        "log": {
          "level": "information"
        },
        "host": {
          "name": "XXXXXX.XXXXXX.XXX"
        },
        "event": {
          "ingested": "2024-07-30T18:51:33.724955676Z",
          "code": "4732",
          "provider": "Microsoft-Windows-Security-Auditing",
          "created": "2024-07-30T18:51:33.147Z",
          "kind": "event",
          "action": "added-member-to-group",
          "category": [
            "iam"
          ],
          "type": [
            "group",
            "change"
          ],
          "outcome": "success"
        },
        "message": "A member was added to a security-enabled local group.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tXXXXXXXXX\n\tAccount Domain:\t\tXXXXXXXXXX\n\tLogon ID:\t\t0x3E7\n\nMember:\n\tSecurity ID:\t\tS-1-5-21-2366788312-814031040-1301217162-1086\n\tAccount Name:\t\t-\n\nGroup:\n\tSecurity ID:\t\tS-1-5-32-544\n\tGroup Name:\t\tAdministrators\n\tGroup Domain:\t\tBuiltin\n\nAdditional Information:\n\tPrivileges:\t\t-",
        "user": {
          "domain": "XXXXXXXXXX",
          "name": "XXXXXXXX",
          "id": "S-1-5-18",
          "target": {
            "name": "-",
            "group": {
              "domain": "Builtin",
              "name": "Administrators",
              "id": "S-1-5-32-544"
            }
          }
        },
        "tags": [
          "forwarded"
        ],
        "group": {
          "domain": "Builtin",
          "name": "Administrators",
          "id": "S-1-5-32-544"
        }
      }
    }
  ]
}

Result

...
         },
          "related": {
            "user_info": {
              "account_name": "brown",
              "sid": "S-1-5-32-544"
            },
            "user": [
              "-",
              "XXXXXXXX"
            ]
          },
14

OK. Let me update those fields and run another simulation here to see if I get the right answer.

15

So I updated the post to enter data into the custom index

POST lenovo_account_mappings/_doc
{
  "sid": "S-1-5-21-2366788312-814031040-1301217162-1086",
  "account_name": "Lenovo_tmp_ej9034hqwa"
}

I then updated the enrich alert events with the below to align with the data elements above.

PUT _ingest/pipeline/enrich_alert_events
{
  "processors": [
    {
      "enrich": {
        "policy_name": "lenovo_account_enrichment_policy",
        "field": "winlog.event_data.TargetSid",
        "target_field": "winlog.event_data.MemberName",
        "max_matches": "1"
      }
    }
  ]
}

I still get memberName equal to "-" when running the simulation

POST _ingest/pipeline/enrich_alert_events/_simulate
{
  "docs": [
    {
      "_source": {
        "agent": {
          "name": "XXXXX",
          "id": "82295858-5992-45e2-bce1-966e78fb8d42",
          "type": "winlogbeat",
          "ephemeral_id": "fd18652b-adeb-49b1-a2e2-97239f97df8a",
          "version": "8.0.0"
        },
        "@timestamp": "2024-07-30T18:50:59.567Z",
        "winlog": {
          "computer_name": "XXXX.XXXXX.XXXXXX",
          "process": {
            "pid": 672,
            "thread": {
              "id": 1096
            }
          },
          "keywords": [
            "Audit Success"
          ],
          "logon": {
            "id": "0x3e7"
          },
          "channel": "Security",
          "event_data": {
            "SubjectUserName": "XXXXXXX",
            "MemberSid": "S-1-5-21-2366788312-814031040-1301217162-1086",
            "TargetSid": "S-1-5-32-544",
            "SubjectDomainName": "XXXXXXXX",
            "SubjectLogonId": "0x3e7",
            "MemberName": "-",
            "TargetUserName": "Administrators",
            "TargetDomainName": "Builtin",
            "SubjectUserSid": "S-1-5-18",
            "PrivilegeList": "-"
          },
          "opcode": "Info",
          "record_id": "529303",
          "task": "Security Group Management",
          "event_id": "4732",
          "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
          "activity_id": "{41b78ff5-d8e1-0002-f890-b741e1d8da01}",
          "api": "wineventlog",
          "provider_name": "Microsoft-Windows-Security-Auditing"
        },
        "ecs": {
          "version": "1.12.0"
        },
        "related": {
          "user": [
            "-",
            "XXXXXXXX"
          ]
        },
        "log": {
          "level": "information"
        },
        "host": {
          "name": "XXXXXX.XXXXXX.XXX"
        },
        "event": {
          "ingested": "2024-07-30T18:51:33.724955676Z",
          "code": "4732",
          "provider": "Microsoft-Windows-Security-Auditing",
          "created": "2024-07-30T18:51:33.147Z",
          "kind": "event",
          "action": "added-member-to-group",
          "category": [
            "iam"
          ],
          "type": [
            "group",
            "change"
          ],
          "outcome": "success"
        },
        "message": """A member was added to a security-enabled local group.

Subject:
	Security ID:		S-1-5-18
	Account Name:		XXXXXXXXX
	Account Domain:		XXXXXXXXXX
	Logon ID:		0x3E7

Member:
	Security ID:		S-1-5-21-2366788312-814031040-1301217162-1086
	Account Name:		-

Group:
	Security ID:		S-1-5-32-544
	Group Name:		Administrators
	Group Domain:		Builtin

Additional Information:
	Privileges:		-""",
        "user": {
          "domain": "XXXXXXXXXX",
          "name": "XXXXXXXX",
          "id": "S-1-5-18",
          "target": {
            "name": "-",
            "group": {
              "domain": "Builtin",
              "name": "Administrators",
              "id": "S-1-5-32-544"
            }
          }
        },
        "tags": [
          "forwarded"
        ],
        "group": {
          "domain": "Builtin",
          "name": "Administrators",
          "id": "S-1-5-32-544"
        }
      }
    }
  ]
}
16

But I also don't think I have applied the enrichment policy to the ingestion pipelines yet.

17

I think my reply got deleted.

I updated my custom index with the event SID and MemberName

POST lenovo_account_mappings/_doc
{
  "sid": "S-1-5-21-2366788312-814031040-1301217162-1086",
  "account_name": "Lenovo_tmp_ej9034hqwa"
}

I then updated the enrich policy with the below to align with the proper field names.

PUT _ingest/pipeline/enrich_alert_events
{
  "processors": [
    {
      "enrich": {
        "policy_name": "lenovo_account_enrichment_policy",
        "field": "winlog.event_data.TargetSid",
        "target_field": "winlog.event_data.MemberName",
        "max_matches": "1"
      }
    }
  ]
}

When running the simulation, I still get a blank "-" MemberName field in the simulation

POST _ingest/pipeline/enrich_alert_events/_simulate
{
  "docs": [
    {
      "_source": {
        "agent": {
          "name": "XXXXX",
          "id": "82295858-5992-45e2-bce1-966e78fb8d42",
          "type": "winlogbeat",
          "ephemeral_id": "fd18652b-adeb-49b1-a2e2-97239f97df8a",
          "version": "8.0.0"
        },
        "@timestamp": "2024-07-30T18:50:59.567Z",
        "winlog": {
          "computer_name": "XXXX.XXXXX.XXXXXX",
          "process": {
            "pid": 672,
            "thread": {
              "id": 1096
            }
          },
          "keywords": [
            "Audit Success"
          ],
          "logon": {
            "id": "0x3e7"
          },
          "channel": "Security",
          "event_data": {
            "SubjectUserName": "XXXXXXX",
            "MemberSid": "S-1-5-21-2366788312-814031040-1301217162-1086",
            "TargetSid": "S-1-5-32-544",
            "SubjectDomainName": "XXXXXXXX",
            "SubjectLogonId": "0x3e7",
            "MemberName": "-",
            "TargetUserName": "Administrators",
            "TargetDomainName": "Builtin",
            "SubjectUserSid": "S-1-5-18",
            "PrivilegeList": "-"
          },
          "opcode": "Info",
          "record_id": "529303",
          "task": "Security Group Management",
          "event_id": "4732",
          "provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
          "activity_id": "{41b78ff5-d8e1-0002-f890-b741e1d8da01}",
          "api": "wineventlog",
          "provider_name": "Microsoft-Windows-Security-Auditing"
        },
        "ecs": {
          "version": "1.12.0"
        },
        "related": {
          "user": [
            "-",
            "XXXXXXXX"
          ]
        },
        "log": {
          "level": "information"
        },
        "host": {
          "name": "XXXXXX.XXXXXX.XXX"
        },
        "event": {
          "ingested": "2024-07-30T18:51:33.724955676Z",
          "code": "4732",
          "provider": "Microsoft-Windows-Security-Auditing",
          "created": "2024-07-30T18:51:33.147Z",
          "kind": "event",
          "action": "added-member-to-group",
          "category": [
            "iam"
          ],
          "type": [
            "group",
            "change"
          ],
          "outcome": "success"
        },
        "message": """A member was added to a security-enabled local group.

Subject:
	Security ID:		S-1-5-18
	Account Name:		XXXXXXXXX
	Account Domain:		XXXXXXXXXX
	Logon ID:		0x3E7

Member:
	Security ID:		S-1-5-21-2366788312-814031040-1301217162-1086
	Account Name:		-

Group:
	Security ID:		S-1-5-32-544
	Group Name:		Administrators
	Group Domain:		Builtin

Additional Information:
	Privileges:		-""",
        "user": {
          "domain": "XXXXXXXXXX",
          "name": "XXXXXXXX",
          "id": "S-1-5-18",
          "target": {
            "name": "-",
            "group": {
              "domain": "Builtin",
              "name": "Administrators",
              "id": "S-1-5-32-544"
            }
          }
        },
        "tags": [
          "forwarded"
        ],
        "group": {
          "domain": "Builtin",
          "name": "Administrators",
          "id": "S-1-5-32-544"
        }
      }
    }
  ]
}
18

Right because winlogbeat has its own ingest pipeline... you will need to add your enrich processor to the end of the OOTB pipeline...

So a couple questions..

Why 8.0.0 which is super old?

Why Winlogbeat maybe Elastic Agent would be better ...

Orr if you get on a newer version of the Stack and Winlogbeat there might be an easier way....

But if you just want to get it going add it to the bottom of the winlogbeat ingest pipeline (of course you will need to re-add if you upgrade)

You are posting too fast as a new user that is why you are getting throttled

19

_source

    "field": "winlog.event_data.TargetSid",

Lookup data

Not sure how you expect those to match... that values need to be the same for the match and enrich to happen

Looks like you used MemberSid .... you just need to get things lined up looks like your pipeline should be

"field": "winlog.event_data.MemberSid",

20

@FlyNavy

Did you get it working?