Developing Custom Parser for Logs


In my installation, the output of Logstash is set to Azure Sentinel. The installation is working fine. However, the client prefers to send logs from inhouse applications to Sentinel via Syslog.

Is it possible to build a GROK that completely relies on Regex? I plan to build a regex like below.

Sample Event: node=host234 type=CRED_ACQ msg=audit(1660312102.483:3553546): pid=122262 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="test" exe="/usr/sbin/sshd" hostname= addr= terminal=ssh res=success'

Sample Regex: node=(\S+)\s+type=(\S+)...

Is this a feasible solution? If no, what can be alternate solution? Please advise.

Thanks in advance


Why use grok instead of a kv filter?

Hello @Badger,
Thank you for your input.

However, in my scenario, multiple devices are sending events over syslog on the same port, and I am not sure if KV filter will be successful. Below are the sample events.

Event 1:
<182>Aug 17 16:50:40 audispd: node=lnvfr99720091 type=CRYPTO_KEY_USER msg=audit(1660747840.906:61384532): user pid=18722 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=session fp=? direction=both spid=18725 suid=74 rport=51578 laddr= lport=22 exe="/usr/sbin/sshd" hostname=? addr= terminal=? res=success'

Event 2:
{"EventTime":"2022-08-17 17:06:55","Hostname":"","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":4624,"SourceName":"Microsoft-Windows-Security-Auditing","ProviderGuid":"{54849625-5478-4994-A5BA-3E3B0328C30D}","Version":1,"Task":12544,"OpcodeValue":0,"RecordNumber":1375740,"ProcessID":528,"ThreadID":6136,"Channel":"Security","Message":"An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\t-\r\n\tAccount Domain:\t\t-\r\n\tLogon ID:\t\t0x0\r\n\r\nLogon Type:\t\t\t3\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tS-1-5-21-2867169748-1450568154-1061018960-34832\r\n\tAccount Name:\t\tF16\r\n\tAccount Domain:\t\tOFFICE\r\n\tLogon ID:\t\t0x861AB2D7\r\n\tLogon GUID:\t\t{0EF828A3-8F00-7FE0-72ED-24C6FABA27C3}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x0\r\n\tProcess Name:\t\t-\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\t-\r\n\tSource Network Address:\t1.2.2.1\r\n\tSource Port:\t\t57856\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tKerberos\r\n\tAuthentication Package:\tKerberos\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\t-\r\n\tKey Length:\t\t0\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.","Category":"Logon","Opcode":"Info","SubjectUserSid":"S-1-0-0","SubjectUserName":"-","SubjectDomainName":"-","SubjectLogonId":"0x0","TargetUserSid":"S-1-5-21-2867169748-1450568154-1061018960-34832","TargetUserName":"F6","TargetDomainName":"OFFICE","TargetLogonId":"0x861ab2d7","LogonType":"3","LogonProcessName":"Kerberos","AuthenticationPackageName":"Kerberos","WorkstationName":"-","LogonGuid":"{0EF828A3-8F00-7FE0-72ED-24C6FABA27C3}","TransmittedServices":"-","LmPackageName":"-","KeyLength":"0","ProcessName":"-","IpAddress":"","IpPort":"57856","ImpersonationLevel":"%%1833","EventReceivedTime":"2022-08-17 17:06:56","SourceModuleName":"i_dc","SourceModuleType":"im_msvistalog"}

Please advise.

Thanks and Regards,

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.