Winlogbeats not exporitng event.id or xml data

Elastic Stack Beats
1

I have events from a server being sent to logstash, but no eventids or xml data is being passed.
Below are examples of the winlogbeats config and the logs I'm getting. It's sending all events, but information I would need to filter (drop) a packet is missing. I tried to include the full XML data because I felt that could possibly contain the information I need, but I'm not getting that either. Any thoughts?

winlogbeat.event_logs:

  • name: Application
    include_xml: true
    ignore_older: 72h
  • name: Security
    include_xml: true
  • name: System
    include_xml: true

Here is what a log output looks like:

2018-06-07T18:45:21.482Z la-security-6 The Windows Time service entered the running state.

or:

2018-06-07T18:44:29.856Z la-security-6 Permissions on an object were changed.

Subject:
Security ID: S-1-5-18
Account Name: LA-SECURITY-6$
Account Domain: KINO
Logon ID: 0x3E7

Object:
Object Server: Security
Object Type: Token
Object Name: -
Handle ID: 0x234

Process:
Process ID: 0x20c
Process Name: C:\Windows\System32\services.exe

Permissions Change:
Original Security Descriptor: D:(A;;GA;;;SY)(A;;GA;;;LS)
New Security Descriptor:

2

Winlogbeat always sends the event_id. The output you posted looks like the content of the message field only. Winlogbeat events contain data like this:

{
  "@timestamp": "2015-01-13T09:33:15.812Z",
  "beat": {
    "hostname": "vagrant-2012-r2",
    "name": "vagrant-2012-r2"
  },
  "computer_name": "vagrant-2012-r2",
  "count": 1,
  "event_data": {
    "LogonType": "8",
    "TargetDomainName": "VAGRANT-2012-R2",
    "TargetLogonId": "0x837f2",
    "TargetUserName": "vagrant",
    "TargetUserSid": "S-1-5-21-3541430928-2051711210-1391384369-1001"
  },
  "event_id": 4634,
  "keywords": [
    "Audit Success"
  ],
  "level": "Information",
  "log_name": "Security",
  "message": "An account was logged off.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-21-3541430928-2051711210-1391384369-1001\n\tAccount Name:\t\tvagrant\n\tAccount Domain:\t\tVAGRANT-2012-R2\n\tLogon ID:\t\t0x837F2\n\nLogon Type:\t\t\t8\n\nThis event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.",
  "opcode": "Info",
  "process_id": 452,
  "provider_guid": "{54849625-5478-4994-A5BA-3E3B0328C30D}",
  "record_number": "193",
  "source_name": "Microsoft-Windows-Security-Auditing",
  "task": "Logoff",
  "thread_id": 484,
  "type": "wineventlog"
}

Perhaps there's an issue with your Logstash configuration that's causing you to not get all of the event's data? Can you please share your Logstash configuration. It should look similar to this.

If you are planning to drop events then you should consider doing it inside of Winlogbeat. Either use event_id or a drop_event processor.

3

This is my current config. I am currently not using elasticsearch since my goal is to have logrythm ingest these directly. I have been told that if I ingest into elasticsearch then maybe logrythm can read from there, but I'm not that far along yet. Right now I'm just trying to get the correct content to output to a log file.

Am I missing something?

input {
beats{
port => "5044"
}
}

output {
file {
path => "/logs/servers-%{@timestamp}.log"
codec => "line"
}
}

4

The line code writes the message field. Try a codec like codec => "json_lines" if you want all of the structured data from the event to be logged.

5

Andrew, this was huge. I'm now getting the entire log. It's not as structured as the output you provided above, but I'm assuming I can use grok now to structure it?

6

Can you share what you are seeing? It should have a very similar set of fields to what I posted earlier.

Depending on the application that logged the individual event log record the number of fields under event_data can vary. But for a lot of cases grok is not required for Windows event logs because the data is already structured.

Here are some example LS pipelines for processing Winlogbeat data. You'll notice there are very few cases that actually require grok. Most of the filters are for normalizing the events (e.g. renaming fields that are inconsistent across events). https://github.com/Cyb3rWard0g/HELK/tree/master/helk-logstash/pipeline

7

{"activity_id":"{A00D4CFC-FC44-0003-104D-0DA044FCD301}","tags":["beats_input_codec_plain_applied"],"log_name":"Security","level":"Information","beat":{"version":"6.2.4","name":"n/A","hostname":"n/A"},"event_id":4624,"process_id":796,"event_data":{"TargetDomainName":"NT AUTHORITY","IpPort":"-","ProcessId":"0x30c","RestrictedAdminMode":"-","VirtualAccount":"%%1843","LogonGuid":"{00000000-0000-0000-0000-000000000000}","TargetLogonId":"0x3e7","KeyLength":"0","TargetOutboundUserName":"-","SubjectDomainName":"-","TargetUserSid":"S-1-5-18","ImpersonationLevel":"%%1833","TargetLinkedLogonId":"0x0","WorkstationName":"-","SubjectLogonId":"0x3e7","ElevatedToken":"%%1842","SubjectUserSid":"S-1-5-18","TargetUserName":"SYSTEM","SubjectUserName":"n/A","LogonProcessName":"Advapi ","AuthenticationPackageName":"Negotiate","TargetOutboundDomainName":"-","LogonType":"5","IpAddress":"-","LmPackageName":"-","ProcessName":"C:\Windows\System32\services.exe","TransmittedServices":"-"},"@timestamp":"2018-06-08T21:50:47.448Z","type":"wineventlog","source_name":"Microsoft-Windows-Security-Auditing","thread_id":7164,"message":"An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\N/A$\n\tAccount Domain:\t\NA\n\tLogon ID:\t\t0x3E7\n\nLogon Information:\n\tLogon Type:\t\t5\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tSYSTEM\n\tAccount Domain:\t\tNT AUTHORITY\n\tLogon ID:\t\t0x3E7\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x30c\n\tProcess Name:\t\tC:\Windows\System32\services.exe\n\nNetwork Information:\n\tWorkstation Name:\t-\n\tSource Network Address:\t-\n\tSource Port:\t\t-\n\nDetailed Authentication Information:\n\tLogon Process:\t\tAdvapi \n\tAuthentication Package:\tNegotiate\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\t-\n\tKey Length:\t\t0\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.","host":"N/A","record_number":"1850314","task":"Logon","provider_guid":"{54849625-5478-4994-A5BA-3E3B0328C30D}","version":2,"@version":"1","computer_name":"N/A","opcode":"Info","keywords":["Audit Success"]}

8 
{
   "activity_id":"{A00D4CFC-FC44-0003-104D-0DA044FCD301}",
   "tags":[
      "beats_input_codec_plain_applied"
   ],
   "log_name":"Security",
   "level":"Information",
   "beat":{
      "version":"6.2.4",
      "name":"n/A",
      "hostname":"n/A"
   },
   "event_id":4624,
   "process_id":796,
   "event_data":{
      "TargetDomainName":"NT AUTHORITY",
      "IpPort":"-",
      "ProcessId":"0x30c",
      "RestrictedAdminMode":"-",
      "VirtualAccount":"%%1843",
      "LogonGuid":"{00000000-0000-0000-0000-000000000000}",
      "TargetLogonId":"0x3e7",
      "KeyLength":"0",
      "TargetOutboundUserName":"-",
      "SubjectDomainName":"-",
      "TargetUserSid":"S-1-5-18",
      "ImpersonationLevel":"%%1833",
      "TargetLinkedLogonId":"0x0",
      "WorkstationName":"-",
      "SubjectLogonId":"0x3e7",
      "ElevatedToken":"%%1842",
      "SubjectUserSid":"S-1-5-18",
      "TargetUserName":"SYSTEM",
      "SubjectUserName":"n/A",
      "LogonProcessName":"Advapi ",
      "AuthenticationPackageName":"Negotiate",
      "TargetOutboundDomainName":"-",
      "LogonType":"5",
      "IpAddress":"-",
      "LmPackageName":"-",
      "ProcessName":"C:\Windows\System32\services.exe",
      "TransmittedServices":"-"
   },
   "@timestamp":"2018-06-08T21:50:47.448Z",
   "type":"wineventlog",
   "source_name":"Microsoft-Windows-Security-Auditing",
   "thread_id":7164,
   "message":"An account was successfully logged on.

Subject:
	Security ID:		S-1-5-18
	Account Name:	N/A$
	Account Domain:	NA
	Logon ID:		0x3E7

Logon Information:
	Logon Type:		5
	Restricted Admin Mode:	-
	Virtual Account:		No
	Elevated Token:		Yes

Impersonation Level:		Impersonation

New Logon:
	Security ID:		S-1-5-18
	Account Name:		SYSTEM
	Account Domain:		NT AUTHORITY
	Logon ID:		0x3E7
	Linked Logon ID:		0x0
	Network Account Name:	-
	Network Account Domain:	-
	Logon GUID:		{00000000-0000-0000-0000-000000000000}

Process Information:
	Process ID:		0x30c
	Process Name:		C:WindowsSystem32services.exe

Network Information:
	Workstation Name:	-
	Source Network Address:	-
	Source Port:		-

Detailed Authentication Information:
	Logon Process:		Advapi 
	Authentication Package:	Negotiate
	Transited Services:	-
	Package Name (NTLM only):	-
	Key Length:		0

This event is generated when a logon session is created. It is generated on the computer that was accessed.

The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The impersonation level field indicates the extent to which a process in the logon session can impersonate.

The authentication information fields provide detailed information about this specific logon request.
	- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
	- Transited services indicate which intermediate services have participated in this logon request.
	- Package name indicates which sub-protocol was used among the NTLM protocols.
	- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",
   "host":"N/A",
   "record_number":"1850314",
   "task":"Logon",
   "provider_guid":"{54849625-5478-4994-A5BA-3E3B0328C30D}",
   "version":2,
   "@version":"1",
   "computer_name":"N/A",
   "opcode":"Info",
   "keywords":[
      "Audit Success"
   ]
}

Looks like it contains lots of detail outside of message. Hopefully grok'ing won't be required.

9

Is there a parameter or plugin in the config that you used to structure it?

10

It's already fully structured inside of Logstash. I just formatted the JSON output you posted.

Inside of Logstash you can access fields for doing conditional processing. See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html. For example:

filters {
  if [event_id] == 4624 {
    mutate {
      rename => {
          "[event_data][IpAddress]" => "src_ip"
      }    
    }
}
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.