McAfee EPO logs to Logstash, XML parsing failure, XML: missing tag start, XML fails to generate datastructure

Purk · February 4, 2020, 2:02pm

Hi

I am looking for a help on how to parse the McAfee ePO XML logs into the logstash.
I tried to use a XML filter plugin, but it fails to create an actual datastructure of the XML file.

I have seen a similar topic in this forum, but there are still no solution on how to parse the XML content in order to create the datastructure.

Linux journalctl log show that XML plugin provide: "_xmlparsefailure" with the following message: "malformed XML: missing tag start":

...
#<REXML::ParseException: malformed XML: missing tag start
Feb 04 15:40:31 siem logstash[15122]: Line: 1
Feb 04 15:40:31 siem logstash[15122]: Position: 3204
Feb 04 15:40:31 siem logstash[15122]: Last 80 unconsumed characters:
Feb 04 15:40:31 siem logstash[15122]: <29>1 2020-02-04T13:40:44.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId>, :backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/parsers/baseparser.rb:375:in `pull_event'", "uri:classloader:
...

It seems that the problem is related to the first non-xml formatted line. I am looking for a workaround?

I try to parse the following log content in a way to create the datastructure
between <EPOevent> </EPOevent>

<29>1 2020-02-04T13:40:44.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId="1" bpsId="1" tenantGUID="{00000000-0000-0000-0000-000000000000}" tenantNodePath="1\2"] <?xml version="1.0" encoding="UTF-8"?>``<EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName="McAfee Endpoint Security" ProductVersion="10.7.0" ProductFamily="TVD"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3971.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-04T13:38:25</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-04T13:38:25Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\Program Files\Double Commander\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\kasutaja1</TargetUserName><TargetFileName>C:\Users\kasutaja1.HIN\Downloads\eicar.com</TargetFileName></CommonFields><CustomFields target="EPExtendedEventMT"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-03T06:20:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\Users\kasutaja1.HIN\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-04T13:38:25Z</TargetAccessTime><TargetCreateTime>2020-02-04T13:38:24Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\Users\kasutaja1.HIN\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\Program Files\Double Commander\doublecmd.exe|ThreatType=test|TargetUserName=HIN\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3971.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>

I am using the following conf file:

input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	}
}

filter {
    xml {
	source => "message"
	store_xml => true
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

Purk · February 4, 2020, 3:02pm

I was able to get rid of this "malformed XML: missing tag start" error message, by using the following configuration, but the XML plugin still failed to create a datastructure:

input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	type => "xml"
	codec => multiline {
	    pattern => "<EPOevent>"
	    negate => "true"
	    what => "next"
	    }
	}
}

filter {
    xml {
	source => "message"
	store_xml => false
	remove_namespaces => "true"
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

Journalctl shows the following message:

Feb 04 16:56:09 siem logstash[16285]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
Feb 04 16:56:09 siem logstash[16285]: {
Feb 04 16:56:09 siem logstash[16285]:           "type" => "xml",
Feb 04 16:56:09 siem logstash[16285]:           "port" => 54087,
Feb 04 16:56:09 siem logstash[16285]:        "message" => "<29>1 2020-02-04T14:56:19.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId=\"1\" bpsId=\"1\" tenantGUID=\"{00000000-0000-0000-0000-000000000000}\" tenantNodePath=\"1\\2\"] <?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3971.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-04T14:52:59</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-04T14:52:59Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-03T06:20:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-04T14:52:59Z</TargetAccessTime><TargetCreateTime>2020-02-04T14:52:46Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>13</DurationBeforeDetection><Natura",
Feb 04 16:56:09 siem logstash[16285]:     "@timestamp" => 2020-02-04T14:56:08.215Z,
Feb 04 16:56:09 siem logstash[16285]:       "@version" => "1",
Feb 04 16:56:09 siem logstash[16285]:           "host" => "192.168.10.30"
Feb 04 16:56:09 siem logstash[16285]: }
Feb 04 16:56:09 siem logstash[16285]: {
Feb 04 16:56:09 siem logstash[16285]:           "type" => "xml",
Feb 04 16:56:09 siem logstash[16285]:           "port" => 54086,

Badger · February 4, 2020, 5:11pm

Use mutate+gsub to remove everything before <?xml>.

When using store_xml => true the xml filter is very forgiving of random junk before the start of the XML (xpath is not). However, <29> looks like an XML element, so I suspect that is causing it to be less forgiving.

Purk · February 5, 2020, 8:59am

Thanks. I did as reccommended, I cussessfully deleted all the contents before <?xml....>, but the XML plugin still failed to generate the datastrusture.

input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	}
}

filter {
    mutate {
	gsub => [
	    "message", ".*(?=<\?xml)", ""
	
	]
    }
    xml {
	source => "message"
	store_xml => false
#	remove_namespaces => "true"
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

As you see, there are no errors listed in the journalctl:

Feb 05 10:51:19 siem logstash[27333]: {
Feb 05 10:51:19 siem logstash[27333]:     "@timestamp" => 2020-02-05T08:51:18.881Z,
Feb 05 10:51:19 siem logstash[27333]:           "port" => 56006,
Feb 05 10:51:19 siem logstash[27333]:       "@version" => "1",
Feb 05 10:51:19 siem logstash[27333]:        "message" => "<?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3972.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-05T08:51:06</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-05T08:51:06Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-04T06:26:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-05T08:51:06Z</TargetAccessTime><TargetCreateTime>2020-02-05T08:51:05Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\\Users\\kasutaja1.HIN\\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\\Program Files\\Double Commander\\doublecmd.exe|ThreatType=test|TargetUserName=HIN\\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3972.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
Feb 05 10:51:19 siem logstash[27333]:           "host" => "192.168.10.30"
Feb 05 10:51:19 siem logstash[27333]: }

The main problem is that I am unable to generate the datastructure:

Badger · February 5, 2020, 5:45pm

Change that from false to true. Also, you may want to set 'force_array => false'.

Purk · February 6, 2020, 10:23am

I tried all the following options several times with different combinations, but the XML plugin still fails to generate datastructure:

store_xml => "true|false"
force_array => "true|false"
force_content => "true|false"

There are no errors in the jurnalctl log, so I think that the problem is related to the XML plugin itself.

Feb 06 12:16:18 siem logstash[10593]: {
Feb 06 12:16:18 siem logstash[10593]:        "message" => "<?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3973.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-06T10:14:51</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-06T10:14:51Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-05T06:57:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-06T10:14:51Z</TargetAccessTime><TargetCreateTime>2020-02-06T10:14:40Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>11</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\\Users\\kasutaja1.HIN\\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\\Program Files\\Double Commander\\doublecmd.exe|ThreatType=test|TargetUserName=HIN\\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3973.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
Feb 06 12:16:18 siem logstash[10593]:     "@timestamp" => 2020-02-06T10:16:17.251Z,
Feb 06 12:16:18 siem logstash[10593]:       "@version" => "1",
Feb 06 12:16:18 siem logstash[10593]:           "port" => 53247,
Feb 06 12:16:18 siem logstash[10593]:           "host" => "192.168.10.30",
Feb 06 12:16:18 siem logstash[10593]:           "type" => "xml"
Feb 06 12:16:18 siem logstash[10593]: }

Are there any possibility to create datastructure of the XML?

Badger · February 6, 2020, 6:21pm

Not sure what the issue could be. When I run

input { generator { count => 1 lines => [ '<?xml version="1.0" encoding="UTF-8"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName="McAfee Endpoint Security" ProductVersion="10.7.0" ProductFamily="TVD"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3972.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-05T08:51:06</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-05T08:51:06Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\Program Files\Double Commander\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\kasutaja1</TargetUserName><TargetFileName>C:\Users\kasutaja1.HIN\Downloads\eicar.com</TargetFileName></CommonFields><CustomFields target="EPExtendedEventMT"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-04T06:26:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\Users\kasutaja1.HIN\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-05T08:51:06Z</TargetAccessTime><TargetCreateTime>2020-02-05T08:51:05Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\Users\kasutaja1.HIN\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\Program Files\Double Commander\doublecmd.exe|ThreatType=test|TargetUserName=HIN\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3972.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>^M' ] } }
filter {
    xml {
        source => "message"
        store_xml => true
        target => "doc"
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

I get

       "doc" => {
    "SoftwareInfo" => [
        [0] {
             "ProductFamily" => "TVD",
                     "Event" => [
                [0] {
                         "GMTTime" => [
                        [0] "2020-02-05T08:51:06"
                    ],
                        "Severity" => [
                        [0] "3"
                    ],
                         "EventID" => [
                        [0] "1278"
                    ],

etc.

Purk · February 13, 2020, 6:55am

I was able to make a workaround by using XPATH feature
In my case, the Logstash's XML plugin failed to parse the XML file

system · March 12, 2020, 6:55am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
XML encoded logs from Mcafee EPO logs to logstash Logstash	10	4556	November 24, 2017
Parsing McAfee ePO Logs Logstash	3	3020	May 9, 2017
How To remove \ from the existing log formate Logstash	4	345	February 28, 2020
XML filter issue Logstash	9	455	April 13, 2018
Error parsing xml with XmlSimple: Bad encoding name Logstash	3	997	September 14, 2020

McAfee EPO logs to Logstash, XML parsing failure, XML: missing tag start, XML fails to generate datastructure

Related topics