# McAfee EPO logs to Logstash, XML parsing failure, XML: missing tag start, XML fails to generate datastructure

**URL:** <https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827>\
**Category:** Logstash\
**Created:** [February 4, 2020, 2:02pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827 "2020-02-04T14:02:39Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Purk](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Purk](https://discuss.elastic.co/u/Purk)\
**Post date:** [February 4, 2020, 2:02pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/1 "2020-02-04T14:02:40Z")

</div>

Hi

I am looking for a help on how to parse the McAfee ePO XML logs into the logstash.  
I tried to use a XML filter plugin, but it fails to create an actual datastructure of the XML file.

I have seen a similar topic in this forum, but there are still no solution on how to parse the XML content in order to create the datastructure.

> [@XML encoded logs from Mcafee EPO logs to logstash](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/5):
>
> Use a grok filter to extract timestamp and other parts of the message to separate fields. One of the fields should contain the XML payload and that's the field you then reference in your xml filter.

Linux journalctl log show that XML plugin provide: "\_xmlparsefailure" with the following message: "malformed XML: missing tag start":

...  
#\<REXML::ParseException: malformed XML: missing tag start  
Feb 04 15:40:31 siem logstash[15122]: Line: 1  
Feb 04 15:40:31 siem logstash[15122]: Position: 3204  
Feb 04 15:40:31 siem logstash[15122]: Last 80 unconsumed characters:  
Feb 04 15:40:31 siem logstash[15122]: \<29\>1 2020-02-04T13:40:44.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId\>, :backtrace=\>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rexml/parsers/baseparser.rb:375:in `pull\_event'", "uri:classloader:  
...

It seems that the problem is related to the first non-xml formatted line. I am looking for a workaround?

I try to parse the following log content in a way to create the datastructure  
between `<EPOevent> </EPOevent>`

`<29>1 2020-02-04T13:40:44.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId="1" bpsId="1" tenantGUID="{00000000-0000-0000-0000-000000000000}" tenantNodePath="1\2"] <?xml version="1.0" encoding="UTF-8"?>``<EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName="McAfee Endpoint Security" ProductVersion="10.7.0" ProductFamily="TVD"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3971.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-04T13:38:25</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-04T13:38:25Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\Program Files\Double Commander\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\kasutaja1</TargetUserName><TargetFileName>C:\Users\kasutaja1.HIN\Downloads\eicar.com</TargetFileName></CommonFields><CustomFields target="EPExtendedEventMT"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-03T06:20:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\Users\kasutaja1.HIN\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-04T13:38:25Z</TargetAccessTime><TargetCreateTime>2020-02-04T13:38:24Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\Users\kasutaja1.HIN\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\Program Files\Double Commander\doublecmd.exe|ThreatType=test|TargetUserName=HIN\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3971.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>`

I am using the following conf file:

```
input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	}
}

filter {
    xml {
	source => "message"
	store_xml => true
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

```

---

<div class="post-metadata">

**Author:** ![Purk](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Purk](https://discuss.elastic.co/u/Purk)\
**Post date:** [February 4, 2020, 3:02pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/2 "2020-02-04T15:02:37Z")

</div>

I was able to get rid of this "malformed XML: missing tag start" error message, by using the following configuration, but the XML plugin still failed to create a datastructure:

```
input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	type => "xml"
	codec => multiline {
	    pattern => "<EPOevent>"
	    negate => "true"
	    what => "next"
	    }
	}
}

filter {
    xml {
	source => "message"
	store_xml => false
	remove_namespaces => "true"
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

```

Journalctl shows the following message:

```
Feb 04 16:56:09 siem logstash[16285]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
Feb 04 16:56:09 siem logstash[16285]: {
Feb 04 16:56:09 siem logstash[16285]: "type" => "xml",
Feb 04 16:56:09 siem logstash[16285]: "port" => 54087,
Feb 04 16:56:09 siem logstash[16285]: "message" => "<29>1 2020-02-04T14:56:19.0Z HIN-DC EPOEvents - EventFwd [agentInfo@3401 tenantId=\"1\" bpsId=\"1\" tenantGUID=\"{00000000-0000-0000-0000-000000000000}\" tenantNodePath=\"1\\2\"] <?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3971.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-04T14:52:59</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-04T14:52:59Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-03T06:20:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-04T14:52:59Z</TargetAccessTime><TargetCreateTime>2020-02-04T14:52:46Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>13</DurationBeforeDetection><Natura",
Feb 04 16:56:09 siem logstash[16285]: "@timestamp" => 2020-02-04T14:56:08.215Z,
Feb 04 16:56:09 siem logstash[16285]: "@version" => "1",
Feb 04 16:56:09 siem logstash[16285]: "host" => "192.168.10.30"
Feb 04 16:56:09 siem logstash[16285]: }
Feb 04 16:56:09 siem logstash[16285]: {
Feb 04 16:56:09 siem logstash[16285]: "type" => "xml",
Feb 04 16:56:09 siem logstash[16285]: "port" => 54086,
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 4, 2020, 5:11pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/3 "2020-02-04T17:11:00Z")

</div>

Use mutate+gsub to remove everything before \<?xml\>.

When using store\_xml =\> true the xml filter is very forgiving of random junk before the start of the XML (xpath is not). However, \<29\> looks like an XML element, so I suspect that is causing it to be less forgiving.

---

<div class="post-metadata">

**Author:** ![Purk](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Purk](https://discuss.elastic.co/u/Purk)\
**Post date:** [February 5, 2020, 8:59am UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/4 "2020-02-05T08:59:33Z")

</div>

Thanks. I did as reccommended, I cussessfully deleted all the contents before \<?xml....\>, but the XML plugin still failed to generate the datastrusture.

```
input {
    tcp {
	port => 6514
	ssl_cert => '/etc/logstash/conf.d/syslogselfsigned.crt'
	ssl_key => '/etc/logstash/conf.d/syslogselfsigned.key'
	ssl_enable => true
	ssl_verify => false
	}
}

filter {
    mutate {
	gsub => [
	    "message", ".*(?=<\?xml)", ""
	
	]
    }
    xml {
	source => "message"
	store_xml => false
#	remove_namespaces => "true"
	target => "doc"
    }
}

output {
    elasticsearch {
	hosts => ["127.0.0.1:9200"]
	index => "epo-%{+YYYY.MM.dd}"
    }
    stdout {
	codec => rubydebug
    }
}

```

As you see, there are no errors listed in the journalctl:

```
Feb 05 10:51:19 siem logstash[27333]: {
Feb 05 10:51:19 siem logstash[27333]: "@timestamp" => 2020-02-05T08:51:18.881Z,
Feb 05 10:51:19 siem logstash[27333]: "port" => 56006,
Feb 05 10:51:19 siem logstash[27333]: "@version" => "1",
Feb 05 10:51:19 siem logstash[27333]: "message" => "<?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3972.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-05T08:51:06</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-05T08:51:06Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-04T06:26:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-05T08:51:06Z</TargetAccessTime><TargetCreateTime>2020-02-05T08:51:05Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\\Users\\kasutaja1.HIN\\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\\Program Files\\Double Commander\\doublecmd.exe|ThreatType=test|TargetUserName=HIN\\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3972.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
Feb 05 10:51:19 siem logstash[27333]: "host" => "192.168.10.30"
Feb 05 10:51:19 siem logstash[27333]: }

```

The main problem is that I am unable to generate the datastructure:

 ![Capture_xml1](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0f4e92fe8beb19c8c832d3ee0086e152ad2c46bd.png)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 5, 2020, 5:45pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/5 "2020-02-05T17:45:28Z")

</div>

> [@Purk](#):
>
> store\_xml =\> false

Change that from false to true. Also, you may want to set 'force\_array =\> false'.

---

<div class="post-metadata">

**Author:** ![Purk](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Purk](https://discuss.elastic.co/u/Purk)\
**Post date:** [February 6, 2020, 10:23am UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/6 "2020-02-06T10:23:09Z")

</div>

I tried all the following options several times with different combinations, but the XML plugin still fails to generate datastructure:

```
store_xml => "true|false"
force_array => "true|false"
force_content => "true|false"

```

There are no errors in the jurnalctl log, so I think that the problem is related to the XML plugin itself.

```
Feb 06 12:16:18 siem logstash[10593]: {
Feb 06 12:16:18 siem logstash[10593]: "message" => "<?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.7.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3973.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-06T10:14:51</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-06T10:14:51Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\\Program Files\\Double Commander\\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\\kasutaja1</TargetUserName><TargetFileName>C:\\Users\\kasutaja1.HIN\\Downloads\\eicar.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-05T06:57:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\\Users\\kasutaja1.HIN\\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-06T10:14:51Z</TargetAccessTime><TargetCreateTime>2020-02-06T10:14:40Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>11</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\\Users\\kasutaja1.HIN\\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\\Program Files\\Double Commander\\doublecmd.exe|ThreatType=test|TargetUserName=HIN\\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3973.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
Feb 06 12:16:18 siem logstash[10593]: "@timestamp" => 2020-02-06T10:16:17.251Z,
Feb 06 12:16:18 siem logstash[10593]: "@version" => "1",
Feb 06 12:16:18 siem logstash[10593]: "port" => 53247,
Feb 06 12:16:18 siem logstash[10593]: "host" => "192.168.10.30",
Feb 06 12:16:18 siem logstash[10593]: "type" => "xml"
Feb 06 12:16:18 siem logstash[10593]: }

```

Are there any possibility to create datastructure of the XML?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 6, 2020, 6:21pm UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/7 "2020-02-06T18:21:04Z")

</div>

Not sure what the issue could be. When I run

```
input { generator { count => 1 lines => ['<?xml version="1.0" encoding="UTF-8"?><EPOevent><MachineInfo><MachineName>DESKTOP1</MachineName><AgentGUID>{5d900e98-436e-11ea-0edc-080027784c43}</AgentGUID><IPAddress>192.168.10.50</IPAddress><OSName>Windows 10 Workstation</OSName><UserName>SYSTEM</UserName><TimeZoneBias>-120</TimeZoneBias><RawMACAddress>083327862222</RawMACAddress></MachineInfo><SoftwareInfo ProductName="McAfee Endpoint Security" ProductVersion="10.7.0" ProductFamily="TVD"><CommonFields><Analyzer>ENDP_AM_1070</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.7.0</AnalyzerVersion><AnalyzerHostName>DESKTOP1</AnalyzerHostName><AnalyzerEngineVersion>6010.8670</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3972.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2020-02-05T08:51:06</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2020-02-05T08:51:06Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>DESKTOP1</SourceHostName><SourceProcessName>C:\Program Files\Double Commander\doublecmd.exe</SourceProcessName><TargetHostName>DESKTOP1</TargetHostName><TargetUserName>HIN\kasutaja1</TargetUserName><TargetFileName>C:\Users\kasutaja1.HIN\Downloads\eicar.com</TargetFileName></CommonFields><CustomFields target="EPExtendedEventMT"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2020-02-04T06:26:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>eicar.com</TargetName><TargetPath>C:\Users\kasutaja1.HIN\Downloads</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2000-05-24T16:07:00Z</TargetModifyTime><TargetAccessTime>2020-02-05T08:51:06Z</TargetAccessTime><TargetCreateTime>2020-02-05T08:51:05Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>1</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=eicar.com|TargetPath=C:\Users\kasutaja1.HIN\Downloads|ThreatName=EICAR test file|SourceProcessName=C:\Program Files\Double Commander\doublecmd.exe|ThreatType=test|TargetUserName=HIN\kasutaja1</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3972.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>^M'] } }
filter {
    xml {
        source => "message"
        store_xml => true
        target => "doc"
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

```

I get

```
       "doc" => {
    "SoftwareInfo" => [
        [0] {
             "ProductFamily" => "TVD",
                     "Event" => [
                [0] {
                         "GMTTime" => [
                        [0] "2020-02-05T08:51:06"
                    ],
                        "Severity" => [
                        [0] "3"
                    ],
                         "EventID" => [
                        [0] "1278"
                    ],

```

etc.

---

<div class="post-metadata">

**Author:** ![Purk](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Purk](https://discuss.elastic.co/u/Purk)\
**Post date:** [February 13, 2020, 6:55am UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/8 "2020-02-13T06:55:26Z")

</div>

I was able to make a workaround by using XPATH feature 🙂  
In my case, the Logstash's XML plugin failed to parse the XML file ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 12, 2020, 6:55am UTC](https://discuss.elastic.co/t/mcafee-epo-logs-to-logstash-xml-parsing-failure-xml-missing-tag-start-xml-fails-to-generate-datastructure/217827/9 "2020-03-12T06:55:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.