# XML encoded logs from Mcafee EPO logs to logstash

**URL:** <https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380>\
**Category:** Logstash\
**Created:** [October 18, 2017, 11:05am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380 "2017-10-18T11:05:48Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 18, 2017, 11:05am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/1 "2017-10-18T11:05:49Z")

</div>

Hi my scenario is we need to collect logs from Mcafee EPO and send to our third party cloud logging platform.We have logstash server in between and its receiving logs from EPO and forwarding to logging platform.However mcafee is sending syslogs encoded with xml and xml is not supported by cloud logging platform.  
I am prettty new to logstash can we do some filtering or xml to syslog conversion on logstash to achieve this.?

My Config:-elk-5.5.0-0

input {  
tcp {  
port =\> 514  
ssl\_cert =\> 'C:/Bitnami/elk-5.5.0-0/logstash/ssl/syslogselfsigned.crt'  
ssl\_key =\> 'C:/Bitnami/elk-5.5.0-0/logstash/ssl/syslogselfsigned.key'  
ssl\_enable =\> true  
ssl\_verify =\> false  
}  
}

output {  
elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
}

syslog {  
host =\> "10.x.x.x"  
port =\> 514  
}  
}

Sample message:-1:  
\<29\>1 2017-10-13T06:45:34.0Z SERVERMCAFEE01 EPOEvents - EventFwd [agentInfo@3401 tenantId="1"] \<?xml version="1.0" encoding="utf-8"?\>{3ecdcaaa-45f5-11e7-0a6c-0050569e0a71}SERVER10050569E500110.x.x.x5.0.6.220Linux420root242742017-10-13T06:45:34ENDP\_GS\_1020LYNX040983Property CollectionN/AEPOAGENT3000N/AN/A

---

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 18, 2017, 11:14am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/2 "2017-10-18T11:14:00Z")

</div>

Even if we can covert xml to json using logstash and forward it to cloud logging platform, it will be helpful.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 18, 2017, 8:27pm UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/3 "2017-10-18T20:27:43Z")

</div>

I was about to say that you could use an xml filter to parse the XML, but the example you posted isn't valid XML. I suggest you just use a grok filter to parse the message.

---

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 23, 2017, 6:45am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/4 "2017-10-23T06:45:38Z")

</div>

Hi Magnus,

Thanks for your quick reply, how about below message ( I am more interested in this message, so that I can create some alerting for virus detection).

```
<29>1 2017-10-06T11:21:16.0Z SERVERMCAFEE01 EPOEvents - EventFwd [agentInfo@3401 tenantId="1"] <?xml version="1.0" encoding="UTF-8"?><EPOevent><MachineInfo><MachineName>TS01</MachineName><AgentGUID>{768ab1ba-31aa-11e7-2f62-00505696c547}</AgentGUID><IPAddress>10.x.x.x</IPAddress><OSName>Windows 2008 R2</OSName><UserName>SYSTEM</UserName><TimeZoneBias>420</TimeZoneBias><RawMACAddress></RawMACAddress></MachineInfo><SoftwareInfo ProductName="McAfee Endpoint Security" ProductVersion="10.5.0" ProductFamily="TVD"><CommonFields><Analyzer>ENDP_AM_1050</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.5.0</AnalyzerVersion><AnalyzerHostName>TS01</AnalyzerHostName><AnalyzerEngineVersion>5900.7806</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Demand Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3125.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2017-10-06T11:17:35</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2017-10-06T11:17:35Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>TS01</SourceHostName><SourceProcessName>On-Demand Scan</SourceProcessName><TargetHostName>TS01</TargetHostName><TargetUserName>CASAA\mohab</TargetUserName><TargetFileName>C:\Users\mohab\Desktop\dell.com</TargetFileName></CommonFields><CustomFields target="EPExtendedEventMT"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2017-10-04T15:46:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>False</ThreatDetectedOnCreation><TargetName>dell.com</TargetName><TargetPath>C:\Users\mohab\Desktop</TargetPath><TargetHash>01c65da52c8f3e5959ce9bb405ba6372</TargetHash><TargetFileSize>70</TargetFileSize><TargetModifyTime>2017-10-06T11:16:03Z</TargetModifyTime><TargetAccessTime>2017-10-06T11:16:03Z</TargetAccessTime><TargetCreateTime>2017-10-06T11:16:03Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_ODS_TASK_NAME_RIGHT_CLICK</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>92</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_ODS_DETECTION_DELETED|TargetName=dell.com|TargetPath=C:\Users\mohab\Desktop|ThreatName=EICAR test file|ThreatType=test|TaskName=IDS_ODS_TASK_NAME_RIGHT_CLICK|TargetUserName=CASAA\mohab</NaturalLangDescription><AccessRequested></AccessRequested><AMCoreContentVersion>3125.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>

```

I earlier modified logstash conf but that did not help.

input {  
tcp {  
port =\> 514  
ssl\_cert =\> 'C:/Bitnami/elk-5.5.0-0/logstash/ssl/syslogselfsigned.crt'  
ssl\_key =\> 'C:/Bitnami/elk-5.5.0-0/logstash/ssl/syslogselfsigned.key'  
ssl\_enable =\> true  
ssl\_verify =\> false  
}  
}

filter {  
xml {  
source =\> "message"  
store\_xml =\> false  
}  
}

output {  
elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
}

syslog {  
host =\> "10.x.x.x"  
port =\> 514  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 23, 2017, 1:09pm UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/5 "2017-10-23T13:09:13Z")

</div>

Use a grok filter to extract timestamp and other parts of the message to separate fields. One of the fields should contain the XML payload and that's the field you then reference in your xml filter.

---

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 26, 2017, 11:48am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/6 "2017-10-26T11:48:44Z")

</div>

Hi Magnus,

I tried grok and there is a pattern ready for this sort of messages, %{SYSLOG5424LINE} this pattern extracted all the fields and "syslog5424\_msg" is the field which is mapped to the valid xml payload.

But during the grok process its adding backlash infront of double quotes turning xml content in to an invalid xml, at this point I think that is the reason why xml filtering is not working.I tried to replace the backslash but that is not working?

```
filter {
  grok {
    match => { "message" => '%{SYSLOG5424LINE}' }
  }
mutate {
       gsub => ["syslog5424_msg",'\"','"']
    }
xml {
source => "syslog5424_msg"
store_xml => false
}
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 26, 2017, 1:00pm UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/7 "2017-10-26T13:00:12Z")

</div>

> But during the grok process its adding backlash infront of double quotes turning xml content in to an invalid xml

No, I don't think that's what's happening. Show an example event, e.g. by copy/pasting from Kibana's JSON tab or using a `stdout { codec => rubydebug }` output.

---

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 26, 2017, 4:27pm UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/8 "2017-10-26T16:27:11Z")

</div>

Uploaded json in a file as I cant post it in comment, it exceeded 7000 characters 🙄 .

Scratch that I am not authorized to upload JSON.

---

<div class="post-metadata">

**Author:** ![junaid](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@junaid](https://discuss.elastic.co/u/junaid)\
**Post date:** [October 26, 2017, 4:28pm UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/9 "2017-10-26T16:28:29Z")

</div>

```
{
  "_index": "logstash-2017.10.26",
  "_type": "logs",
  "_id": "AV9YXpGL9BeeCRxNMOe1",
  "_version": 1,
  "_score": 9.158171,
  "_source": {
    "syslog5424_sd": "[agentInfo@3401 tenantId=\"1\"]",
    "syslog5424_ver": "1",
    "message": "<29>1 2017-10-26T11:08:31.0Z SERVERMCAFEE01 EPOEvents - EventFwd [agentInfo@3401 tenantId=\"1\"] <?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>SERVERTS01</MachineName><AgentGUID>{768ab1ba-31aa-11e7-2f62-00505696c547}</AgentGUID><IPAddress>10.0.0.2</IPAddress><OSName>Windows 2008 R2</OSName><UserName>SYSTEM</UserName><TimeZoneBias>420</TimeZoneBias><RawMACAddress></RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.5.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1050</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.5.0</AnalyzerVersion><AnalyzerHostName>SERVERTS01</AnalyzerHostName><AnalyzerEngineVersion>5900.7806</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3146.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2017-10-26T11:04:59</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2017-10-26T11:04:59Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>SERVERTS01</SourceHostName><SourceProcessName>C:\\Windows\\System32\\notepad.exe</SourceProcessName><TargetHostName>SERVERTS01</TargetHostName><TargetUserName>CASAASNP\\testuser</TargetUserName><TargetFileName>C:\\Users\\testuser\\Desktop\\bahu.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2017-10-25T16:32:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>bahu.com</TargetName><TargetPath>C:\\Users\\testuser\\Desktop</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2017-10-26T11:04:59Z</TargetModifyTime><TargetAccessTime>2017-10-26T11:04:59Z</TargetAccessTime><TargetCreateTime>2017-10-26T11:04:59Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>0</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=bahu.com|TargetPath=C:\\Users\\testuser\\Desktop|ThreatName=EICAR test file|SourceProcessName=C:\\Windows\\System32\\notepad.exe|ThreatType=test|TargetUserName=CASAASNP\\testuser</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3146.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
    "syslog5424_app": "EPOEvents",
    "syslog5424_msg": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><EPOevent><MachineInfo><MachineName>SERVERTS01</MachineName><AgentGUID>{768ab1ba-31aa-11e7-2f62-00505696c547}</AgentGUID><IPAddress>10.0.0.2</IPAddress><OSName>Windows 2008 R2</OSName><UserName>SYSTEM</UserName><TimeZoneBias>420</TimeZoneBias><RawMACAddress></RawMACAddress></MachineInfo><SoftwareInfo ProductName=\"McAfee Endpoint Security\" ProductVersion=\"10.5.0\" ProductFamily=\"TVD\"><CommonFields><Analyzer>ENDP_AM_1050</Analyzer><AnalyzerName>McAfee Endpoint Security</AnalyzerName><AnalyzerVersion>10.5.0</AnalyzerVersion><AnalyzerHostName>SERVERTS01</AnalyzerHostName><AnalyzerEngineVersion>5900.7806</AnalyzerEngineVersion><AnalyzerDetectionMethod>On-Access Scan</AnalyzerDetectionMethod><AnalyzerDATVersion>3146.0</AnalyzerDATVersion></CommonFields><Event><EventID>1278</EventID><Severity>3</Severity><GMTTime>2017-10-26T11:04:59</GMTTime><CommonFields><ThreatCategory>av.detect</ThreatCategory><ThreatEventID>1278</ThreatEventID><ThreatSeverity>2</ThreatSeverity><ThreatName>EICAR test file</ThreatName><ThreatType>test</ThreatType><DetectedUTC>2017-10-26T11:04:59Z</DetectedUTC><ThreatActionTaken>IDS_ALERT_ACT_TAK_DEL</ThreatActionTaken><ThreatHandled>True</ThreatHandled><SourceHostName>SERVERTS01</SourceHostName><SourceProcessName>C:\\Windows\\System32\\notepad.exe</SourceProcessName><TargetHostName>SERVERTS01</TargetHostName><TargetUserName>CASAASNP\\testuser</TargetUserName><TargetFileName>C:\\Users\\testuser\\Desktop\\bahu.com</TargetFileName></CommonFields><CustomFields target=\"EPExtendedEventMT\"><BladeName>IDS_BLADE_NAME_SPB</BladeName><AnalyzerContentCreationDate>2017-10-25T16:32:00Z</AnalyzerContentCreationDate><AnalyzerGTIQuery>False</AnalyzerGTIQuery><ThreatDetectedOnCreation>True</ThreatDetectedOnCreation><TargetName>bahu.com</TargetName><TargetPath>C:\\Users\\testuser\\Desktop</TargetPath><TargetHash>44d88612fea8a8f36de82e1278abb02f</TargetHash><TargetFileSize>68</TargetFileSize><TargetModifyTime>2017-10-26T11:04:59Z</TargetModifyTime><TargetAccessTime>2017-10-26T11:04:59Z</TargetAccessTime><TargetCreateTime>2017-10-26T11:04:59Z</TargetCreateTime><Cleanable>False</Cleanable><TaskName>IDS_OAS_TASK_NAME</TaskName><FirstAttemptedAction>IDS_ALERT_THACT_ATT_CLE</FirstAttemptedAction><FirstActionStatus>False</FirstActionStatus><SecondAttemptedAction>IDS_ALERT_THACT_ATT_DEL</SecondAttemptedAction><SecondActionStatus>True</SecondActionStatus><AttackVectorType>4</AttackVectorType><DurationBeforeDetection>0</DurationBeforeDetection><NaturalLangDescription>IDS_NATURAL_LANG_OAS_DETECTION_DEL|TargetName=bahu.com|TargetPath=C:\\Users\\testuser\\Desktop|ThreatName=EICAR test file|SourceProcessName=C:\\Windows\\System32\\notepad.exe|ThreatType=test|TargetUserName=CASAASNP\\testuser</NaturalLangDescription><AccessRequested></AccessRequested><DetectionMessage>IDS_OAS_DEFAULT_THREAT_MESSAGE</DetectionMessage><AMCoreContentVersion>3146.0</AMCoreContentVersion></CustomFields></Event></SoftwareInfo></EPOevent>\r",
    "syslog5424_msgid": "EventFwd",
    "@timestamp": "2017-10-26T11:08:33.141Z",
    "port": 63866,
    "syslog5424_ts": "2017-10-26T11:08:31.0Z",
    "syslog5424_pri": "29",
    "@version": "1",
    "host": "10.0.0.1",
    "syslog5424_host": "SERVERMCAFEE01"
  },
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 27, 2017, 5:02am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/10 "2017-10-27T05:02:40Z")

</div>

That looks perfectly normal. The double quotes in the XML are of course escaped when they're inside a double-quoted string.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2017, 5:03am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380/11 "2017-11-24T05:03:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.