How would I parse further the syslog5424_msg in a syslog event in logstash conf?

Elastic Stack Logstash
1

Hello,

I have parsed the XML log format coming from a syslog source as defined in logstash config file. What shall be done further to parse the syslog message field within the XML?

Below is the config:
input
{
tcp
{
mode => "server"
host => "0.0.0.0"
port => 6514
ssl_enable => true
ssl_verify => false
ssl_cert => "/opt/logstash/ssl/logstash-remote.crt"
ssl_key => "/opt/logstash/ssl/logstash-remote.key"
}
}

filter {
grok {
match => { "message" => '%{SYSLOG5424LINE}' }
}
mutate {
remove_field => ["message"]
remove_field => ["syslog5424_sd"]
remove_field => ["@timestamp"]
rename => { "syslog5424_ts" => "Timestamp" }
rename => { "syslog5424_host" => "Source_Host" }
rename => { "syslog5424_msgid" => "Message_ID" }
rename => { "syslog5424_app" => "Type" }
gsub => ["syslog5424_msg",'"','"']

}

xml {
source => "syslog5424_msg"
store_xml => false
}
}

output

{
elasticsearch {
hosts => ["x.x.x.x:9200"]
index => "index-%{+YYYY.MM.dd}"
}
}

and here is the json log that I am trying to parse further:
{
"_index": "index-2021.06.15",
"_type": "_doc",
"_id": "QGCoDnoBQqtAWexfwnd-",
"_version": 1,
"_score": null,
"_source": {
"host": "x.x.x.x",
"syslog5424_msg": "<?xml version=\"1.0\" encoding=\"utf-8\"?>{d5599ed8-6520-11eb-1f73-00090faa0001}xyzABCDEFGHx.x.x.x1.1.1.116Windows-480<McAfeeCommonUpdater ProductName="product Agent" ProductVersion="5.0.0" ProductFamily="TVD">123402021-06-15T07:06:13Whatever202004090AMCore4467.0syslogagent3000UpdateTaskAnythingN/A \r",
"syslog5424_pri": "29",
"syslog5424_ver": "1",
"port": 56648,
"Type": "Events",
"Source_Host": "syslog server",
"@version": "1",
"Message_ID": "EventFwd",
"@timestamp": "2021-06-15T07:52:58.582Z",
"Timestamp": "2021-06-15T07:52:58.0Z"
},
"fields": {
"@timestamp": [
"2021-06-15T07:52:58.582Z"
],
"X_Timestamp": [
"2021-06-15T07:52:58.000Z"
]
},
"sort": [
1623743578582
]
}

2

If store_xml is false and you are not using xpath what do you expect this to do?

3

Ok so store_xml has to be true and would you please help me to give XPATH syntax applied to my data below(just one line should be really helpful :slight_smile: ) say extracting IP address and it's value. Here's is the XML of syslog message if needed:

<?xml version="1.0" encoding="utf-8"?>
	<UpdateEvents>
		<MachineInfo>
			<AgentGUID>{cfe0cba8-5f98-11eb-31fd-04ea56651f3a}</AgentGUID>
			<MachineName>HOSTNAME</MachineName>
			<RawMACAddress>04EAAJBBS6788GG</RawMACAddress>
			<IPAddress>x.x.x.x</IPAddress>
			<AgentVersion>9.9.10.19</AgentVersion>
			<OSName>Windows 10</OSName>
			<TimeZoneBias>-480</TimeZoneBias>
			<UserName>Mrs. abc</UserName>
		</MachineInfo>
		<McAfeeCommonUpdater ProductName="McAfee Agent" ProductVersion="5.0.0" ProductFamily="TVD">
			<UpdateEvent>
				<EventID>2401</EventID>
				<Severity>0</Severity>
				<GMTTime>2021-06-14T04:05:23</GMTTime>
				<ProductID>AMCORDAT2000</ProductID>
				<Locale>0409</Locale>
				<Error>0</Error>
				<Type>AMCore</Type>
				<Version>4466.0</Version>
				<InitiatorID>EPOAGENT3000</InitiatorID>
				<InitiatorType>UpdateTask</InitiatorType>
				<SiteName>McAfeeHttp</SiteName>
				<Description>N/A</Description>
			</UpdateEvent>
		</McAfeeCommonUpdater>
	</UpdateEvents>

Thank you,
Devashish Singh

4

There are multiple examples of using xpath in this forum. This for example.

5

Well I have been through a lot before but this seems almost matching my own scenario.

Thanks @Badger , I will keep it open.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.