XML Confusion

Elastic Stack Logstash
1

Hello,
I am trying to parse an xml file but am confused on the best way to tackle it. I see some folks using split to process and xml file and others using xpath. My xml file looks like the following:

<blah>
	<blahlist>
		<stuff>
			<Id>1234</Id>
			<Epoch>1488847761</Epoch>
		</stuff>
		<stuff>
			<Id>1235</Id>
			<Epoch>1499947761</Epoch>
		</stuff>
	</blahlist>
</blah>

I have tried a handful of variations for logstash, such as

input {
file {
path => "test.xml"
start_position => beginning
sincedb_path => "NUL"
codec => multiline
{
pattern => "<blah"
negate => true
what => "previous"
}
}
}
filter {
xml {
store_xml => "false"
source => "message"
xpath => ["/blah/blahlist/stuff/Id/text()", "Id"]
xpath => ["/blah/blahlist/stuff/Epoch/text()", "Epoch"]
force_array => "false"
}
mutate {
remove_field => 'message'
}
}

This however doesn't make a entry for each stuff. It puts each element from stuff into an array. Like such

{
"path" => "test.xml",
"@timestamp" => 2017-03-14T16:14:57.277Z,
"@version" => "1",
"host" => "Coompooter.local",
"Epoch" => [
[0] "1488847761",
[1] "1499947761"
],
"Id" => [
[0] "1234",
[1] "1235"
],
"tags" => [
[0] "multiline"
]
}

How can I read the xml and make it have an entry for each "stuff>" and "/stuff>" it reads. I think so it looks like:
{

      "path" => "test.xml",
"@timestamp" => 2017-03-14T16:14:57.277Z,
  "@version" => "1",
      "host" => "Coompooter.local",
     "Epoch" => ["1499947761"],
        "Id" => ["1235"],
      "tags" => [
    [0] "multiline"
]

}
{
"path" => "test.xml",
"@timestamp" => 2017-03-14T16:14:57.277Z,
"@version" => "1",
"host" => "Coompooter.local",
"Epoch" => ["1488847761"],
"Id" => ["1234"],
"tags" => [
[0] "multiline"
]
}

2

Write some code in a ruby filter to turn

"Epoch": [
  "1488847761",
  "1499947761"
],
"Id": [
  "1234",
  "1235"
],

into

"Whatever": [
  {
    "Epoch": "1488847761",
    "Id": "1234"
  },
  {
    "Epoch": "1488847761",
    "Id": "1235"
  }
]

then use a split filter to splice one such event into multiple events.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.