Split XML array Logstash

Elastic Stack Logstash
1

I have data in the following format, repeating each time as follows:
< Data ID="test" ShortDescription="some description" type="this is a type">
< LongDescription>this is a long description< /LongDescription>
< Values>
< Value Set="some text" AnotherType="1234"/>
< /Values>
< Identification>
< ComparisonList>
< Comparison Name="test133" AdditionalValue="260"/>
< Comparison Name="test112" AdditionalValue="124"/>
< Comparison Name="test465" AdditionalValue="624"/>
< Comparison Name="test1234" AdditionalValue="612"/>
< /ComparisonList>
< /Identification>
< /Data>

I have used file as input with following configuration:
codec => multiline {
pattern => "<(Data)"
negate => true
what => "previous"

in filter I have:
filter {
xml {
source => "message"
store_xml => false
suppress_empty => false
xpath => [
"/Data/@ID", "ID",
"/Data/@ShortDescription", "ShortDescription",
"/Data/LongDescription/text()", "LongDescription"

This works for the simple unique fields. How to do the same for the Comparison name fields, since they repeat with same name? When I try:
"/Data/Identification/ComparisonList/@Name", "Name"

I only get the first value.
I have also tried the mutate to take it out of array, but doesnt work
mutate {
rename => { "[Name[0]" => "Name" }
rename => { "[Name[1]" => "Name1" }
rename => { "[Name[2]" => "Name2" }
rename => { "[Name[3]" => "Name3" }
}

2

Without doing a mutate, I can get all the fields as below in array:
"Name" => [
[0] "name1",
[1] "name2",
[2] "name3",
[3] "name4",
[4] "name5"
],
and
"Additional Value" => [
[0] "5",
[1] "1",
[2] "2",
[3] "4",
[4] "51"
]

How do I create fields such as Name1 and AdditionalField1, that has the correct values against the name from above?

3

I seem to almost be there, but not fully, with mutate +rename
mutate {
rename => { "[Name][0]" => "Name" }

In my Json, I can get some of the values from array into new fields.
I get the following:
{
"AdditionalValue2" => "81",
"AdditionalValue1" => "10",
"AdditionalValue3" => "25",
"Name" => [
[0] "testw",
[1] "testz"
],
"Name3" => "testsdd",
"AdditionalValue" => [
[0] "13",
[1] "201"
],
"Name1" => "test",
"Name2" => "test2"

Why do some new fields get generated and others stay in array?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.