Unable to split Arrays into fields -parsing XML Attributes of an unknown number of child elements

Elastic Stack Logstash
1

Hi,
I'm trying to parse some XML content using Logstash. I'm trying to get the attributes as separate fields. Each <CardData> element will have some attributes that need to be parsed as fields and an unknown number of <ListData> child nodes each with their attributes, which also need to be parsed as fields.

This works easily using XPath for the first level - the ID attribute of <CardData>. but when there's more than one <ListData> element, I get an array that I want to split into separate fields - example below.

And in some cases, the child nodes could have one more layer of children as shown in the
<Results> element.

I'm not sure how to proceed.

Sample Input Data:

<CardData ID="C1">
	<ListData SeriesNumber="1" Type="S" ID="A1"/>
	<ListData SeriesNumber="1" Type="S" ID="A2"/>	
	<ListData SeriesNumber="5" Type="H"  ID="A4">
		<Results SeriesNumber="5.1" Name="AA" ID="R1"/>
		<Results SeriesNumber="5.2" Name="Mono" ID="R2"/>
	</ListData>
</CardData>

My XPath from the logstash.conf is as follows.

xml
	{
		source => "message"
		store_xml => false		#default is true
		force_array => false	#default is true
		remove_namespaces => true
		#XPath Parser for LogMessages
		xpath =>
		[
            "/CardData/@ID", "carddata.id",
            "/CardData/ListData/@SeriesNumber", "carddata.listdata.seriesnumber",
            "/CardData/ListData/@Type", "carddata.listdata.type",
            "/CardData/ListData/@ID", "carddata.listdata.id",
        ]
}

Expected Output

carddata.id  - C1
carddata.listdata1.seriesnumber - 1
carddata.listdata1.type - S
carddata.listdata1.id - A1
carddata.listdata2.seriesnumber - 1
carddata.listdata2.type - S
carddata.listdata2.id - A2
carddata.listdata3.seriesnumber - 5
carddata.listdata3.type - H
carddata.listdata3.id - A4

Actual Output in Logstash:

"carddata.id" => "C1",
"carddata.listdata.seriesnumber" => [
        [0] "1",
        [1] "1",
        [2] "5"
    ],
            "carddata.listdata.type" => [
        [0] "S",
        [1] "S",
        [2] "H"
    ],
            "carddata.listdata.id " => [
        [0] "A1",
        [1] "A2",
        [2] "A4"
    ]

Actual Output in Kibana:

carddata.id  - C1
carddata.listdata.seriesnumber - 1 , 1 , 5
carddata.listdata.type - S , S , H
carddata.listdata.id - A1, A2, A4

I've tried using the split function within the mutate filter but that has yielded no results.
Any help would be appreciated. Thanks.

3

You could use something like this.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.