Xml parser with xpath giving wrong result in xml filter plugin

Elastic Stack Logstash
1

Hi All

i am having a xml filter, which looks like this

 xml {
          source => "message"
          target => "message_parsed"
          add_tag => ["xml_parsed"]
          remove_namespaces => true
		  store_xml => true
		  force_array => true
          xpath => [
            "/table/row/number/text()","number"
            ]
     }

Here i am getting only 1 record. I am expecting 5 records
Under the column "number" i am getting value as comma seperated
123,124,125,126,127

My expectation as 5 different events

**Number**
123
124
125
126
127

I have tried split filter but no luck. I think there can be better solution for this

2

Please give an example input document. Post it as preformatted text so the XML doesn't get mangled.

3

Hi Here is the input XML, below is the block of 1 record.
Like this have 2000 records with .......... 

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
   <S:Body>
      <ns0:advancedSearchResponse xmlns:ns0="http://xmlns.xyz.com/Objects/V1">
         <response>
            <messageId xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
            <messageName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
            <statusCode>SUCCESS</statusCode>
            <table>
               <tableIdentifier>
                  <classId xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  <className xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  <objectId>6151127</objectId>
                  <objectName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  <tableId>-102</tableId>
                  <tableName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  <tableDisplayName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
               </tableIdentifier>
               <row rowId="1">
                  <objectReferentId>
                     <classId>2468022</classId>
                     <className>BondWire</className>
                     <classDisplayName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                     <objectId>6118882</objectId>
                     <objectName xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                     <objectVersion xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                     <version xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  </objectReferentId>
                  <additionalRowInfo xsi:nil="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
                  <number attributeId="1001" xsi:type="xs:string" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">22</number>
                  <Type attributeId="1081" xsi:type="common:ListEntryType" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:common="http://xmlns.xyz.com/Objects/Core/Common/V1">
                     <listName xsi:nil="true"/>
                     <selection>
                        <id>2468022</id>
                        <apiName>BondWire</apiName>
                        <value>Bond Wire</value>
                     </selection>
                  </Type>
                  <description attributeId="1002" xsi:type="xs:string" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Lifecycle Phase Testing Part, Testing Comma, OKay</description>
                  <phase attributeId="1084" xsi:type="common:ListEntryType" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:common="http://xmlns.xyz.com/Objects/Core/Common/V1">
                     <listName xsi:nil="true"/>
                     <selection>
                        <id>2481963</id>
                        <apiName>ACTIVE_PHASE</apiName>
                        <value>Active Phase</value>
                     </selection>
                  </phase>
               </row>
            </table>
         </response>
      </ns0:advancedSearchResponse>
   </S:Body>
</S:Envelope>

Here is my xml filter

 xml {
          source => "message"
          target => "message_parsed"
          add_tag => ["xml_parsed"]
          remove_namespaces => true
		  store_xml => true
		  force_array => true
          xpath => [
            "/Envelope/Body/advancedSearchResponse/response/table/row/number/text()","number",
			"/Envelope/Body/advancedSearchResponse/response/table/row/type/selection/value/text()","type",
			"/Envelope/Body/advancedSearchResponse/response/table/row/description/text()","description",
			"/Envelope/Body/advancedSearchResponse/response/table/row/phase/selection/value/text()","phase"
			
            ]
     }

Output is my ES

The response which i get is

number : 22,23,24,25,....
type:  Bond Wire,abc, def, ghi....
description: desc1, desc2......
phase: phase1,phase2.... etc.

I need to split this in multiple events. Which is not working currently.
Any idea how to do this?

4

Hi @magnusbaeck

Any suggestion as how can i achieve this

Thanks

5

Seeking suggestions on this. I feel this is very common problem. Many might have faced similar situation.
Please provide suggestions

6

See this ongoing thread:

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.