Xml filter Xpath not Working

Elastic Stack Logstash
1

Hello,

I have the following below XML:

<AUDITENTRIES>
	<REFERENCENUMBER>CUSTINQ2104001</REFERENCENUMBER>
	<CHANNELNAME>MOB</CHANNELNAME>
	<REFERENCENUMCONSUMER>abcfinance:referenceNumConsumer</REFERENCENUMCONSUMER>
	<UTIMESTAMP>21/04/2021  3:10:37</UTIMESTAMP>
	<SERVICENAME>GPAY.CUST.INQUIRY</SERVICENAME>
	<REQUESTTIME>21/04/2021  3:20:16 PM</REQUESTTIME>
	<RESPONSETIME>21/04/2021  3:23:16 PM</RESPONSETIME>
	<REQUESTDATA> <?xml version="1.0" encoding="UTF-8"?>
		<abcfinance:eAI_MESSAGE xmlns:abcfinance="urn:abcfinance.com/gbo/xml/schemas/v1_0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:abcfinance.com/gbo/xml/schemas/v1_0/EAI.xsd ">
			<abcfinance:eAI_HEADER>
				<abcfinance:serviceName>GPAY.CUST.INQUIRY</abcfinance:serviceName>
				<abcfinance:serviceType>SYNC</abcfinance:serviceType>
				<abcfinance:serviceVersion>1</abcfinance:serviceVersion>
				<abcfinance:client>GPAY</abcfinance:client>
				<abcfinance:clientChannel>MOB</abcfinance:clientChannel>
				<abcfinance:msgChannel>MQ</abcfinance:msgChannel>
				<abcfinance:requestorLanguage>E</abcfinance:requestorLanguage>
				<abcfinance:securityInfo>
					<abcfinance:authentication>
						<abcfinance:UserId>abcfinance:UserId</abcfinance:UserId>
						<abcfinance:Password>abcfinance:Password</abcfinance:Password>
					</abcfinance:authentication>
				</abcfinance:securityInfo>
			</abcfinance:eAI_HEADER>
		</abcfinance:eAI_MESSAGE>
	</REQUESTDATA>
</AUDITENTRIES>

my logstash configuration:

input 
{
  jms {
        yaml_file => "jms.yml"
        yaml_section => "wmq" 
        include_header => true
        include_properties => true
		include_body => true
		use_jms_timestamp => true
		destination => "DEV.QUEUE.1"
        interval => 3
        runner => asyn
  }
}
filter{
    xml{
		source => "message"
		store_xml => false
			xpath => [
				"/AUDITENTRIES/REFERENCENUMBER/text()", "referencenumber",
				"/AUDITENTRIES/CHANNELNAME/text()", "channelname",
				"/AUDITENTRIES/REQUESTDATA/text()", "requestdata"
			]
		}
		mutate { remove_field => [ "message"] }
	}
	
output 
{
    stdout { 
		codec => rubydebug {}
	}
}

my rubydebug output is

{
          "JMS_IBM_PutDate" => "20210603",
               "@timestamp" => 2021-06-03T09:42:58.940Z,
              "channelname" => [
        [0] "MOB"
    ],
           "jms_message_id" => "ID:414d512067617465776179716d202020ed494e5f02aec120",
          "referencenumber" => [
        [0] "CUSTINQ2104001"
    ],
    "JMS_IBM_Character_Set" => "UTF-8",
          "JMS_IBM_MsgType" => 8,
}

requestdata is missing. please help me to solve this.

My expected output llike

{
          "JMS_IBM_PutDate" => "20210603",
               "@timestamp" => 2021-06-03T09:42:58.940Z,
              "channelname" => [
        [0] "MOB"
    ],
           "jms_message_id" => "ID:414d512067617465776179716d202020ed494e5f02aec120",
          "referencenumber" => [
        [0] "CUSTINQ2104001"
    ],
	"requestdata" => [
        [0] "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<abcfinance:eAI_MESSAGE xmlns:abcfinance=\"urn:abcfinance.com/gbo/xml/schemas/v1_0/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"urn:abcfinance.com/gbo/xml/schemas/v1_0/ EAI.xsd \"><abcfinance:eAI_HEADER><abcfinance:serviceName>GPAY.CUST.INQUIRY</abcfinance:serviceName><abcfinance:serviceType>SYNC</abcfinance:serviceType><abcfinance:serviceVersion>1</abcfinance:serviceVersion><abcfinance:client>GPAY</abcfinance:client><abcfinance:clientChannel>MOB</abcfinance:clientChannel><abcfinance:msgChannel>MQ</abcfinance:msgChannel><abcfinance:requestorLanguage>E</abcfinance:requestorLanguage><abcfinance:securityInfo><abcfinance:authentication><abcfinance:UserId>abcfinance:UserId</abcfinance:UserId><abcfinance:Password>abcfinance:Password</abcfinance:Password></abcfinance:authentication></abcfinance:securityInfo></abcfinance:eAI_HEADER></abcfinance:eAI_MESSAGE>"
    "JMS_IBM_Character_Set" => "UTF-8",
          "JMS_IBM_MsgType" => 8,
}

Thank you

2

Now I changed the Xpath configuration in my logstash.conf. It's working fine.

my configuration changes.

filter{
    xml{
		source => "message"
		store_xml => false
			xpath => [
				"/AUDITENTRIES/REFERENCENUMBER/text()", "referencenumber",
				"/AUDITENTRIES/CHANNELNAME/text()", "channelname",
				"/AUDITENTRIES/REQUESTDATA/*", "requestdata"
			]
		}
		mutate { remove_field => [ "message"] }
	}

Thank you

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.