Extract only certain tags from an XML using XML filter plugin

[HariniGajendran (Harini)]

Hi,

I am using XML filter to extract certain tags from an XML input. But the filter doesnt seem to be working correctly.

PFB sample XML.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://capabilities.nat.bt.com/xsd/ManageEventNotification/2006/12/30" xmlns:stan="http://wsi.nat.bt.com/2005/06/StandardHeader/" xmlns:even="http://capabilities.nat.bt.com/xsd/ManageEventNotification/2006/12/30/CCM/Events">
	<soapenv:Body>
		<ns:subscribe>
			<stan:standardHeader>
				<stan:e2e>
					<stan:E2EDATA>11=REQ,13=logicalQueue:Aib/ManageEventNotification/Men,15=AIB:app12192,16=logicalQueue:Aib/ManageEventNotification/Men,19=2,4=AIB:app12192-default,1.1=APP12192:4we4se4z07:4we4se7tzt,6=PCK002069,7=apl09704aibor2.n,8=UNKNOWN,9=aibbac,10=4we4se4z98</stan:E2EDATA>
				</stan:e2e>
				<stan:serviceState>
					<stan:stateCode>OK</stan:stateCode>
				</stan:serviceState>
				<stan:serviceAddressing>
					<stan:from>Aib</stan:from>
					<stan:to>
						<stan:address>http://capabilities.nat.bt.com/ManageEventNotification/2006/12/30</stan:address>
					</stan:to>
					<stan:replyTo>
						<stan:address>mq:queue/?destination=BT.AL.AIB.MEN_PUB_TO_AIB_RES_BAC.01@MqMgr</stan:address>
					</stan:replyTo>
					<stan:faultTo>
						<stan:address>mq:queue/?destination=BT.AL.AIB.MEN_PUB_TO_AIB_RES_BAC.01@MqMgr</stan:address>
					</stan:faultTo>
					<stan:messageId>AIBOR000034514</stan:messageId>
					<stan:serviceName>http://capabilities.nat.bt.com/ManageEventNotification</stan:serviceName>
					<stan:action>http://capabilities.nat.bt.com/ManageEventNotification/2006/12/30#subscribe</stan:action>
				</stan:serviceAddressing>
			</stan:standardHeader>
			<ns:subscribe>
				<even:laxTopicValidation>false</even:laxTopicValidation>
				<even:consumerReference>
					<even:consumerId>AIB</even:consumerId>
					<even:address>mq:queue/?destination=BT.AL.AIB.MEN_PUB_TO_AIB_RES_BAC.01@MqMgr</even:address>
				</even:consumerReference>
				<even:subscriptionPolicy>
					<even:expiryNotification>false</even:expiryNotification>
					<even:preExpiryNotification>false</even:preExpiryNotification>
					<even:advancedSubscriptionAcknowledgement>false</even:advancedSubscriptionAcknowledgement>
					<even:consumeExpiredMessages>false</even:consumeExpiredMessages>
					<even:eventRetention>false</even:eventRetention>
					<even:useRaw>false</even:useRaw>
				</even:subscriptionPolicy>
				<even:filter>
					<even:topicExpressionDialect>http://docs.oasis-open.org/wsn/t-1/TopicExpression/Full</even:topicExpressionDialect>
					<even:topicExpression>http://itprogrammes.intra.bt.com/topics/Openreach:/OPENREACH/ESB/KCI</even:topicExpression>
					<even:messageContentDialect>http://www.w3.org/TR/1999/REC-xpath-19991116</even:messageContentDialect>
					<even:messageContent>/SOAP-ENV:Envelope/SOAP-ENV:Body/x1:HubServiceMessage/x1:ServiceBusinessMessage/osscor:ListOfOrderCommittedKCI/osscor:OrderInfo/osscor:SellersOrderReference=&apos;2-104644416078&apos; or /SOAP-ENV:Envelope/SOAP-ENV:Body/x1:HubServiceMessage/x1:ServiceBusinessMessage/osscadk:ListOfOrderCancelledKCI/osscadk:OrderInfo/osscadk:SellersOrderReference=&apos;2-104644416078&apos; or SOAP-ENV:Envelope/SOAP-ENV:Body/llu:HubServiceMessage/llu:ServiceBusinessMessage/b2bKCI:ListOfOrder/b2bKCI:Order/b2bKCI:OrderNumber=&apos;2-104644416078&apos; or SOAP-ENV:Envelope/SOAP-ENV:Body/llu:HubServiceMessage/llu:ServiceBusinessMessage/b2bOutbound:ListOfOrder/b2bOutbound:Order/b2bOutbound:OrderNumber=&apos;2-104644416078&apos;</even:messageContent>
					<even:namespace>SOAP-ENV:http://schemas.xmlsoap.org/soap/envelope/</even:namespace>
					<even:namespace>xsi:http://www.w3.org/2001/XMLSchema-instance</even:namespace>
					<even:namespace>xsd:http://www.w3.org/2001/XMLSchema</even:namespace>
					<even:namespace>sh:http://wsi.nat.bt.com/2005/06/StandardHeader/</even:namespace>
					<even:namespace>m:http://capabilities.nat.bt.com/xsd/ManageEventNotification/2006/12/30</even:namespace>
					<even:namespace>m1:http://capabilities.nat.bt.com/xsd/ManageEventNotification/2006/12/30/CCM/Events</even:namespace>
					<even:namespace>ms1:http://ccm.intra.bt.com/manageServiceFault/Events/ACTION_REQUEST/2007/02/05</even:namespace>
					<even:namespace>x1:http://www.bt.com/eai/hub/or</even:namespace>
					<even:namespace>osscor:http://www.siebel.com/xml/OrderCommittedKCIEIO</even:namespace>
					<even:namespace>osscadk:http://www.siebel.com/xml/OrderStatusUpdateCancelledKCIEIO</even:namespace>
					<even:namespace>llu:http://www.bt.com/eai/hub/llu</even:namespace>
					<even:namespace>b2bKCI:http://www.siebel.com/xml/B2BOutboundKCI</even:namespace>
					<even:namespace>b2bOutbound:http://www.siebel.com/xml/B2BFulfilmentOutbound</even:namespace>
				</even:filter>
			</ns:subscribe>
		</ns:subscribe>
	</soapenv:Body>
</soapenv:Envelope>

PFB snippet from my logstash.conf

xml {
        source => "inputxml"
		target => "xml"
		store_xml => "false"
		force_array => "false"
		xpath => [
            "/xml/Body/subscribe/subscribe/consumerReference/address/text()", "even_address",
            "/xml/Body/subscribe/standardHeader/serviceAddressing/messageId/text()", "sub_messageId",
			"/xml/Body/unsubscribe/subscriptionReference/address/text()", "uuid",
            "/xml/Body/unsubscribe/standardHeader/serviceAddressing/messageId/text()", "unsub_messageId"
		]
	}

I would require only 4 fields from the above XML. TIA.

[Badger]

Try

    remove_namespaces => true
    xpath => [
        "/Envelope/Body/subscribe/subscribe/consumerReference/address/text()", "even_address",
        "/Envelope/Body/subscribe/standardHeader/serviceAddressing/messageId/text()", "sub_messageId",
        "/Envelope/Body/unsubscribe/subscriptionReference/address/text()", "uuid",
        "/Envelope/Body/unsubscribe/standardHeader/serviceAddressing/messageId/text()", "unsub_messageId"
    ]

[HariniGajendran (Harini)]

Thanks Badger.. This worked!!!