Extract Info from Field using Logstash

Hi
I have a field in ELK and I am using logstash v7 to extract the logs from Elastic to CSV and I have a field that part of it contains XML and Json how can I write a filter plugin that can get the XML Part or (JSON) it usually starts with Body:

 Method:POST
 query string:
 Browser: Unknown
 Headers:
 Cache-Control=no-cache
 Connection=Keep-Alive
 Pragma=no-cache
 Content-Length=667
 Content-Type=text/xml; charset=utf-8
 Accept=application/soap+xml, application/dime, multipart/related, text/
 Host=live.xxxxx.me
 User-Agent=Axis/1.4
 X_FORWARDED_FOR=xx.xx.xx.xx
SOAPAction=""
MS-ASPNETCORE-TOKEN=xxxxxxxxxxxxxxxxx
X-Original-Proto=http
X-Original-For=127.0.0.1:xxxxx
singularityheader=notxdetect=True
 ============================================================================
Body:
<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><notifySmsReception xmlns="http://www.csapi.org/schema/parlayx/sms/notification/v2_1/local"><correlator>tel99808885</correlator><message><message xmlns="">XXXXXX;XXXXXXXXX;XXXXXX;;D;SM_fce63667-4b04-48b5-8fa1-c26ed0277820;;T;1010</message><senderAddress xmlns="">tel:XXXXXXXXXXXX</senderAddress><smsServiceActivationNumber xmlns="">tel:XXXXXXXXXX</smsServiceActivationNumber></message></notifySmsReception></soapenv:Body></soapenv:Envelope>
========================================================================

If that is your [message] field then just pass it to an XML filter.

xml { store_xml => true source => "message" target => "someField" }

The filter will figure out where the XML starts and ends (provided you do not use xpath -- that requires the source to be valid XML).

thanks Badger it works but I can't extract any field from the result field as it is not valid xml or json

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.