Extract Info from Field using Logstash

Elastic Stack Logstash
1

Hi
I have a field in ELK and I am using logstash v7 to extract the logs from Elastic to CSV and I have a field that part of it contains XML and Json how can I write a filter plugin that can get the XML Part or (JSON) it usually starts with Body:

 Method:POST
 query string:
 Browser: Unknown
 Headers:
 Cache-Control=no-cache
 Connection=Keep-Alive
 Pragma=no-cache
 Content-Length=667
 Content-Type=text/xml; charset=utf-8
 Accept=application/soap+xml, application/dime, multipart/related, text/
 Host=live.xxxxx.me
 User-Agent=Axis/1.4
 X_FORWARDED_FOR=xx.xx.xx.xx
SOAPAction=""
MS-ASPNETCORE-TOKEN=xxxxxxxxxxxxxxxxx
X-Original-Proto=http
X-Original-For=127.0.0.1:xxxxx
singularityheader=notxdetect=True
 ============================================================================
Body:
<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><notifySmsReception xmlns="http://www.csapi.org/schema/parlayx/sms/notification/v2_1/local"><correlator>tel99808885</correlator><message><message xmlns="">XXXXXX;XXXXXXXXX;XXXXXX;;D;SM_fce63667-4b04-48b5-8fa1-c26ed0277820;;T;1010</message><senderAddress xmlns="">tel:XXXXXXXXXXXX</senderAddress><smsServiceActivationNumber xmlns="">tel:XXXXXXXXXX</smsServiceActivationNumber></message></notifySmsReception></soapenv:Body></soapenv:Envelope>
========================================================================
2

If that is your [message] field then just pass it to an XML filter.

xml { store_xml => true source => "message" target => "someField" }

The filter will figure out where the XML starts and ends (provided you do not use xpath -- that requires the source to be valid XML).

3

thanks Badger it works but I can't extract any field from the result field as it is not valid xml or json

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.