Need proper example and resource for xml filter plugin

Elastic Stack Kibana
1

I want to parse unstructured xml log data.I am not getting any proper resource for reference.The xml filter plugin documentation is not that able to sort my problem.
I am new to the ELK please help me out of this....

2

Hello Nikhil,

Welcome to the community :tada:

Although you added a Kibana tag I guess you mean the Logstash XML Filter Plugin?

Without knowing your problem we cannot help you find a solution. A good start would be to post your log data, what your pipeline looks like so far and to describe what you have difficulties with.

Best regards
Wolfram

3

This is my xml log file Its above 40mb but i made it short i am confused for how to made its config file .
<?xml version="1.0" encoding="UTF-8"?>

<logger class="org.jpos.util.DailyHostLogListener">
<log realm="AHC - ahc.1.APP" at="2022-12-12T09:28:52.043" lifespan="1ms">
  <info>
    rtsp-app-txnmgr:26205 prepare com.sar.QueryHttp 2022-12-12T09:28:52.041
    com.sar.cmon.httpClient.ApiClientConfig@77vg5678[name=ahc.1.APP,maxPerRoute=200,maxConnection=1000]
    {ahc.1.APP=[leased: 0; pending: 0; available: 1; max: 200], ahc.1.PSO-FIN=[leased: 0; pending: 0; available: 1; max: 200]}
  </info>
</log>
<log realm="AHC - ahc.1.APP" at="2022-12-12T09:28:52.518" lifespan="476ms">
  <info>
    rtsp-app-txnmgr:26205 prepare com.sarv.QueryHttp 2022-12-12T09:28:52.041
    {ahc.1.APP=[leased: 0; pending: 0; available: 1; max: 200], ahc.1.PSO-FIN=[leased: 0; pending: 0; available: 1; max: 200]}
    {ahc.1.APP=[leased: 0; pending: 0; available: 1; max: 200], ahc.1.PSO-FIN=[leased: 0; pending: 0; available: 1; max: 200]}
    <profiler>
      init [0.0/0.0]
      get-client [1.9/1.9]
      execute [444.8/396.5]
      end [0.0/285.6]
    </profiler>
  </info>
</log>
<log realm="org.jpos.send.testlogs" at="2022-12-12T09:28:52.524" lifespan="489ms">
  <commit>
    rtsp-app-txnmgr-89:idle:563253453
    <context>
      TXNNAME: APP.ReqCreateDevice
      POJO: org.test.798789797gxuaygs
      RAW_REQUEST:
       <![CDATA[
<token:ReqCreateDevice xmlns:token="http://jhvhkvk.org/token/schema/">
  <Head msgId="bhcjkshb" orgId="56856578" ts="2022-12-12T09:28:50" ver="2.0" />
  <ReqDetails type="Type2">
    <User>
      <DeviceInfo>
        <Tag name="DEVICEID" value="a0f1158jhvhjvbjh" />
        <Tag name="GEOCODE" value="123,456" />
        <Tag name="LOCATION" value="0.0,0.0" />
        <Tag name="IP" value="127.0.0.2" />
        <Tag name="TYPE" value="Android" />
        <Tag name="ID" value="a0f1158dcd457a42" />
        <Tag name="OS" value="Android" />
        <Tag name="CAPABILITY" value="1" />
        <Tag name="ISP" value="SIM2" />
        <Tag name="OSVERSION" value="AndroidVERSION" />
        <Tag name="MODEL" value="RMX1851" /> 
        <Tag name="MVERSION" value="30" />
      </DeviceInfo>
      <Strategies>
        <Strategy>
          <Detail name="isRooted" value="Not Rooted" />
        </Strategy>
      </Strategies>
    </User>
  </ReqDetails>
</token:ReqCreateDevice>
</log>

plz help me outPreformatted text``Preformatted text

4

Basically, you use Filebeat to read the XML file but as Filebeat is line-based and your XML file is not you have to configure the multiline options:

      multiline:
        pattern: '<log'
        negate: true
        match: after

Next, you create a logstash pipeline with a beats input to receive data from FileBeat. In the filters, you can parse the whole XML or read only specific fields using XPath expressions using the XML filter. The results can then be stoed in Elasticsearch.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.