Xpath xml parsing problem

Elastic Stack Logstash
1

Hi,

I try to parse simple xml data by xml filter.
This is my multiline message read by filebeat:

[2017-06-28 09:26:31] Received message from localhost
[2017-06-28 09:26:31]
<?xml version="1.0" encoding="utf-8"?>
<doctestRequest>
  <customers>
    <customer>
      <number>123</number>
    </customer>
    <customer>
      <number>456</number>
    </customer>
    <customer>
      <number>789</number>
    </customer>
  </customers>
</doctestRequest>

I try to read all number values to array. So I'd like to get
number: ["123","456","789"] but xpath in xml filter doesn't work.

Below a attach configuration:

filebeat.input
- type: log
  enabled: true
  paths:
    - *****.log
  fields:
    WebServices: false
    WebApps: false
    doc_type: test
  fields_under_root: true
  multiline.pattern: 'Received message from localhost'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 5000

logstash config:

input {
  beats {
    port => 5044
  }
}

filter {


  xml {
    source => "message"
    store_xml => "true"
    target => "xml"
    xpath => [ "//number/text()","number" ]
    remove_namespaces => true
  }

}
output {
	stdout {
		codec => json
	}
}

logstash output from console:

{
  "offset": 0,
  "WebServices": false,
  "prospector": {
    "type": "log"
  },
  "doc_type": "test",
  "source": "***.log",
  "message": "[2017-06-28 09:26:31] Received message from localhost\n[2017-06-28 09:26:31]\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<doctestRequest>\n  <customers>\n    <customer>\n      <number>123</number>\n    </customer>\n    <customer>\n      <number>456</number>\n    </customer>\n    <customer>\n      <number>789</number>\n    </customer>\n  </customers>\n</doctestRequest>",
  "tags": [
    "beats_input_codec_plain_applied"
  ],
  "input": {
    "type": "log"
  },
  "@timestamp": "2018-09-11T23:02:17.178Z",
  "xml": {
    "customers": [
      {
        "customer": [
          {
            "number": [
              "123"
            ]
          },
          {
            "number": [
              "456"
            ]
          },
          {
            "number": [
              "789"
            ]
          }
        ]
      }
    ]
  },
  "@version": "1",
  "beat": {
    "name": "***",
    "hostname": "***",
    "version": "6.3.2"
  },
  "host": {
    "name": "***"
  },
  "WebApps": false
}

As you can see in output there is no number field. What I do wrong. Please help.

2

I would like to add additional info:
I use ELK in 6.3.2 version.
I tried a lot of xpath options. e.g
xpath => [
"//number","number1"
"//number/text()","number2"
"/doctestRequest","doctestRequest"
"/doctestRequest/customers","customers"
]
I think that xml xpath should add tag parsing exception if I configure wrong xpath.
Unfortunately I don't see any additional tags or errors in output.

I really need a help, please :slight_smile:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.