Help with Logstash xml filter plug in

[byoungman (Bill Youngman)]

I'm having trouble parsing out fields from my xml file using the logstash xml filter plugin.

Here is my xml file data--


733812
JVM|Garbage Collection|Number of Minor Collections Per Min
Application Infrastructure Performance|OSP Servers|JVM|Garbage Collection|Number of Minor Collections Per Min
ONE_MIN


1579723620000
0
0
0
0
true
30
0
0
0

And this is my logstash.conf set up--
input {
file {
path => "C:/AppDynamics/Metrics-Data/*.xml"
type => "xml"
codec => multiline {
pattern => "\r\n"
negate => "true"
what => "previous"
}
mode => "read"
file_completed_action => "delete"
add_field => {
"es_index" => "appdynamics-metrics"
}
}
}

filter
{
	xml {
		source => "message"
		target => "theXML"
		store_xml => "false"
    xpath => ["//metric-datas/metric-data/metricId/text()","metricId"]
	}
}

output {
	elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "%{[es_index]}-%{+YYYY.MM.dd}"
	}
}

My end goal is to create fields for all of the metric-data attributes.

What am I missing here as I can not get it to work. Attached is my Kibana output.

TIA,
Bill Youngman

[byoungman (Bill Youngman)]

The xml data didn't come across correctly formatted so attaching it as an image.

[Badger]

Use markdown to protect the XML from being interpreted as HTML. Edit your post, select the XML and click on </> in the toolbar above the edit pane. Then do the same with your configuration.

[byoungman (Bill Youngman)]

Thanks for the suggestion Badger

[byoungman (Bill Youngman)]

<metric-datas>
  <metric-data>
    <metricId>733812</metricId>
    <metricName>JVM|Garbage Collection|Number of Minor Collections Per Min</metricName>
    <metricPath>Application Infrastructure Performance|OSP Servers|JVM|Garbage Collection|Number of Minor Collections Per Min</metricPath>
    <frequency>CUSTOM</frequency>
    <metricValues>
      <metric-value>
        <startTimeInMillis>1579805400000</startTimeInMillis>
        <occurrences>16</occurrences>
        <current>0</current>
        <min>0</min>
        <max>0</max>
        <useRange>true</useRange>
        <count>32</count>
        <sum>0</sum>
        <value>0</value>
        <standardDeviation>0</standardDeviation>
      </metric-value>
    </metricValues>
  </metric-data>
</metric-datas>

[byoungman (Bill Youngman)]

So I've set my logging level to 'trace' and it appears that the xpath is being processed but the tag is not making it into my ES index--

xpath => ["/metric-datas/metric-data/metricId/text()","metricId"]

[Badger]

Running the filter you showed against the xml you showed gets me

  "metricId" => [
    [0] "733812"
]

on the event.

[byoungman (Bill Youngman)]

Thanks Badger - I'm new with working with this plugin so not understanding what I am doing wrong and how to fix it.

-Bill

[byoungman (Bill Youngman)]

Latest update --

I updated my xml filter to this --

xml {
	remove_namespaces => "true"
	source => "message"
	store_xml => "true"
	target => "doc"
	force_array => false
    xpath => ["//metricId/text()","metricId"]
}

I know see the xml attributes as below but still no 'metricId' field in ES

doc.metric-data.metricId
doc.metric-data.metricName
doc.metric-data.metricPath
doc.metric-data.metricValues.metric-value.count
etc...