Logstash XML Parser

Elastic Stack Logstash
1

I'm parsing a lot of xml files using xml filter.
But for some files I'm getting errors like

"Can't merge a non object mapping [doc.Body.EnumerateResponse.Items.DCIM_BaseMetricValue.Duration] with an object mapping"`

and

"error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper [doc.Body.EnumerateResponse.Items.DCIM_PSNumericSensor.SettableThresholds] of different type, current_type [text], merged_type [ObjectMapper]"}}},

so how can i ignore these wrongly formatted xml tags and put the rest of the parsed data in ES.

2

This isn't a problem with the XML parser but with the mappings in Elasticsearch. A given field in an index needs to have exactly one type but in your example the same field has initially been mapped as a text field and now you're trying to pass a document where the same field is an object.

I don't think there's a way to drop the offending field and insert the rest of the document.

3

Thank you Magnus.
I found a solution, i ended up using exclude_lines in filebeat to skip the xml fields having more than one type.

<n1:BuildNumber>0</n1:BuildNumber>

<n1:BuildNumber xsi:nil="true"/>

these fields were causing the problem ,so in filebeat i used exclude_line to remove any line having

xsi:nil="true"

This did the trick.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.