# Error indexing using xml plugin
**URL:** https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774
**Category:** Logstash
**Created:** [March 16, 2020, 1:54pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774 "2020-03-16T13:54:47Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 1:54pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/1 "2020-03-16T13:54:48Z")
Hi,
logstash 6.8.3
ES 6.3
I am getting constant errors in my logstash log about mapping an xml field:
Mar 16 13:09:04 ip-10-152-3-140 logstash[16367]: [2020-03-16T13:09:04,614][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"filebeat-xxx-2020.12", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x2297c725], :response=\>{"index"=\>{"\_index"=\>"filebeat-xxx-2020.12", "\_type"=\>"doc", "\_id"=\>"pjZ243AB5TP9fEZrR9XM", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for [xml\_content.Request.TranslatedMessage] tried to parse field [null] as object, but found a concrete value"}}}}
I looked at the mapping for this index:
When I checked another index mapping I can see it is set as this:
"xml\_content": {
"properties": {
"AgentID": {
"Duration": {
"EndpointAddress": {
"ErrorDetails": {
"MessageName": {
"Request": {
"RawMessage": {
"Time": {
"TranslatedMessage": {
"type": "keyword",
"ignore\_above": 1024
}
}
},
We are parsing api logs the contain information each user is exchanging so I was using dynamic templates because the format of the messages can change between calls. I am wondering if the first record into the index is creating the mapping and then subsequent records that don't hit the same format are being rejected?
I am using the xml plugin because there are some fields we want to capture and work with in logstash like:
xml {
source =\> "message"
target =\> "xml\_content"
}
grok {
match =\> { "xml\_content[Duration]" =\> "%{HOUR:duration\_hours}:%{HOUR:duration\_mins}:%{HOUR:duration\_secs}%{GREEDYDATA:duration\_milli}" }
}
The only reason I really noticed all these errors is because users were complaining not all the records seemed to be making it through to ES. I am stuck as how to be able to get ALL the information in to ES now. Help?...🙂
Fiona
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 16, 2020, 5:15pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/2 "2020-03-16T17:15:17Z")
> [@fionamacd](#):
>
> "object mapping for [xml\_content.Request.TranslatedMessage] tried to parse field [null] as object, but found a concrete value"
Your mapping says that [xml\_content][Request][TranslatedMessage] should be an object containing multiple fields. In elasticsearch a field can be either an object or a value, it cannot be an object in some documents and a value in others. So any events where [xml\_content][Request][TranslatedMessage] is a concrete value rather than an object will get rejected. The solution is to detect that it is a value and rename it so that the value is field within it instead.
---
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 5:59pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/3 "2020-03-16T17:59:56Z")
Thanks for that! I don't have any good examples to test with, I am wondering if I can just drop that whole field in Logstash as I don't think we use the contents anywhere. Support just want to get all messages in Kibana so they can perform searches on the original message field anyway.
Do you think I can drop the field?
---
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 6:15pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/4 "2020-03-16T18:15:50Z")
I tried adding:
mutate {
remove\_field =\> ["xml\_content.Response.TranslatedMessage", "Request.TranslatedMessage", "TranslatedMessage"]
}
But I still see the errors coming out. Don't understand why it is still trying to map the field if told it to remove it?
---
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 6:29pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/5 "2020-03-16T18:29:36Z")
My logstash code looks like:
```auto
} else if "incoming" in [tags] {
mutate {
add_tag => "integration_incoming"
add_field => { "cust_name" => "%{[fields][customer]}" }
add_field => { "Index_Type" => "%{[@metadata][beat]}" }
}
xml {
source => "message"
target => "xml_content"
suppress_empty => true
}
mutate {
remove_field => ["xml_content.Response.TranslatedMessage", "Request.TranslatedMessage", "TranslatedMessage"]
}
grok {
match => { "xml_content[Duration]" => "%{HOUR:duration_hours}:%{HOUR:duration_mins}:%{HOUR:duration_secs}%{GREEDYDATA:duration_milli}" }
}
```
I keep the mutate out of the xml filter, but it is still going through.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 16, 2020, 6:40pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/6 "2020-03-16T18:40:44Z")
In elasticsearch a nested field is referenced using periods in the name, such as xml\_content.Request.TranslatedMessage, whilst in logstash that would be referred to as [xml\_content][Request][TranslatedMessage]
---
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 6:55pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/7 "2020-03-16T18:55:10Z")
Hah thanks again!
I changed logstash thus:
```auto
xml {
source => "message"
target => "xml_content"
suppress_empty => true
}
mutate {
remove_field => ["[xml_content][Response][TranslatedMessage]", "[xml_content][Request][TranslatedMessage]" ]
}
```
But after restarting logstash I am still seeing these errors:
```auto
Mar 16 18:52:09 ip-10-152-3-140 logstash[17984]: [2020-03-16T18:52:09,406][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-xxx-2020.12", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"filebeat-xxx-2020.12", "_type"=>"doc", "_id"=>"jMKw5HAB5TP9fEZrYU0j", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [xml_content.Request.TranslatedMessage] tried to parse field [null] as object, but found a concrete value"}}}}
Mar 16 18:52:09 ip-10-152-3-140 logstash[17984]: [2020-03-16T18:52:09,407][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-xxx-2020.12", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"filebeat-xxx-2020.12", "_type"=>"doc", "_id"=>"nMKw5HAB5TP9fEZrYU0j", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [xml_content.Request.TranslatedMessage] tried to parse field [null] as object, but found a concrete value"}}}}
```
I thought removing the field would fix the issue, but it doesn't seem to be making a difference.
---
### Author:  [@fionamacd](https://discuss.elastic.co/u/fionamacd)
#### Post date: [March 16, 2020, 7:49pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/8 "2020-03-16T19:49:45Z")
Oh I think I fixed it! Bloody hell, I amaze myself when I get something working half the time 🙂
Changed the line to remove the whole Response/Request fields and that seems to have stopped the errors. The data is still part of the message so the user can still search for it.
```auto
mutate {
remove_field => ["[xml_content][Response][TranslatedMessage]", "[xml_content][Request][TranslatedMessage]", "[xml_content][Request]", "[xml_content][Response]" ]
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 13, 2020, 7:49pm UTC](https://discuss.elastic.co/t/error-indexing-using-xml-plugin/223774/9 "2020-04-13T19:49:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.