Problem indexing mapper_parsing_exception

[joshsmoore (Josh Moore)]

I am placing this in Logstash because I think the problem is in the log stash processing even though it is raised by Elasticsearch. This is the exception I am seeing:

central_logger_logstash.1.iz1mulhde4jx@logger-swarm1 | [2018-07-20T02:00:29,078][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-6.3.0-2018.07.20", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x66c1b75b], :response=>{"index"=>{"_index"=>"logstash-6.3.0-2018.07.20", "_type"=>"doc", "_id"=>"1mVqtWQBG03hsxOZvSTv", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [host] tried to parse field [host] as object, but found a concrete value"}}}}

Can anybody point me to how to log the actual document that is causing the problem so I can see where it is coming from?

[magnusbaeck (Magnus Bäck)]

If you google the error message ("object mapping for [host] tried to parse field [host] as object, but found a concrete value") you'll find previous posts about this topic.

Can anybody point me to how to log the actual document that is causing the problem so I can see where it is coming from?

If you enable the dead letter queue feature Logstash will save rejected events to a file. You can then use a dead_letter_queue input to read from that file and do whatever with the events. See Dead letter queues (DLQ) | Logstash Reference [8.11] | Elastic.

[joshsmoore (Josh Moore)]

That helped me fix it. Thanks.