Error logging: is it possible to customise mapper_parsing_exception logs?

Elastic Stack Logstash
1

I have a central index which many services write logs to.

However when I get unexpected data (e.g. a date instead of a boolean) the log looks like this:
[2019-07-02T11:36:32,836][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"my-services-write", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x35052f4b], :response=>{"index"=>{"_index"=>"my-services-000002", "_type"=>"_doc", "_id"=>"bjF4smsB1cGh2-nB2IK8", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [params.valueDate] of type [boolean] in document with id 'bjF4smsB1cGh2-nB2IK8'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Failed to parse value [2019-06-27T12:45:54.007+0000] as only [true] or [false] are allowed."}}}}}

Which is great, but doesn't tell me which service is logging the wrong data.

Is there a way to customise mapper_parsing_exceptions so they will show custom fields, i.e. the service name?

2

I don't think so, but you can send events that get a 400 response to a dead-letter queue and inspect them yourself to identify the source.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.