Trapping mapper_parsing_exception original messages

Elastic Stack Logstash
1

Hi there. I am debugging a Logstash filter pipeline that occasionally throws an Elasticsearch indexing error.

Long story short is that I am trying to parse a log message from an application into key-value pairs using a Ruby filter. Sometimes the log message is broken into key/value pairs incorrectly, and a string value is returned for a field that should be a Long.

I don't need to go into too much detail; I'm just looking for a way to make Logstash trap the mapper_parsing_exception and save the original event data to another index or to a Kafka topic or a file... really, to anything that lets me examine the original event and try to work backward to find my bug.

Is there such a mechanism in Logstash? I'm currently using version 6.7.1.

Thanks!

Jeff Kirk

2

Are you aware of dead letter queues?

3

Hi Badger,

That looks perfect! Have never used that feature up to now. Thanks!

Jeff

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.