Write data that have failed in Logstash filter transforms to file

Elastic Stack Logstash
1

My logstash config has a filter performing several transformations. But I see many records failing to get processed because of incorrect format/encoding in source data.
I want Logstash to write these failed inputs to a file, which I can further use to validate data loss and/or improve source data and logstash transforms.

How can I do this currently?

2

if you set log.level to debug or trace, Logstash write more information about failures to log file. Is it not enough for you?

3

But using debug I can't identify the ones that have failed. I want to view only the failed messages.

4

This post help you?

5

You need to share your logstash configuration, show what filters you are using.

Some parser filters, like grok, json, kv and csv will add a tag when it fails, other filters do that as well.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.