Can a logstash filter error be forwarded to elastic?

[DavidMarcu]

I'm having these json parsing errors from time to time:

2022-01-07T12:15:19,872][WARN ][logstash.filters.json    ] Error parsing json
 {:source=>"message", :raw=>" { the invalid json }", :exception=>#<LogStash::Json::ParserError: Unrecognized character escape 'x' (code 120)

Is there a way to get the :exception field in the logstash config file?
Since what I want is to create a field containing that exception message, though I'm not sure if that's possible.

[hocho]

If there is a json parse failure the tags field is populated with "_jsonparsefailure". If there is a general exception field I do not know.

[DavidMarcu]

Yeah, that seems to be case. I wanted to add in Elasticsearch more details regarding as to why and where the json filter fails to parse. I think the only chance I got if I want this behaviour is to implement a raw ruby script in a ruby filter and try to manually parse it.

[Badger]

You can re-purpose the core of the json filter. The following code

    ruby {
        code => '
            @source = "message"
            source = event.get(@source)
            return unless source

            begin
                parsed = LogStash::Json.load(source)
            rescue => e
                event.set("jsonException", e.to_s)
                return
            end

            @target = "jsonData"
            if @target
                event.set(@target, parsed)
            end
        '
    }

results in

"jsonException" => "Unexpected character (',' (code 44)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{ \"baz\", \"oh!\" }\r\"; line: 1, column: 9]",

ETA: Come to think of it, I would remove the code that deals with @target, and use a json filter to do the actual parsing so that you have all the other options of it available. Just re-parse in ruby to capture the exception.

[DavidMarcu]

I was going to do something similar, but I think this solution is even more complete than what I had in mind.