I am capturing logs that primarily look like this:
[2020-10-21 22:12:44,067][INFO][audit]{"JSON formatted stuff"}
However, I will occasionally get lines that look like this:
[2020-10-21 22:12:44,067][INFO][audit]text goes here with occasional {"JSON stuff after text"}
I'm trying to use logic and mutate to wrap the non-json formatted events into JSON, but it logstash throws an error when it gets to the JSON filter and the lines that are supposed to match, don't appear to be. It appears that my regular expression or logic format is failing, but I'm not sure why. What am I doing wrong?
if "message" =~ /^\[.*?\]\[.*?\]\[.*?\]\w/ {
mutate {
gsub => [
"message", '(\[.*?\]\[.*?\]\[.*?\])(.*)', '\1{"info": "\2"}'
]
}
}
Error in logstash-plain.log:
[2020-10-22T12:25:11,758][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"text goes here with occasional {"JSON stuff after text"}", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'text': was expecting ('true', 'false' or 'null') at [Source: (byte[])"text goes here with occasional {"JSON stuff after text"}"; line: 1, column: 6]>}