Mutate gsub json with backslash

(stash2logs) #1


I have the follwing json message and I'm looking to use the mutate gsub filter to do two things:
1 - remove the \n
2 - remove the \ (backslashes)

Ideally i would like to get the following format:

It seems like the mutate gsub is not working as expected on serialized json... Here is a snippet of my logstash.conf

          source => "message"
          target => "parsedJson"
       if [parsedJson][logger_name] ==  "com.test" {
              gsub => ["[parsedJson][message]", "[\n]", ""]
              gsub => ["[parsedJson][message]", "[\\]", ""]
           json {
             source => "message"
             target => "newmessage"

Here is the event output
"@version" => "1",
"path" => "/test/api-logstash.log",
"parsedJson" => {
"logger_name" => "com.test",
"level" => "INFO",
"@version" => 1,
"thread_name" => "dw-29",
"level_value" => 20000,
"@timestamp" => "2019-04-15T18:34:49.679-04:00",
"message" => "{"protocol":"HTTP/1.1","remote_addr":""}"
"host" => "myserver",
"@timestamp" => 2019-04-15T22:34:50.465Z

As you can see the newline is removed but the backslashes are still there. Any idea on how to achieve this?


In your second json filter you want to replace message with [parsedJson][message].

I was very surprised to find that your mutate works for me.

input { generator { count => 1 message => 'foo
bar\baz' } }

gets me

   "message" => "foo\nbar\\baz",

and if I add

filter {
        gsub => ["[message]", "[\n]", ""]
        gsub => ["[message]", "[\\]", ""]

that is reduced to

   "message" => "foobarbaz",

I would expect to have to use a literal newline in the character group

filter {
        gsub => ["[message]", "[\\
]", ""]

(stash2logs) #3

Did you try running that same filter on a test JSON? It does not work as expected.
Try it on this JSON


You are not using either markdown or block-quoting on the text you want parsed.

I cannot un-parse a parser. If you show us what a message looks like then someone will help you...

(stash2logs) #5

Here is what the message looks like:



Please either surround your example message with lines containing just


or else indent it using 4 spaces. Then tell us whether that is literally what the message looks like, or whether it is the rubydebug output of the message.