Mutate gsub json with backslash

Elastic Stack Logstash
1

{\n"protocol":"HTTP/1.1",\n"remote_addr":"127.0.0.1"}

I have the follwing json message and I'm looking to use the mutate gsub filter to do two things:
1 - remove the \n
2 - remove the \ (backslashes)

Ideally i would like to get the following format:
{"protocol":"HTTP/1.1","remote_addr":"127.0.0.1"}

It seems like the mutate gsub is not working as expected on serialized json... Here is a snippet of my logstash.conf

json{
          source => "message"
          target => "parsedJson"
          remove_field=>["message"]
       }
       if [parsedJson][logger_name] ==  "com.test" {
           mutate{
              gsub => ["[parsedJson][message]", "[\n]", ""]
              gsub => ["[parsedJson][message]", "[\\]", ""]
           }
           json {
             source => "message"
             target => "newmessage"
           }
       }
}

Here is the event output
{
"@version" => "1",
"path" => "/test/api-logstash.log",
"parsedJson" => {
"logger_name" => "com.test",
"level" => "INFO",
"@version" => 1,
"thread_name" => "dw-29",
"level_value" => 20000,
"@timestamp" => "2019-04-15T18:34:49.679-04:00",
"message" => "{"protocol":"HTTP/1.1","remote_addr":"127.0.0.1"}"
},
"host" => "myserver",
"@timestamp" => 2019-04-15T22:34:50.465Z
}

As you can see the newline is removed but the backslashes are still there. Any idea on how to achieve this?

2

In your second json filter you want to replace message with [parsedJson][message].

I was very surprised to find that your mutate works for me.

input { generator { count => 1 message => 'foo
bar\baz' } }

gets me

   "message" => "foo\nbar\\baz",

and if I add

filter {
    mutate{
        gsub => ["[message]", "[\n]", ""]
        gsub => ["[message]", "[\\]", ""]
    }
}

that is reduced to

   "message" => "foobarbaz",

I would expect to have to use a literal newline in the character group

filter {
    mutate{
        gsub => ["[message]", "[\\
]", ""]
    }
}
3

Did you try running that same filter on a test JSON? It does not work as expected.
Try it on this JSON
{\n"protocol":"HTTP/1.1",\n"remote_addr":"127.0.0.1"}

4

You are not using either markdown or block-quoting on the text you want parsed.

I cannot un-parse a parser. If you show us what a message looks like then someone will help you...

5

Here is what the message looks like:

{\n\"protocol\":\"HTTP/1.1\",\n\"remote_addr\":\"127.0.0.1\"}

6

Please either surround your example message with lines containing just

```

or else indent it using 4 spaces. Then tell us whether that is literally what the message looks like, or whether it is the rubydebug output of the message.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.