How to gsub \ in json message

I have a message below that is failing due to jsonparse error. If I remove all the \ (backslashes) from the message in text editor and check in json parser, it parsed it properly. Now, to get rid of the \ in the json message filed, I tried the mutate + gsub as below (from the official docs) with [\] option which is not working! Need some help with this:

config file:

filter {
    mutate {
      gsub => [ "message", "[\\]", "" ]  # << 2 backslashes
}
}
       [0] "_jsonparsefailure"
    ],
        "message" => [
        [ 0] "{\"node_id_str\":\"xxxx\",\"subscription_id_str\":\"Sub2\",\"encoding_path\":\"Cisco-IOS-XR-shellutil-oper:system-time/uptime\",\"collection_id\":\"151649\",\"collection_start_time\":\"1560440022382\",\"msg_timestamp\":\"1560440022396\",\"data_json\":[{\"timestamp\":\"1560440022394\",\"keys\":[],\"content\":{\"host-name\":\"xxxx\",\"uptime\":6117928}}],\"collection_end_time\":\"1560440022396\"}",

So, how do I get rid of the \ in this message? Interestingly, if I try to match it and replace with some char B, it does it next to the \ so not sure what this is matching....

filter {
   mutate {
     gsub => [ "message", "[\\]", "B" ] # << 2 back slashes
 }
}
      "message" => [
        [0] "{B\"node_id_strB\":B\"xxxx-IE1B\",B\"subscription_id_strB\":B\"Sub2B\",B\"encoding_pathB\":B\"Cisco-IOS-XR-shellutil-oper:system-time/uptimeB\",B\"collection_idB\":B\"151843B\",B\"collection_start_timeB\":B\"1560447583109B\",B\"msg_timestampB\":B\"1560447583123B\",B\"data_jsonB\":[{B\"timestampB\":B\"1560447583120B\",B\"keysB\":[],B\"contentB\":{B\"host-nameB\":B\"xxxx-IE1B\",B\"uptimeB\":6125489}}],B\"collection_end_timeB\":B\"1560447583123B\"}",
        [1] "{B\"node_id_strB\":B\"xxxx-IE1B\",B\"subscription_id_strB\":B\"Sub2B\",B\"encoding_pathB\":B\"Cisco-IOS-XR-shellutil-oper:system-time/uptimeB\",B\"collection_idB\":B\"151844B\",B\"collection_start_timeB\":B\"1560447613123B\",B\"msg_timestampB\":B\"1560447613136B\",B\"data_jsonB\":[{B\"timestampB\":B\"1560447613132B\",B\"keysB\":[],B\"contentB\":{B\"host-nameB\":B\"xxxx-IE1B\",B\"uptimeB\":6125519}}],B\"collection_end_timeB\":B\"1560447613136B\"}"

[EDIT: added code-fences around code blocks to increase readability -- @yaauie]

If that is from rubydebug your message does not contain backslash, it is just rubydebug escaping the double quotes. Not sure how your replacement with B could occur in that case.

:-). If that is from ruby debug output plugin, why does logstash fail to parse the json?

What does the json filter configuration look like?

filter {
mutate {
split => ["message", " " ]
}
}

filter {
mutate {
gsub => [ "message", '\"', '"' ]
}
}

Do you not have a json filter configured? logstash is not going to parse the JSON unless you tell it to.

I have tried codec json and json_lines hoping that it will parse the incoming lines as json. I even added json filter to the end of the config before output but no luck parsing the message to json format. This is what I see when I run logstash:

[WARN ] 2019-06-14 12:38:45.267 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>["{"node_id_str":"xxxx","subscription_id_str":"Sub2","encoding_path":"Cisco-IOS-XR-shellutil-oper:system-time/uptime","collection_id":"152429","collection_start_time":"1560515908973","msg_timestamp":"1560515908990","data_json":[{"timestamp":"1560515908987","keys":,"content":{"host-name":"xxxx","uptime":6193815}}],"collection_end_time":"1560515908990"}"], :exception=>java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO}
[WARN ] 2019-06-14 12:38:45.272 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>["{"node_id_str":"xxxx","subscription_id_str":"Sub2","encoding_path":"Cisco-IOS-XR-shellutil-oper:system-time/uptime","collection_id":"152429","collection_start_time":"1560515908973","msg_timestamp":"1560515908990","data_json":[{"timestamp":"1560515908987","keys":,"content":{"host-name":"xxxx","uptime":6193815}}],"collection_end_time":"1560515908990"}"], :exception=>java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO}
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"tags" => [
[0] "_jsonparsefailure"
],
"port" => 48700,
"@timestamp" => 2019-06-14T12:38:44.988Z,
"host" => "xxxx-v0.xxxx.com",
"@version" => "1",
"message" => [
[0] "{"node_id_str":"xxxx","subscription_id_str":"Sub2","encoding_path":"Cisco-IOS-XR-shellutil-oper:system-time/uptime","collection_id":"152429","collection_start_time":"1560515908973","msg_timestamp":"1560515908990","data_json":[{"timestamp":"1560515908987","keys":,"content":{"host-name":"xxxx","uptime":6193815}}],"collection_end_time":"1560515908990"}"
],
"type" => "jsonotcp"
}

How did you configure the json filter?

filter {
json {
source => "message"
target => "jsonmessage"
}
}

You used a mutate+split filter to convert message to an array, so that should be

json {
    source => "[message][0]"
    target => "jsonmessage"
}

That worked for the first element! How do I make it parse every element in the array?

{
"tags" => [
[0] "_jsonparsefailure"
],
"jsonmessage" => {
"collection_end_time" => "1560531678180",
"node_id_str" => "xxxx",
"collection_id" => "152579",
"encoding_path" => "Cisco-IOS-XR-shellutil-oper:system-time/uptime",
"subscription_id_str" => "Sub2",
"collection_start_time" => "1560531678163",
"msg_timestamp" => "1560531678180",
"data_json" => [
[0] {
"timestamp" => "1560531678176",
"content" => {
"host-name" => "xxxx",
"uptime" => 6209584
},
"keys" =>
}
]
},
"@timestamp" => 2019-06-14T17:01:25.510Z,
"@version" => "1",
"port" => 34850,
"message" => [
[0] "{"node_id_str":"xxxx","subscription_id_str":"Sub2","encoding_path":"Cisco-IOS-XR-shellutil-oper:system-time/uptime","collection_id":"152579","collection_start_time":"1560531678163","msg_timestamp":"1560531678180","data_json":[{"timestamp":"1560531678176","keys":,"content":{"host-name":"xxxx","uptime":6209584}}],"collection_end_time":"1560531678180"}"
],
"host" => "xxxx-v0.xxxx.com",
"type" => "jsonotcp"
}

I think you would have to use ruby for that.

Can you share the code to do so?