Json parse breaks with TCP input

Elastic Stack Logstash
1

I tried test.config with 'stdin' input which works fine, but when I change to 'tcp' input which I use with nxlog for other logs breaks...what I am missing?

Works:

input {
       stdin {
              codec => multiline {
              pattern => '(s+)?\{'
              negate => true
              what => "previous"
              max_lines => 100
             }
     }
}
filter{
      if [message] =~ /^\,$/ {
                            mutate { add_tag => ["bad_log_line"] }
                            drop {}
       }
       mutate { gsub => [ "message", "},$", "}" ] }
       json { source => "message" }
}
output { stdout { codec => rubydebug } }

Doesn't work:

input {
          tcp {
               port => "3529"
               tags => [ "tcpjson" , "3529" ]
               ssl_enable => true
               ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"
               ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
               ssl_verify => false
               type => "some_log"
               codec => multiline {
                                  pattern => '(s+)?\{'
                                  negate => true
                                  what => "previous"
                                  max_lines => 100
                                  }
                }
        }
filter{
      if [type] == "some_log" and "3529" in [tags] {
                                    if [message] =~ /^\,$/ {
                                                           mutate { add_tag => ["bad_log_line"] }
                                                           drop {}
                                     }
                                    mutate { gsub => [ "message", "},$", "}" ] }
                                    json { source => "message" }
      }
}
output { stdout { codec => rubydebug } }

log file:

,
{
"logger":"somelogline01",
"timestamp":"2017-03-31 09:22:28",
"level":"INFO",
"hostName":"XYZ-456",
"userId":"user01",
"message":"Modify timestamp =20170331082224.Z",
"throwable":""
},
{
"logger":"somelogline02",
"timestamp":"2017-03-31 09:22:29",
"level":"INFO",
"hostName":"ABC-123",
"userId":"user02",
"message":"-- CALL ctxt.search() ...",
"throwable":""
},
{
2

The problem in this case is probably that the tcp input doesn't split the input into lines like the stdin or file inputs do, i.e. what's handed to the multiline codec could potentially be a kilobyte of data containing many log events. You can use a lines codec to split those blobs into multiple events but codecs can't be chained so then you couldn't use the multiline codec.

Anyway, multiline processing should be done as close to the source as possible, i.e. in NXLog in this case. Doing it on the Logstash side would never be as reliable.

3

Thanks magnus. I use xm_multiline from nxlog input and able to receive a line event on logstash side.

 <Extension multiline>
      Module xm_multiline
      Headerline /^\s*\{$/
      EndLine /^\s*\}\,$/
 </Extension>

Now, my message looks like this...

{\r\n    \"logger\":\"some log line (line:417).logSystemInfo()\",\r\n    \"timestamp\":\"2017-07-10 20:15:10\",\r\n    \"level\":\"INFO\",\r\n    \"hostName\":\"host01\",\r\n    \"userId\":\"user01\",\r\n    \"message\":\"FqHostname [ host01 ]\",\r\n    \"throwable\":\"\"\r\n  },\r

I am able to gsub \r and \n using...

mutate {gsub => ['message', "\\r", ""] }
mutate {gsub => ['message', "\\n", ""] }

but, gsub doeesn't work with when try to replace ' " ' with ' " '

mutate {gsub => ['message', '\\"', ''] }

json filter fails with current message format, so I have to change it someway.

4

The json filter shouldn't have a problem with the \r and \n characters, but the trailing comma is a problem.

5

I have taken care trailing comma with gsub. now, message looks like this but still json parse doesn't work.

   "  {    \"logger\":\"some long line(line:417).logSystemInfo()\",    \"timestamp\":\"2017-07-10 20:15:10\",    \"level\":\"INFO\",    \"hostName\":\"host01\",    \"userId\":\"user01\",    \"message\":\"FqHostname [ host01 ]\",    \"throwable\":\"\"  }"
6

Please show the exact error message from the Logstash log.

Logstash has no problems with that exact string (if I remove the outer double-quotes and unescape the rest of the string).

$ cat data 
  {    "logger":"some long line(line:417).logSystemInfo()", "timestamp":"2017-07-10 20:15:10",    "level":"INFO", "hostName":"host01",    "userId":"user01", "message":"FqHostname [ host01 ]",    "throwable":""  }
$ cat test.config 
input { stdin { codec => json } }
output { stdout { codec => rubydebug } }
$ logstash -f test.config < data 
Settings: Default pipeline workers: 8
Pipeline main started
{
        "logger" => "some long line(line:417).logSystemInfo()",
     "timestamp" => "2017-07-10 20:15:10",
         "level" => "INFO",
      "hostName" => "host01",
        "userId" => "user01",
       "message" => "FqHostname [ host01 ]",
     "throwable" => "",
      "@version" => "1",
    "@timestamp" => "2017-07-13T19:27:46.092Z",
          "host" => "bertie"
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}
7

How to unescape the string in logstash filter? that is the problem here. json filter is expecting data in { "x": "y" , "a": "b" } format whereas now, my data is coming from nxlog in this format { "x": "y" , "a": "b" }

8

Let's see exactly what your events look like. Please use a stdout { codec => rubydebug } output and show the results of an example event.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.