Multiline json over TCP is not parsed

Elastic Stack Logstash
1

I have the following multiline json:
"
{"businessId":"335","technicId":"41180883","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0664","sursa":"test.mbv","nr_linie":"391","mesaj":"MESSAGE PUT TIME IS 16033104"}
{"businessId":"335","technicId":"41180884","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0665","sursa":"test.mbk","nr_linie":"392","mesaj":"MESSAGE PUT DATE IS 20200902"}
{"businessId":"335","technicId":"11808853","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0665","sursa":"test.mbx","nr_linie":"401","mesaj":"ended with reason code 000000000 "}
"
and the following logstash conf:

input {
tcp {
port => 9601
codec => multiline {
pattern => "^{"
negate => true
what => previous
}
}
}

filter {
json {
source => "message"
}
}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "next_index"
}
stdout {
codec => rubydebug
}
}

and I get the following error:

[WARN ] 2020-09-22 05:42:17.480 [[main]>worker7] json - Error parsing json {:source=>"message", :raw=>"52 <13>1 2020-09-22T05:41:45+03:00 bndikdev}, - - - \n263 <13>1 2020-09-22T05:41:45+03:00 bndikdev{"businessId" - - - "335","technicId":"41180883","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0897","sursa":"opnAAmq1.cbl","nr_linie":"940","mesaj":"TECHNICID = 41180883 "},\n232 <13>1 2020-09-22T05:41:45+03:00 bndikdev{"businessId" - - - "335","technicId":"41180883","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0897","sursa":"opnAAmq1.cbl","nr_linie":"941","mesaj":"MSGBRCH = 1000"},\n259 <13>1 2020-09-22T05:41:45+03:00 bndikdev {"businessId" - - - "335","technicId":"41180884","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0897","sursa":"opnAAmq1.cbl","nr_linie":"942","mesaj":"MSGTYPE = OPNAA "}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (byte)"52 <13>1 2020-09-22T05:41:45+03:00 bndikdev}, - - -
263 <13>1 2020-09-22T05:41:45+03:00 bndikdev{"businessId" - - - "335","technicId":"41180884","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0897","sursa":"opnAAmq1.cbl","nr_linie":"940","mesaj":"TECHNICID = 11808853 "},
232 <13>1 2020-09-22T05:41:45+03:00 bndikdev{"businessId" - - - "335","technicId":"11808853","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0897","sursa":""[truncated 320 bytes]; line: 1, column: 5]>}

I also tried to parse the multiline json using the following conf:
input {
tcp {
port => 9601
codec => plain
}
}

filter {
json {
source => "message"
}
}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "next_index"
}
stdout {
codec => rubydebug
}
}
and I still get json parse failure:
[WARN ] 2020-09-22 03:59:44.495 [[main]>worker0] json - Error parsing json {:source=>"message", :raw=>"250 <13>1 2020-09-22T03:59:44+03:00 bndikdev {"businessId" - - - "335","technicId":"641180883","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0856","sursa":"opnAAmq1.cbl","nr_linie":"370","mesaj":"alternative nespecificate",", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (byte)"250 <13>1 2020-09-22T03:59:44+03:00 bndikdev {"businessId" - - - "335","technicId":"641180883","teller":"progr_1 ","pid":"0024444930","timestamp":"16:03:31.0856","sursa":"opnAAmq1.cbl","nr_linie":"370","mesaj":" alternative nespecificate","; line: 1, column: 6]>}

2

Your data does not appear to have the format you think it has. It has syslog headers prepended to it. You will need to remove those.

3

It’s true because I receive the data from an AIX 7.1 that uses syslog-ng to send logs to the ELK server. How can i remove those headers ?

4

Use dissect

dissect { mapping => { "message" => "%{someNumber} <%{syslogPri}> %{timestamp} %{hostname} %{jsonData}" } }

then pass [jsonData] to the json filter.

5

As instructed I added dissect:
filter {

dissect {
mapping => {
"message" => "%{someNumber} <%{syslogPri}>1 %{timestamp} %{hostname} %{jsonData}"
}
}

json {
source => "jsonData"
}

mutate {
remove_field => [ "someNumber" ]
}
}
I used the following json : {"messageType": "cagw","uniqueID": "8e760b6b7c4e937ebb7063746dd945ecee782cac","sessionID": "6c43b719e0e457784a0a6d5dfef96a82"}
And I checked it with https://jsonlint.com/ and it was correct
Now I get a json parse failure:
[WARN ] 2020-09-23 21:00:08.244 [[main]>worker2] json - Error parsing json {:source=>"jsonData", :raw=>"{"messageType" - - - "cagw","uniqueID": "8e760b6b7c4e937ebb7063746dd945ecee782cac","sessionID": "6c43b719e0e457784a0a6d5dfef96a82"}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): was expecting a colon to separate field name and value
at [Source: (byte)"{"messageType" - - - "cagw","uniqueID": "8e760b6b7c4e937ebb7063746dd945ecee782cac","sessionID": "6c43b719e0e457784a0a6d5dfef96a82"}"; line: 1, column: 17]>}
{
"message" => "174 <13>1 2020-09-23T21:00:08+03:00 andibkdev {"messageType" - - - "cagw","uniqueID": "8e760b6b7c4e937ebb7063746dd945ecee782cac","sessionID": "6c43b719e0e457784a0a6d5dfef96a82"}",
"jsonData" => "{"messageType" - - - "cagw","uniqueID": "8e760b6b7c4e937ebb7063746dd945ecee782cac","sessionID": "6c43b719e0e457784a0a6d5dfef96a82"}",
"hostname" => "andibkdev",
"timestamp" => "2020-09-23T21:00:08+03:00",
"@timestamp" => 2020-09-23T18:00:08.141Z,
"host" => "10.188.208.34",
"port" => 36362,
"tags" => [
[0] "_jsonparsefailure"
],
"@version" => "1",
"syslogPri" => "13"
}

6

I see that the value of the json gets in elastic with "---" although it's not like that in the original json

7

Looking at your first post I see you have the same mutation in the JSON there. I have no suggestions on what could be causing that.

If you are unable to hunt it down I guess you could

mutate { gsub => [ "jsonData", " - - -", ":" ] }
8

Hi Badger,

Thank you for you advices, my issues got fixed.

Cheers!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.