JSON data split up (_jsonparseerror) in logstash-7.2.0-1. Works fine in logstash-7.1.1-1

[sjaganna (Shubha)]

Hello!

I've been in the midst of working with the ELK stack for the first time. I'm currently sending data from Metricbeat -> Logstash -> an external script (output sent over TCP) -> Logstash -> Elasticsearch. However, when I upgrade the logstash version to logstash-7.2.0-1 from logstash-7.1.1-1, My JSON is split up and sent in multiple documents instead of a single document which naturally leads to a _jsonparseerror tag. For example:

An output with logstash-7.2.0-1 where the "message field" is half the message JSON and the "..." indicates the middle of the message. This shows up with a _jsonparsefailure tag:

[2019-07-15T18:17:29,435][ERROR][logstash.codecs.json ] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (String)"{"service": {"type": "system"}, "tags": "... \"ios\": 66"}

The output with logstash-7.1.1-1 is clean and is parsed properly by elasticsearch. The only difference between the two is the logstash version. Any ideas?

[sjabiulla (JB)]

Check this thread once.

Logs on two lines

Hi, I am trying to get logs whose are on two different lines : [2019-06-18T06:57:32.032Z] [org.kie.api.internal.utils.tydhiks] [main] [141] [INFO ] Adding Service org.drools.core.concurrent.EEEeeF In my logstash I tried : grok { patterns_dir => "/u01/app/elk-config/logstash/patterns" match => ["message", "(?m)\[(?<timestamp>%{TIMESTAMP_ISO8601})\]\[%{DATA:servlet}\]\[%{WORD:branch}\]\[%{DATA:PID}\]\[%{DATA:level}\]%{DATA:log_message}"] } but it does not take the sec…

[johank]

We are having the same problem.

• Using LogstashTcpSocketAppender from Java to log to our logstash

7.1.1 works fine
7.2.1 gives the error
7.3.0 also gives the error

• Discovered when initially wanting to upgrade our old logstash 5.6.0

[johank]

Possible solution.

switching from codec json => json_lines seems to solve it. (have only tested for 5 minutes so far)
Probably worked before because of different code used to convert tcp stream into messages.

[sjaganna (Shubha)]

This solution seemed to work for me as well! Thanks.
This would have been nice to document as a breaking change in documentation.

[johank]

We noticed it is actually mentioned in the documentation of the logstash logback encoder. But aside from that, what made it difficult to find is that the setting "json" almost works - I suppose "json" probably has a maximum object size it will read up/buffer to before splitting.