JSON data split up (_jsonparseerror) in logstash-7.2.0-1. Works fine in logstash-7.1.1-1

Hello!

I've been in the midst of working with the ELK stack for the first time. I'm currently sending data from Metricbeat -> Logstash -> an external script (output sent over TCP) -> Logstash -> Elasticsearch. However, when I upgrade the logstash version to logstash-7.2.0-1 from logstash-7.1.1-1, My JSON is split up and sent in multiple documents instead of a single document which naturally leads to a _jsonparseerror tag. For example:

An output with logstash-7.2.0-1 where the "message field" is half the message JSON and the "..." indicates the middle of the message. This shows up with a _jsonparsefailure tag:

[2019-07-15T18:17:29,435][ERROR][logstash.codecs.json ] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (String)"{"service": {"type": "system"}, "tags": "... \"ios\": 66"}

The output with logstash-7.1.1-1 is clean and is parsed properly by elasticsearch. The only difference between the two is the logstash version. Any ideas?

Check this thread once.

We are having the same problem.

  • Using LogstashTcpSocketAppender from Java to log to our logstash

7.1.1 works fine
7.2.1 gives the error
7.3.0 also gives the error

  • Discovered when initially wanting to upgrade our old logstash 5.6.0

Possible solution.

switching from codec json => json_lines seems to solve it. (have only tested for 5 minutes so far)
Probably worked before because of different code used to convert tcp stream into messages.

This solution seemed to work for me as well! Thanks.
This would have been nice to document as a breaking change in documentation.

We noticed it is actually mentioned in the documentation of the logstash logback encoder. But aside from that, what made it difficult to find is that the setting "json" almost works :slight_smile: - I suppose "json" probably has a maximum object size it will read up/buffer to before splitting.

1 Like